Layerwise universal adversarial attack on NLP models (2023)

Abstract: In this work, we examine the vulnerability of language models to universal adversarial triggers (UATs). We propose a new white-box approach to the construction of layerwise UATs (LUATs), which searches the triggers by perturbing hidden layers of a network. On the example of three transformer models and three datasets from the GLUE benchmark, we demonstrate that our method provides better transferability in a model-to-model setting with an average gain of 9.3% in the fooling rate over the baseline. Moreover, we investigate triggers transferability in the task-to-task setting. Using small subsets from the datasets similar to the target tasks for choosing a perturbed layer, we show that LUATs are more efficient than vanilla UATs by 7.1% in the fooling rate.

## introduction one of the fundamental drawbacks of modern neural networks is their vulnerability to adversarial attacks @xcite , imperceptible perturbations to the data samples that leave the ground truth label unchanged but are able to modify model prediction drastically. the samples obtained as the result of these perturbations are called adversarial examples. first discovered for image datasets @xcite , this phenomenon was then demonstrated for other types of data, including natural language @xcite @xcite . the originally proposed methods of adversarial attack construction were sample-dependent, which means that one can not apply the same perturbation to different dataset items and expect equal success. sample-agnostic, or in other words, universal, adversarial perturbations (uaps) were proposed in @xcite , where based on a small subset of image data, the authors constructed perturbations leading to prediction change of 80-90% (depending on the model) of samples. they also showed that uaps discovered on one model could successfully fool another. the generalization of the universal attacks to natural language data was made by @xcite . short additives (triggers) were inserted at the beginning of data samples, and then a search over the token space, in order to maximize the probability of the negative class on the chosen data subset, was performed. the found triggers turned out to be very efficient at fooling the model, repeating the success of uaps proposed for images. the conventional way to look for adversarial examples is to perturb the output of a model. considering image classification neural networks, @xcite proposed to search for perturbations to hidden layers by approximating the so-called (p, q)-singular vectors @xcite of the corresponding jacobian matrix. then one can hope that the error will propagate through the whole network end, resulting in model prediction change. they showed that this approach allows obtaining a high fooling rate based on significantly smaller data subsets than those leveraged by @xcite . in this paper, we aim to continue the investigation of neural networks' vulnerability to universal adversarial attacks in the case of natural language data. inspired by the approach considered by @xcite , we look for the perturbations to hidden layers of a model instead of the loss function. in order to avoid projection from embedding to discrete space, we use simplex parametrization of the search space (see, e.g. @xcite ). we formulate the corresponding optimization problem and propose the algorithm for obtaining its approximate solution. on the example of three transformer models: bert @xcite , roberta @xcite , albert @xcite and three glue datasets @xcite we demonstrate higher efficiency of our method over the original approach of @xcite in the setting of modelto-model and task-to-task transfer. we also show that in the case of direct attack application, our method demonstrates the results which are on par with the baseline, where perturbation of the loss function was realized. we hope that this technique will serve as a useful tool for discovering flaws and deepening our understanding of language neural networks. ## framework let f : @xmath0, y i } n @xmath1, where x i is an input and y i is a corresponding label. our goal is to find a small perturbation to the original input sequences that leads to a change in a model prediction for the maximum possible number of samples. such perturbation could be modelled as an insertion of a trigger, a token sequence t of small length l, into a particular part of an input. following @xcite , we call such text perturbations universal adversarial triggers (uats). in this paper, we focus on triggers concatenated to the front of a sentence and denote a corrupted sample as @xmath2. it is also important to note that, in contrast to @xcite , we consider the unsupervised scenario when an attacker does not have access to the labels. for evaluation we follow @xcite and use the fooling rate (fr): universal adversarial triggers. before we dive into the description of our approach, it is worth depicting the original method of uats. restricting themselves to the white-box setting, @xcite showed that an efficient way to find uats could be performed through the optimization of the first-order expected loss approximation: in order to find the optimal perturbation within each iteration, the expectation in (2) is relaxed with the average over a batch, and maximization is performed for each trigger token independently: our approach: layerwise universal adversarial triggers (luats). in (3), we face the optimization problem over a discrete set of tokens from the vocabulary v. to overcome this issue, let us relax (3) using the probability simplex model for every token. in this case, a trigger can be represented as @xmath3, where v is a vocabulary matrix and w mn is a probability of the n-th token from vocabulary to be selected at position m. then (3) can be rewritten as follows: our approach can be seen as an extension of (4) to the perturbation of hidden layers. this is inspired by @xcite , who applied this idea to find uaps for fooling image classification neural networks. given a layer l, the optimization problem, in this case, can be similarly obtained via the taylor expansion: in contrast to (3), finding the optimal solution of (7) is computationally infeasible. indeed, the problem (3) allows a brute-force approach for finding the optimal token for each trigger position since it requires computing the gradient only once for each iteration. on the other hand, in our case, a bruteforce computation is very cumbersome since, for each iteration, it would require computing the jacobian action for every batch, token candidate and position, resulting in o(lb|v|) forward-backward passes, where b is the number of batches in an iteration. luckily, f l (w ) is convex and can be lower bounded via a tangent line, where the gradient is calculated as follows: concerning the initialization of w * , we take the uniform distribution over all the vocabulary tokens for each token position in a trigger. within each iteration, we perform only one step with respect to w in order to reduce computation time and observe that it is sufficient for breaking the models efficiently. finally, since the found after a given iteration weight matrix w in the worst case has only one non-zero element per row (see appendix a), we can get into a local maximum unless we guess the proper initialization. therefore, similarly to wallace et al. (2019) , we perform a beam search over the top-k candidates. in order to realize it, it is necessary to define the ranking criterion for choosing the best option at each search step. for this purpose, we use the fr. the overall algorithm is presented in algorithm 1 and ## experiments in this section, we present a numerical study of the proposed layerwise adversarial attack framework on text classification datasets. the code is publicly available on github 1 . ## related work there are already quite a few works devoted to the universal attack on nlp models in the literature. we briefly discuss them here. ribeiro et al. (2018) proposed a sample-agnostic approach to generate adversarial examples in the case of nlp models by applying a semanticspreserving set of rules consisting of specific word substitutions. however, found rules were not completely universal, resulting in model prediction change only for 1% -4% out of targeted samples. similarly to @xcite , the realization of text adversarial attacks as short insertions to the input were proposed by @xcite . though, they did not perform the search over the whole vocabulary for each word position in the trigger but instead exploited cosine-similarity projected gradient descent, which does not appear that efficient in the sense of attack performance. adversarial triggers generated by the method proposed by @xcite in general turned out to be semantically meaningless, which makes them easier to detect by defence systems. an attempt to make triggers more natural was undertaken by @xcite . leveraging an adversarially regularized autoencoder @xcite to generate the triggers, they managed to improve their semantic meaning without significantly decreasing the attack efficiency. another interesting direction is to minimize the amount of data needed for finding uaps. singla et al. (2022) created representatives of each class by minimizing the loss function linear approximation over the text sequences of a certain size. afterwards, adversarial triggers were appended to these class representatives and the rest of the procedure followed @xcite . although no data was used explicitly for training the attack, this approach demonstrated solid performance on considered datasets. ## conclusion we present a new layerwise framework for the construction of universal adversarial attacks on nlp models. following @xcite , we look for them in the form of triggers; however, in order to find the best attack, we do not perturb the loss function but neural network intermediate layers. we show that our approach demonstrates better performance when the triggers are transferred to different models and tasks. the latter might be related to the fact that in order to be transferred successfully between different datasets, a trigger should reflect network flaws that are task-independent. in this case, reducing the attack search to perturbation of the lower or middle layers might be more beneficial since the higher layers are highly influenced by fine-tuning. we hope this method will serve as a good tool for investigating the shortcomings of language models and improving our understanding of neural networks' intrinsic mechanisms. we would like to conclude by discussing the potential risks. as any type of technology, machine learning methods can be used for good and evil. in particular, adversarial attacks can be used for misleading released machine learning models. nevertheless, we think that revealing the weaknesses of modern neural networks is very important for making them more secure in the future and also for being able to make conscious decisions when deploying them.