Exploring the Vulnerability of Natural Language Processing Models via Universal Adversarial Texts (2021)

Abstract: Universal adversarial texts (UATs) refer to short pieces of text units that can largely affect the predictions of NLP models. Recent studies on universal adversarial attacks assume the accessibility of datasets for the task, which is not realistic. We propose two types of Data-Free Adjusted Gradient (DFAG) attacks to show that it is possible to generate effective UATs with only one arbitrary example which could be manually crafted. Based on the proposed DFAG attacks, this paper explores the vulnerability of commonly used NLP models in terms of two factors: network architectures and pre-trained embeddings. Our empirical studies on three text classification datasets reveal that: 1) CNN based models are more extremely vulnerable to UATs while self-attention models show the most robustness, 2) the vulnerability of CNN and LSTM models and robustness of self-attention models could be attributed to whether they rely on training data artifacts for their predictions, and 3) the pre-trained embeddings could expose vulnerability to both universal adversarial attack and the UAT transfer attack.

## introduction deep neural networks (dnns) have enabled significant advancement in a range of natural language processing (nlp) applications such as sentiment analysis @xcite and topic classification @xcite . despite the superior performance, dnns are known to be vulnerable to adversarial perturbations @xcite @xcite @xcite , i.e., small changes on the input could lead to entirely incorrect predictions @xcite . it has raised practical security concerns for the deployment of dnns in safety-critical scenarios @xcite . adver-sarially perturbed inputs are known as adversarial examples and the process of generating adversarial examples is known as adversarial attack. it has become a common practice to examine the vulnerability of dnns to adversarial examples and mitigate the vulnerability by involving adversarial examples during the training process as a type of augmented data @xcite @xcite @xcite . most adversarial attack methods for nlp models @xcite @xcite are sample-wise methods that craft adversarial examples by manipulating each clean example. different from sample-wise attacks, universal adversarial attack @xcite aims to generate universal adversarial texts (uats) or universal triggers @xcite for each class or the entire dataset to fool nlp models. however, existing methods @xcite @xcite all require the validation/test dataset of the task or some proxy datasets in a similar domain to craft uats. to more easily and efficiently generate uats, we propose data-free adjusted gradient (dfag) attacks. according to the evaluation, our proposed dfag attacks achieve a comparable performance as the original linear approximation method @xcite on most of the nlp models. we find that uats generated by our method highly overlap with those from the original linear approximation method @xcite . this indicates that the vulnerability of uats may be inherent in the models. to better understand the vulnerability, we take text classification as an example and dive into different neural network architectures. empirical results show that cnn and lstm models are notebly more vulnerable to uats than self-attention models. we also reveal that the effectiveness of uats generated for lstm and cnn models exposes certain training data artifacts, i.e., important words in the training data that are more closely correlated with the targeted class. in contrast, self-attention models are relatively more robust to uats. this finding is consistent with previous study on model robustness to training data artifacts, so it is likely that self-attention models suffer less from training data artifacts. apart from the neural architectures, we also examine pre-trained embeddings, including static pre-trained word embeddings @xcite and contextualized ones from the pre-trained language model bert @xcite . these embeddings have been widely used in different nlp applications. our experiments show that pre-trained word embeddings could deteriorate model robustness to uats, and even self-attention models can become vulnerable with pre-trained embeddings. upon further investigation, we find that uats are often transferable among models that use the same pre-trained embeddings. this reveals one unique vulnerability of nlp models to uats. ## generating universal adversarial texts problem formulation. consider a text classifier f mapping from input x to label y. the goal of universal adversarial attack is to generate a small sequence of tokens @xmath0, t 2 , ..., t k ) (i.e., an uat), which can be inserted into any clean example x to cause misclassification towards a targeted wrong label ỹ. previous work @xcite showed the effectiveness of uat when three words are inserted at the beginning of the input sequence. here, we follow their settings and predetermine the adversarial target class ỹ. the attack problem can be formally definedd as: for any clean example {(x, y)|(x, y) ∈ d and @xmath1, we aim to make the classifier f predict the perturbed example t ⊕ x as the targeted label ỹ, i.e., f (t ⊕ x) = ỹ. the problem can be solved by minimizing an adversarial loss l adv (t ⊕ x, ỹ), which is the cross-entropy loss defined with the targeted label. the approximation scores for all the possible substitution words in the vocabulary can be efficiently calculated via matrix multiplication, where @xmath2. it only needs one forward and backward pass to compute the gradients for all the positions of uat tokens. the equation is shown below where ∇ et l adv has the dimensions for positions of uat tokens and embedding size m. both approaches require batches of data to update the uat t. however, we can still use the linear approximation approach as a baseline for our experiment due to its efficiency. this approach requires a batch of examples to calculate the gradient for each update of the uat, as shown in equation (4) where n examples are consumed. ## conclusion in this work, we investigated the vulnerability of natural language processing could make uats transferable across different nlp tasks. moreover, our result should also be verified on large-scale models. more detailed analyses of different filter sizes and attention heads are also interesting future works.