Bridge the Gap Between CV and NLP! A Gradient-based Textual Adversarial Attack Framework
Year: 2023

Abstract:
Despite recent success on various tasks, deep learning techniques still perform poorly on adversarial examples with small perturbations. While optimization-based methods for adversarial attacks are well-explored in the field of computer vision, it is impractical to directly apply them in natural language processing due to the discrete nature of the text. To address the problem, we propose a unified framework to extend the existing optimization-based adversarial attack methods in the vision domain to craft textual adversarial samples. In this framework, continuously optimized perturbations are added to the embedding layer and amplified in the forward propagation process. Then the final perturbed latent representations are decoded with a masked language model head to obtain potential adversarial samples. In this paper, we instantiate our framework with an attack algorithm named Textual Projected Gradient Descent (T-PGD). We find our algorithm effective even using proxy gradient information. Therefore, we perform the more challenging transfer black-box attack and conduct comprehensive experiments to evaluate our attack algorithm with several models on three benchmark datasets. Experimental results demonstrate that our method achieves overall better performance and produces more fluent and grammatical adversarial samples compared to strong baseline methods. The code and data are available at https: //github.com/Phantivia/T-PGD.

heading: 1 Introduction
text: Despite great success in real-world applications, deep neural networks (DNNs) are still vulnerable to adversarial samples, which are crafted by adding small and human-imperceptible perturbations to the inputs and can change the prediction
∗Work done during internship at CCIIP lab. †Equally contribution ‡Corresponding author
label of the victim model (Szegedy et al., 2014; Goodfellow et al., 2015).
In the field of computer vision (CV), numerous adversarial attack methods have been proposed to evaluate the robustness of DNNs (Papernot et al., 2016a; Madry et al., 2019), and corresponding defense methods are also well-explored (Papernot et al., 2016c; Ross and Doshi-Velez, 2018). Adversarial attacks on images are defined as an optimization problem of maximizing the loss function of the model on specific samples, which can be approximated by gradient ascent algorithms.
However, the textual adversarial attack is more challenging due to the discrete and nondifferentiable nature of the text space. In Natural Language Processing (NLP), the methods that directly employ the gradients to optimize adversarial samples are not applicable in either the white-box or black-box settings, since they cannot obtain valid discrete texts. For this reason, most works in NLP explore some heuristic methods to produce discrete perturbations, such as manipulating the most important words in the text using corpus knowledge or contextualized information (Ren et al., 2019; Zang et al., 2020; Li et al., 2020). Besides, there are some practices of textual adversarial attacks that employ
7132
gradients for first-order approximation to find optimal candidates in vocabulary for word substitution, but the one-off search is less effective and can violate the local linearization assumption (Cheng et al., 2019; Behjati et al., 2019; Xu and Du, 2020).
To bridge this gap, we propose a general framework to adapt the existing optimization-based adversarial attack methods to NLP (See Figure 1). Essentially, we succeed in obtaining high-quality adversarial samples from the perturbed embedding space. Specifically, we employ gradients to produce perturbations on token embeddings rather than on the original text, thus transforming the problem of searching for adversarial samples in the discrete text space into searching in the continuous and differentiable embedding space. This provides the basis for applying adversarial attack methods investigated in CV to craft textual adversarial samples. In this paper, we adapt the gradient-based algorithm PGD (Madry et al., 2019) within our framework to perform textual adversarial attacks, denoted as T-PGD. Considering that in practical scenarios attackers may not hold the gradient information of the victim model, we explore the possibility of conducting a decision-based transfer attack. To this end, besides the true victim model, we have another model dubbed the local proxy model in the attack process. Gradient information comes from the local proxy model and only the decision of the victim model can be accessed.
Then the perturbed latent representations should be transferred back to the discrete text. Although there have been some works exploring the feasibility of directly perturbing token embeddings (Sato et al., 2018; Cheng et al., 2019; Behjati et al., 2019), they simply use the first-order approximation of the gradient to select candidate words from vocabulary, which might break the local linearization hypothesis. However, recent work finds that the mask language modeling (MLM) head can reconstruct input sentences from their hidden states with high accuracy, even after models have been fine-tuned on specific tasks (Kao et al., 2021). Inspired by this, we employ an MLM head to decode the perturbed latent representations. With the extensive linguistic knowledge of MLM-head, the coherence and grammaticality of adversarial samples can be guaranteed.
We conduct comprehensive experiments to evaluate the effectiveness of our method by performing transfer black-box adversarial attacks, where only
the final decisions of victim models are accessible, against three victim models on three benchmark datasets. Experimental results demonstrate the effectiveness of our framework and T-PGD algorithm, with a higher attack success rate and more fluent and grammatical adversarial examples produced.
To summarize, the main contributions of this paper are as follows: (1) We propose a general textual adversarial attack framework facilitating NLP researchers to produce adversarial texts using optimization-based methods, bridging the gap between CV and NLP in the study of adversarial attacks. (2) Based on the framework, we propose an effective adversarial transfer attack method called T-PGD, handling the challenge of decision-based black-box attack, which is rarely investigated in NLP.

heading: 2 Related Work

heading: 2.1 Adversarial Attack in CV
text: In the field of computer vision, adding a small amount of perturbations to input images to mislead the classifier is possible (Szegedy et al., 2014). Based on this observation, various adversarial attack methods have been explored. FGSM (Goodfellow et al., 2015) crafts adversarial samples using the gradient of the model’s loss function to the input images. BIM (Kurakin et al., 2017) straightforwardly extends FGSM, iteratively applying adversarial perturbations multiple times with a smaller step size. MIM (Dong et al., 2018) exploits momentum when updating inputs, obtaining adversary samples with superior quality. PGD (Madry et al., 2019) employs uniform random noise as initialization. Both MIM and PGD are variants of BIM.
Although well explored in CV, these methods are not directly transferable to NLP due to the discrete nature of the text. A recent work GBDA (Guo et al., 2021) generates adversarial samples by searching an adversarial distribution, optimizing with a gradient-based algorithm that has been previously used in image adversarial attacks (Carlini and Wagner, 2017). In this paper, we propose a general framework enabling the application of adversarial attacks in CV to text without many adaptions.

heading: 2.2 Adversarial Attack in NLP
text: Existing textual attacks can be roughly categorized into white-box and black-box attacks according to the accessibility to the victim models.
White-box attack methods, also known as
gradient-based attack methods, assume that the attacker has full knowledge of the victim models, including model structures and all parameters. There are few application scenarios of white-box attacks in real-world situations, so most white-box attack models are explored to reveal the weakness of victim models, including universal adversarial triggers (Wallace et al., 2019), and fast gradient sign inspired methods (Ebrahimi et al., 2018; Papernot et al., 2016b).
Black-box attack models can be further divided into two different attack settings, i.e. score-based and decision-based. The first one assumes the attacker can obtain the decisions and corresponding confidence scores from victim models. Most research works on black-box attacks focus on this setting, exploring different word substitution methods and search algorithms to reduce the victim models’ confidence scores (Jin et al., 2020; Ren et al., 2019; Zang et al., 2020; Li et al., 2020; Alzantot et al., 2018). The other attack setting assumes the attackers can only obtain decisions from victim models, which is more challenging and less studied. Maheshwary et al. (2021) first substitutes some words in the input sentences to flip the labels and then conducts a search based on a genetic algorithm, expecting to find the most semantic preserved adversarial samples. Chen et al. (2021) propose a learnable attack agent trained by imitation learning to perform a decision-based attack. Some works also explore sentence-level transformation, including syntax (Iyyer et al., 2018) and text style (Qi et al., 2021), to launch attacks. In this work, we consider the latter setting and show that even with less information, our decision-based attack can still be as effective as score-based ones.

heading: 3 Framework
text: In this section, we first present an overview of our framework, and next, we will give the details of how to add continuous perturbations and reconstruct the text.

heading: 3.1 Overview
text: We have two models in the perturbation generation process: (1) a local proxy model which provides gradient information to optimize the adversarial samples, and (2) the true victim model that the attacker attempts to deceive. Specifically, a proxy BERT model fine-tuned on the attacker’s local dataset encodes each discrete text instance into
continuous token embeddings and then adds continuous perturbation to it. The perturbation would be iteratively optimized using the gradient of the proxy model, according to the prediction output of the victim model. After perturbation, an MLM head will decode the perturbed latent representation to generate candidate adversarial samples. The overview of the framework is shown in Figure 2.
With the help of our proposed framework, it is feasible to perform textual adversarial attacks with various gradient-based methods in CV. In this paper, we examine PGD (Madry et al., 2019) as a case (See Section 4).

heading: 3.2 Notation
text: We denote each sample as (x ∈ X , y ∈ Y), where x denotes the input text, y denotes its corresponding label. In particular, the embeddings of x is e, the hidden state is h, and final prediction is ŷ. The local neural network is implied by a mapping function f , which consists of three components, f0, f1, and f2, holding:
f (x) = f2 (f1 (f0 (x))) , (1)
where f0 is the embedding layer, f1 denotes the hidden layers from the first layer to m-th layer, and f2 denotes the rest of the neural network. Then the forward propagation process can be described as:
e = f0 (x) , h = f1 (e) , ŷ = f2 (h) (2)

heading: 3.3 Latent-space Perturbation
text: Previous work has shown that the latent representations of transformer-based pre-trained language models are effective in providing semantic and syntactic features (Clark et al., 2019; Jawahar et al., 2019), and thus we use a local BERT model fine-tuned on our local dataset as the encoder for our framework.
For each text input, we first calculate the taskspecific loss in the forward propagation process, and then perform backward propagation to obtain the gradients of the loss with respect to the token embeddings of the input text. The generated gradients are viewed as the information for updating the perturbations added to the token embeddings, which can be obtained by solving an optimization problem as follows:
δ = arg max δ:‖δ‖2≤ε
L (f2 (f1 (f0 (x) + δ)) , y) , (3)
where δ is the perturbation and L (·) is the loss function.
The closed-form solution to the optimization problem is hard to directly obtain (Goodfellow et al., 2015), which is thus relaxed to obtain an approximate solution. For example, various methods in CV usually linearize the loss function with gradient information to approximate the perturbations δ (Goodfellow et al., 2015; Kurakin et al., 2017; Madry et al., 2019).
In NLP, most existing gradient-based methods commonly employ first-order approximation to obtain substitution words (Cheng et al., 2019; Behjati et al., 2019; Xu and Du, 2020). However, these one-off approaches may result in large step size perturbations, violating the hypothesis of local linearization (See Figure 3). To ensure the local linearization hypothesis, we consider adjusting the continuous perturbations added to the token embeddings with a minor change at each step, and then iteratively update the token embeddings of the input instance with the perturbations until generating a meaningful adversarial sample for attacking.

heading: 3.4 Reconstruction
text: Using continuous perturbations, we need to reconstruct the meaningful adversarial text from the optimized token embeddings. The MLM-head is observed to be able to reconstruct input sentences from hidden states in middle layers with high
accuracy, even after models have been fine-tuned on specific tasks (Kao et al., 2021). Specifically, MLM-head is a pre-trained H × V linear layer, where H is the size of hidden states and V is the size of the vocabulary. Given continuous input hidden states h, it can predict token IDs by t = hAT + b, where A and b are tuned parameters. The IDs can later be decoded into texts by the tokenizer using a predefined ID-token mapping. Inspired by this, we adopt the MLM head as the decoder for two reasons: 1) MLM-head is capable of interpreting any representation embeddings in the hidden space, which is crucial to search adversarial examples continuously; 2) MLM-head has been fully trained during the pre-trained stage so it acquires linguistic knowledge together with the language model and can reconstruct sentences considering the contextual information.
Without loss of generality, we take an example in Figure 3 to illustrate the discrepancy between the one-off-based attack models and our proposed iterative-attack-based model. One-off attack models are prone to choose the token b to serve as the substitute of token a because cos( −→ at1, −→ ab) < cos( −→ at1, −→ac). However, in our framework, the onestep perturbation −→ at1 does not cross the decoding boundary, and thus the decoding results remain unchanged if only using one-step perturbation. Based on the iterative search, the perturbations can be accumulated to the extent to cross the decision boundary and reach the transition point t3, which will be decoded as the optimal solution c. Then a is replaced by c to obtain the adversarial sample to query the victim model for its decision. If this ad-
versarial sample fails to fool the victim model, we start the next searching iteration from the current perturbed token embedding, i.e. t3 in Figure 3, but not from the embedding of the decoded token c. By exploiting virtual embeddings as transition points, this iterative attack framework can preserve accumulated gradient information and avoid breaking local linearization assumptions.

heading: 4 Method

heading: 4.1 T-PGD Algorithm
text: We instantiate our framework with PGD (Madry et al., 2019) algorithm, and name our attack model as Textual-PGD (T-PGD). The algorithm flow of T-PGD is shown in Algorithm 1.
Algorithm 1 T-PGD Require: Original input x sampled from X Ensure: Adversary of x
1: Randomly mask one word in x 2: AdvList = [] 3: BestSim = 0 4: for j ∈ [1, . . . ,MaxIter] do 5: e0 = f0 (x) 6: δ0 =
1 Ne0 Uniform (−ε, ε) 7: for i ∈ [1, . . . ,MaxStep] do 8: ei = ei−1 + δi−1 9: hi = f1(ei)
10: Advi = Dec(hi) 11: Sim = USE (Advi, x) 12: if Advi not in AdvList and Sim >
BestSim then 13: Append Advi to AdvList 14: BestSim = Sim 15: Query victim model with Advi 16: if attack succeed and Sim >
Threshold and no antonyms then 17: return Advi 18: end if 19: end if 20: gadv = ∇δi−1L (f2 (hi) , y) 21: δi = Proj‖δ‖F≤ε ( δi−1 + α gadv ‖gadv‖F ) 22: end for 23: end for
To solve the optimization problem in Eq. (3), we iteratively search for the optimal solution by adding the gradient-based perturbations to the token embeddings.For each sample, we first pre-defined a maximum iteration of the searching
process to avoid the infinite loop problem. In each iteration, we first map input x to the token embeddings and initialize the perturbation by sampling noise from a uniform distribution. In the i-th step, we obtain new embeddings Ei by adding δi−1, the perturbation generated in the last step, to ei−1. Then, ei will be forward propagated to obtain a hidden representation: hi = f1 (ei). Next, the hidden states with perturbations are decoded for reconstructing the crafted adversarial samples, Advi = Dec(hi), where Advi denotes the adversarial sample obtained in this step. We then compute the semantic similarity Simi between Advi and input x using Universal Sentence Encoder (USE) score (Cer et al., 2018).
We query the victim model only when Advi satisfying: (1) it varies from all potential adversarial samples that have been queried before; (2) it is more similar to the original sentences, compared to previous potential adversarial samples. If the attack succeeds and Sim is higher than a hyperparameter Threshold, then Advi is considered as the final adversarial sample of the original input. Otherwise, hi will be forwarded to obtain the prediction of the local model with respect to the input x. We then compute the loss between the predicted label and the golden label y and then calculate the gradient w.r.t. δi, and update the perturbation for next step, with the following formula:
gadv = ∇δi−1L (f2 (hi) , y)
δi = Proj‖δ‖F≤ε
( δi−1 + α
gadv ‖gadv‖F
) , (4)
where gadv is the gradient of the loss with respect to the continuous perturbation δi−1, α is the step size of δi−1, and i denotes the current iteration step. Proj (·) performs a re-initialization when δ reaches beyond the -neighborhood of the original embedding.

heading: 4.2 Heuristic Strategies
text: Random Masking for Diversity. To enhance the diversity of adversarial samples, we randomly mask one token in each input sentence to randomly initialize the search for a broader search scope. Specifically, we tokenize x to a list of tokens, xtoken = [x0, ..., xi, ..., xn]. Then we randomly select i-th index token using the uniform distribution and replace it with a special token [MASK]. Next, the MLM-head-based decoder will predict
the masked word according to its context, which will diversify the generated adversarial samples with semantically consistent consideration. Then, these processed sentences are embedded into continuous token embeddings as aforementioned.
Input Reconstruction Loss. Intuitively, the quality of generated adversarial samples is largely affected by the reconstruction accuracy of the MLM-head-based decoder. If failing to recover the original sentence even with no perturbations added, its capacity to generate fluent adversarial samples from perturbed hidden states might be limited. Therefore, the MLM-head-based decoder should be constrained with external constraints to ensure reconstruction accuracy, thus guaranteeing the quality of generated adversarial samples. Note that the MLM-head has been pre-trained to precisely fill the masked word, which is also fitted to our task. Hence, to preserve the reconstruction performance of the MLM-head in optimization, we add the MLM loss as a regularization term to the loss function. Specifically, the loss function used in Eq. 4 consists of two components:
L(f(x), y) = L1(f(x), y) + βL2(f(x), y), (5) where L1 (f(x), y) is the original loss of the local model on specific tasks (e.g. cross-entropy loss in sentiment classification), L2 (f(x), y) is the CE loss of the input reconstruction task, and β is a weighting constant. Considering that we aim to reduce the reconstruction loss L2 while increasing L (f(x), y) along the gradient direction, β should be negative. Taking two losses into account jointly, we adjust the perturbation searching target to successfully fool the victim models with fewer modifications.
Selection for Layer Index m. The layer index m is dataset-specific but victim-agnostic. This is because there is a trade-off between ASR and USE when decoding different layers (layer index ↑, USE ↑, ASR ↓ ). Therefore, we determine the m by tuning the USE score on a sampled dataset. In practice, we sample 100 examples and adopt BERT as the victim to conduct pilot experiments. We compute the USE scores of decoding different layers. We then set a USE threshold t = 0.8 and disregard layers which leads to a USE score lower than t. Finally, we find the lowest USE among the rest of the layers and set m as the index of the corresponding layer. We set h = 10,11, and 7 for SST-2, MNLI, and AG, respectively.
Antonym Filtering. Li et al. (2019) reports that semantically opposite words locate closely in their representation embeddings since antonyms usually appear in similar contexts. Therefore, we filter antonyms of original words using WordNet (Fellbaum, 2010) to prevent invalid adversarial samples.

heading: 5 Experiments
text: We conduct comprehensive experiments to evaluate our general framework and T-PGD algorithm on the task of sentiment analysis, natural language inference, and news classification. We consider both automatic and human evaluations to analyze our method in terms of attack performance, semantic consistency, and grammaticality.

heading: 5.1 Datasets and Victim Models
text: For sentiment analysis, we choose SST-2 (Socher et al., 2013), a binary sentiment classification benchmark dataset. For natural language inference, we choose the mismatched MNLI (Williams et al., 2018) dataset. For news classification, we choose AG’s News (Zhang et al., 2015) multi-classification datasets with four categories: World, Sports, Business, and Science/Technology. We randomly sample 1,000 samples that models can classify correctly from the test set and perform adversarial attacks on those samples.
For each dataset, we evaluate T-PGD by attacking BERT (Devlin et al., 2019), RoBERTa (Liu et al., 2019), ALBERT (Lan et al., 2020) and XLNet (Yang et al., 2019) with a local fine-tuned BERT model to generate potential adversarial samples. Details of datasets and the original accuracy of victim models are listed in Table 1.

heading: 5.2 Experimental Setting
text: Baseline Methods. We select four strong scorebased attacks as baselines: (1) PWWS (Ren et al., 2019); (2) Textfooler (Jin et al., 2020); (3) PSO (Zang et al., 2020); (4) BERT-Attack (Li et al., 2020). Note that all of them require the confidence scores of victim models, while our model only assumes the decisions are available, which is more challenging. We also make a comparison with the decision-based GBDA (Guo et al., 2021). Evaluation Metrics. We evaluate our method considering the attack success rate and adversarial sample quality. (1) Attack Success Rate (ASR) is the proportion of adversarial samples that successfully mislead victim models’ predictions. (2)
Quality of adversarial samples is evaluated by two automatic metrics and human evaluation, including their semantic consistency, grammaticality, and fluency. Specifically, we use Universal Sentence Encoder (Cer et al., 2018) to compute the semantic similarity between the original text and the corresponding adversarial sample, Language-Tool1 to calculate the increase of grammar errors in texts after being perturbed, and GPT-2 (Radford et al., 2019) to compute the increase of perplexity to measure fluency. We also conduct a human evaluation to measure the validity and quality of adversarial samples.

heading: 5.3 Experimental Results
text: The results of automatic evaluation metrics are listed in Table 2.
Attack Performance. T-PGD consistently outperforms the strong score-based attack methods considering the attack success rate. We attribute the success of our attack method to the more effective searching process following the guidance of the gradient information, which is verified in the ablation study (Section 6).
1https://github.com/jxmorris12/ language_tool_python
Adversarial Sample Quality. We observe that the quality of the adversarial samples generated by T-PGD increases with the text length. Our adversarial samples yield overall higher USE scores than baseline models, indicating that our method can manipulate adversarial samples more precisely with explicit gradient information. And although the grammatical performance of T-PGD is not the best on SST-2, which mostly contains shorter text (See Table 1), MNLI and AG’s News T-PGD produce the fewest grammatical errors and the lowest perplexity, since the embedding space of longer text is broader and has a better optimal solution. Finally, we attribute the overall high quality of our adversarial samples to the introduction of reconstruction loss, which is demonstrated in Section 6.

heading: 5.4 Human Evaluations
text: To further study the quality and validity of adversarial samples, we randomly selected 100 original SST-2 sentences and 100 adversarial samples from the SOTA baseline BERT-Attack and T-PGD respectively for human evaluation. Following (Li et al., 2020), we shuffle the 300 samples and ask 3 independent human judges to evaluate the quality (300 samples per person). For semantic consistency evaluation, we ask humans to predict the labels of mixed texts. For grammar and fluency,
human judges score from 1 to 5 on the above examples. All annotators have no knowledge about the source of the text, and all their evaluation results are averaged (shown in Table 3).
Semantic Consistency. Since human judges have high accuracy on the original text, the prediction results on texts can be regarded as ground truth labels. Therefore, human accuracy can be a criterion for semantic consistency between original sentences and adversarial ones. From the results, human judges achieve 0.68 accuracies on adversarial samples crafted by T-PGD, significantly higher than the baseline method. This result verifies that the adversarial samples crafted by T-PGD have a better semantic consistency.
Grammar and Fluency. We can also conclude from Table 3 that adversarial samples crafted by T-PGD have better quality compared to the baseline method considering the grammar and fluency, evaluated by human annotators. However, both BERT-Attack and T-PGD suffer a decline in grammatical correctness and fluency of adversarial text, leaving room for improvement in future research.

heading: 6 Further Analysis
text: Importance of Gradient Information. T-PGD employs the gradient of the proxy local BERT model to approximate the perturbations. To verify the effectiveness of the gradient information, we conduct an ablation experiment on SST-2 by adding only random perturbations in the embedding space without exploiting the gradient information. In detail, we generate a Gaussian noise with the same mean and variance as our gradient-based perturbations. The results in Table 4 shows that without exploiting the direction of the gradient, the search in embedding space may deviate from the vicinity where the optimal and original points are located, reflected by the low ASR and USE score respectively.
Importance of Reconstruction Task. We show the importance of adding a reconstruction loss (L2 in Eq.( 5)) for generating more accurate reconstruc-
tions. We conduct an ablation study on SST-2. The results are shown in Table 5. On all three victim models, the attack performances (ASR) improve significantly (up to 100) while the quality of adversarial samples deteriorates, with USE score decreasing and grammar errors and perplexity increasing. This validates our claim that without reconstruction loss, the adversarial samples attempt to change the predictions of the model, ignoring whether the semantics is preserved and the linguistic quality is guaranteed. We further tune β to study the trend of ASR and USE score. Results on BERT are shown in Figure 4. We observe that as the absolute value of β increases, at the early stage ASR declines while USE increases, suggesting that at first the effectiveness is sacrificed for sample quality; at the later stage ASR continues to decline and so does the USE, showing that the reconstruction loss should not be over-weighted either.
Efficiency and Imperceptibility. Despite TPGD presenting impressive effectiveness in Table 2, it is also important to figure out if it is obtained by sacrificing efficiency and imperceptibility. Therefore, we examine the query number and perturbation rate by attacking XLNET on SST-2. Re-
sults are shown in Table 6. We observe that TPGD has the lowest perturbation rate, but the query number is relatively high. Hence, we conduct a more detailed experiment to set different MaxStep to track the trend of ASR, USE, and query number. As shown in Figure 5, we can see that by fixing MaxStep to 500, TPGD can still perform a strong attack (ASR=89.27) with a low query budget (Query=89.91). In conclusion, despite we require a relatively high query number to achieve the reported result, we can resort to an efficient version of TPGD which still achieves very competitive ASR.
Transferability Across Models. We investigate the transferability of adversarial examples. We sample 1,000 samples from SST-2 and craft adversarial samples by T-PGD and baseline methods by attacking BERT. Then we test the attack success rate of these adversarial samples on RoBERTa to evaluate the transferability of adversarial samples. As seen in Table 7, adversarial samples crafted by T-PGD achieve the best transferability performance.
Transferability Across Training Datasets. We consider a more practical setting in which the attacker does not have the same downstream training dataset as the victim, i.e. the local proxy model is trained on a different dataset from the victim model. To this end, we train a local proxy BERT model on another sentiment analysis dataset, IMDB or Amazon, and attack the victim model on SST-2. We compared the results with attacking with the local proxy model trained on the same dataset as the true victim model in Table 8. We can see that T-PGD can also achieve great attack performance in these practical circumstances, although slightly worse than training on the same dataset.

heading: 7 Conclusion and Future Work
text: In this paper, we propose a general framework to facilitate generating discrete adversarial texts using optimization-based methods. In our framework, the problem of searching textual adversarial samples in discrete text space is transformed into the continuous embedding space, where the perturbation can be optimized by gradient information, as explored in CV. The perturbations in embeddings will be amplified in the forward propagation process, then decoded by an MLM head from the latent representations. We instantiate our framework with T-PGD, where the gradient comes from the local proxy model instead of the true victim model, i.e. T-PGD performs a decision-based black-box attack. Experimental results show the superiority of our method in terms of attack performance and adversarial sample quality.
In the future, we will adopt other methods in CV with our framework. Besides, we find that our framework can serve as a general optimization framework for discrete texts, and thus has the potential to provide solutions to other tasks like text generation. We will further explore this direction.
Limitations
In experiments we only take PLMs into account because of their prevalence, hence the transferability to non-pretrained models is still unknown. However, due to the generality of PLMs, this can be a minor point in practical scenarios. Moreover, although we successfully transfer adversarial attack methods in CV to NLP using a unified framework, we only instantiate the framework with the PGD attack as an example. It would be interesting to transfer more attack methods in CV and conduct a comprehensive analysis of what methods can benefit NLP, aiming to have a deeper understanding of PLMs.
Ethical Consideration
In this section, we discuss the potential broader impact and ethical considerations of our paper.
Intended Use. In this paper, we design a general framework to adapt existing gradient-based methods in CV to NLP, and further, propose a decisionbased textual attack method with impressive performance. Our motivations are twofold. First, we attempt to introduce adversarial attack methods of CV to NLP, since image attack methods have been well-explored and proved to be effective, therefore helping these two fields better share research resources hence accelerating the research process on both sides. Second, we hope to find insights into the interpretability and robustness of current blackbox DNNs from our study.
Potential Risk. There is a possibility that our attack methods may be used maliciously to launch adversarial attacks against off-the-shelf commercial systems. However, studies on adversarial attacks are still necessary since it is important for the research community to understand these powerful attack models before defending against these attacks.
Energy Saving. We will public the settings of hyper-parameters of our method, to prevent people from conducting unnecessary tuning and help researchers quickly reproduce our results. We will also release the checkpoints including all victim models to avoid repeated energy costs.

heading: Acknowledgement
text: This work was supported in part by the National Natural Science Foundation of China under Grant
No. 62276110, in part by CCF-AFSG Research Fund under Grant No.RF20210005, and in part by the fund of Joint Laboratory of HUST and Pingan Property & Casualty Research (HPL). Thanks to Naixi Chen from SRU for providing hardware maintainance support for this work. The authors would also like to thank the anonymous reviewers for their comments on improving the quality of this paper.

heading: A Adversarial Training
text: We explore to enhance models’ robustness against adversarial attacks through adversarial training on SST-2 with BERT. Specifically, we first generate adversarial samples using the original training dataset. Then we fine-tune the BERT model using the training dataset augmented with generated adversarial samples. We evaluate the model’s original accuracy on the test set and robustness against different adversarial attack methods. As seen in Table 9, the model shows generally better robustness through adversarial training. Besides, the accuracy on the test set is also improved from 89.90 to 90.48, which is different from previous textual adversarial attacks where accuracy is sacrificed for robustness (Ren et al., 2019; Zang et al., 2020).

heading: B Ablation Study of Random Masking
text: We conduct an ablation study of random masking. Our intuition is that random masking can broaden the searching scope of adversarial examples, and thus lead to diverse adversarial samples and higher attack success rate. To prove this, we attack BERT on SST-2, with and without our random masking strategy. Result are shown in Table 10.

heading: C Case Study
text: In Table 11, we present some cases of our adversarial samples which successfully fooled XLNET.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
The section following Sec.7
3 A2. Did you discuss any potential risks of your work? The section after Limitation
3 A3. Do the abstract and introduction summarize the paper’s main claims? 1
7 A4. Have you used AI writing assistants when working on this paper? Left blank.
B 7 Did you use or create scientific artifacts? Left blank.
B1. Did you cite the creators of artifacts you used? No response.
B2. Did you discuss the license or terms for use and / or distribution of any artifacts? No response.
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? No response.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? No response.
B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? No response.
B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. No response.
C 3 Did you run computational experiments? 5, 6
C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Not applicable. Left blank.
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? 4.2, 5
7 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? just a single run
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? 4.2
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? 5.4
7 D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? Left blank.
7 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? Left blank.
7 D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? Left blank.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.
7 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? Left blank.

heading: Future Work
text: In the future, we will adopt other methods in CV with our framework. Besides, we find that our framework can serve as a general optimization framework for discrete texts, and thus has the potential to provide solutions to other tasks like text generation. We will further explore this direction. Limitations
In experiments we only take PLMs into account because of their prevalence, hence the transferability to non-pretrained models is still unknown. However, due to the generality of PLMs, this can be a minor point in practical scenarios. Moreover, although we successfully transfer adversarial attack methods in CV to NLP using a unified framework, we only instantiate the framework with the PGD attack as an example. It would be interesting to transfer more attack methods in CV and conduct a comprehensive analysis of what methods can benefit NLP, aiming to have a deeper understanding of PLMs