TextGrad: Advancing Robustness Evaluation in NLP by Gradient-Driven Optimization
Authors: Bairu Hou
Jinghan Jia
Yihua Zhang
Guanhua Zhang
Yang Zhang
Sijia Liu
Shiyu Chang
Venue: ICLR 2023 poster (2023)

Abstract: Robustness evaluation against adversarial examples has become increasingly important to unveil the trustworthiness of the prevailing deep models in natural language processing (NLP). However, in contrast to the computer vision domain where the first-order projected gradient descent (PGD) is used as the benchmark approach to generate adversarial examples for robustness evaluation, there lacks a principled first-order gradient-based robustness evaluation framework in NLP. The emerging optimization challenges lie in 1) the discrete nature of textual inputs together with the strong coupling between the perturbation location and the actual content, and 2) the additional constraint that the perturbed text should be fluent and achieve a low perplexity under a language model. These challenges make the development of PGD-like NLP attacks difficult. To bridge the gap, we propose TextGrad, a new attack generator using gradient-driven optimization, supporting high-accuracy and high-quality assessment of adversarial robustness in NLP. Specifically, we address the aforementioned challenges in a unified optimization framework. And we develop an effective convex relaxation method to co-optimize the continuously-relaxed site selection and perturbation variables and leverage an effective sampling method to establish an accurate mapping from the continuous optimization variables to the discrete textual perturbations. Moreover, as a first-order attack generation method, TextGrad can be baked into adversarial training to further improve the robustness of NLP models. Extensive experiments are provided to demonstrate the effectiveness of TextGrad not only in attack generation for robustness evaluation but also in adversarial defense. From the attack perspective, we show that TextGrad achieves remarkable improvements in both the attack success rate and the perplexity score over five state-of-the-art baselines. From the defense perspective, TextGrad-enabled adversarial training yields the most robust NLP model against a wide spectrum of NLP attacks. 

Published as a conference paper at ICLR 2023
TEXTGRAD: ADVANCING ROBUSTNESS EVALUATION IN NLP BY GRADIENT-DRIVEN OPTIMIZATION
Bairu Hou', Jinghan Jia Yihua Zhang?*, Guanhua Zhang
Yang Zhang', Sijia Liu Shiyu Chang" UC Santa Barbara, "Michigan State University, "MIT-IBM Watson Al Lab
ABSTRACT
Robustness evaluation against adversarial examples has become increasingly im- portant to unveil the trustworthiness of the prevailing deep models in natural lan- guage processing (NLP). However, contrast to the computer vision (CV) do- main where the first-order projected gradient descent (PGD) is used as the bench- mark approach to generate adversarial examples for robustness evaluation, there lacks principled first-order gradient-based robustness evaluation framework in NLP. The emerging optimization challenges lie in 1) the discrete nature of textual inputs together with the strong coupling between the perturbation location and the actual content, and 2) the additional constraint that the perturbed text should be fluent and achieve a low perplexity under a language model. These challenges make the development of PGD-like NLP attacks difficult. To bridge the gap, we propose TEXTGRAD, new attack generator using gradient-driven optimization, supporting accuracy and high-quality assessment of adversarial robustness in NLP. Specifically, we address the aforementioned challenges in a unified opti- mization framework. And develop an effective convex relaxation method co-optimize the continuously-relann site selection and perturbation variables, and leverage an effective sampling method to establish accurate mapping from the continuous optimization variables to the discrete textual perturbations. Moreover, as a first-order attack generation method, TEXTGRAD can be baked in adversarial training to further improve the robustness of NLP models. Extensive experiments are provided to demonstrate the effectiveness of TEXTGRAD not only in attack generation for robustness evaluation but also in adversarial defense. From the at- tack perspective, we show that TEXTGRAD achieves remarkable improvements in both the attack success rate and the perplexity score over five state-of-the-ar baselines. From the defense perspective, BEXTGRRA-enabbl adversarial train- ing yields the most robust NLP model against wide spectrum of NLP attacks.
INTRODUCTION
The assessment of adversarial robustness machine learning (ML) models has received increasing research attention because of their vulnerability to adversarial input perturbations (known as adver-
sarial attacks) (Goodfellow et al., 2014; Carlini & Wagner 2017 Papernot et al. 2016). Among
variety of robustness evaluation methods, gradient- adversarial attack generation makes tremendous success in the computer vision (CV) domain (Croce & Hein 2020 Dong et al. 2020).
For example, the projected gradient descent (PGD)-based methods have been widely used bench- mark the adversarial robustness of CV models (Madry et al.) 2018: Zhang et al.] 2019b] Shatahi al. 2019: Wong al., 20200) Zhang et al. 2019a] Athalyy et al.] (2018). However, in the natural language processing (NLP) area, the predominant robustness evaluation tool belongs to query- query-based attack generation methods (Li et al. 2020 Jin al. 2020 Ren al. 2019, Garg & Ramakrishnan, Li et al. 2019), which do not make the full use of gradient information.
2020
Yet, the (query-based) mainstream of NLP robustness evaluation suffers several limitations. First, these query-based attack methods could be prone to generating ambiguous or invalid adversarial textual inputs (Wang et al.) 2021), most of which change the original semantics and could even
"Contributed equally.

---

Published as a conference paper at ICLR 2023
Table 1: Effectiveness of TEXTGRAD at-a-glance the SST-2 dataset (Socher al. 2013 against 5 NLP attack baselines. Each attack method is categorized by the attack principle (gradient-hasee hased vs. query-based), and is evaluated at three aspects: attack success rate (ASR), adversarial texts quality (in terms of language model perplexity), and runtime efficiency (averaged runtime for atlack generation in seconds). Two types of victim models are considered, ie., realizations of BERT achieved by standard training (ST) and adversarial training (AT), respectively. Here AT integrates TEXTFOOLER (Jin al. 2020) with standard training. Across models, higher ASR, lower perplexity, and lower runtime indicate stronger attack. The best performance is
highlighted in bold per metric.
Principle
Attack Success Rate
Perplexity
Runtime Efficiency (s)
Attack
Venue
Gradient
Query
ST
AT
ST
AT
Jin ct al.
2020
AAAI
82.8%
43.6%
431.4
495.7
1.03
La el al.
2020
EMNLP
86.4%
76.4%
410.7
357.8
1.69
Garg Kantakrishiial
2020
EMNLP
86.6%
77.5%
286.6
302.9
1.15
al.
2021
EMNLP
85.7%
79.7%
314.0
381.7
11.44
L,c ct 2022
ICML.
86.0%
65.8%
421.0
554.9
9.41
TEXTGRAD (ours)
93.5 %
81.8%
266.4
285.3
3.65
mislead human annotators. Second, the query-based methods could be hardly integrated with the first-order optimization- based model training recipe, and thus makes it difficult to develop adversar- ial training-based defenses (Madry et al., 2018: Athalye et al. 2018). Even though some first-order optimization- based NLP attack generation methods were developed in the literature, they often come with poor attack effectiveness (Ebrahimi et al. 2018) or high computational cost (Guo et al.) 2021), leaving the question of whether the best optimization framework for NLP attack generation is found. The most relevant work to ours is GBDA attack (Guo et al., (021), which perturbs each token in the input by sampling substitutes from the whole vocabulary of victim model. The sample distribution is optimized using gradients to generate adversarial examples, but yields low computational efficiency and high memory cost. Inspired by above, we ask: How develop a principled gradient-base attack framework in NLP, like PGD in CV?
The main challenges for leveraging gradients generate adversarial attacks in NLP lie in two as- pects. First, the discrete nature of texts makes it difficult to directly employ the gradient information on the inputs. Different from perturbing pixels in imagery data, adversarial perturbations in an textual input need to optimize over the discrete space of words and tokens. Second, the fluency requirement of texts imposes another constraint for optimization. In contrast to e--norm constrained attacks in CV, adversarial examples in NLP are required to keep a low perplexity score. The above two obstacles make the design of gradient- based attack generation method in NLP highly non-trivial.
To bridge the adversarial learning gap between CV and NLP, we develop novel adversarial attack method, termed TEXTGRAV by peering into gradient-driven optimization principles needed for ef- fective attack generation in NLP. Specifically, we propose convex relaxation method to co-optimize the perturbation position selection and token modification. To overcome the discrete optimization difficulty, we propose an effective sampling strategy enable an accurate mapping from the continu- ous optimization space to the discrete textual perturbations. We further leverage a perplexity- loss to optimize the fluency of the generated adversarial examples. In Table we highlight the attack improvement brought by TEXTGRAD over some widely-used NLP attack baselines. More thorough experiment results will be provided in Sec.5]
Our contribution. We propose TEXTGRAD, novel first-order gradient-drived adversarial at- tack method, which takes a firm step to fill the vacancy of principled PGD-based robustness eval- uation framework in NLP. @ We identify few missing optimization principles to boost the power of gradient-bases NLP attacks, such as convex relaxation, sampling-based bassed potinuouustto-dicrees mapping, and site-token co-optimization. 2 We also show that TEXTGRAD is easily integrated with adversarial training and enables effective defenses against adversarial attacks in NLP. O Lastly, we conduct thorough experiments to demonstrate the superiority of TEXTGRAD existing baselines in both adversarial attack generation and adversarial defense.
2 BACKGROUND AND RELATED WORK
Adversarial attacks in CV. Gradient information has played an important role in generating ad- versarial examples, Le., human-imperceptiile perturbed inputs that mislead models, in the CV
2

---

Published as a conference paper at ICLR 2023
area (Goodfellow et al. 2014 Carlini & Wagner 2017: ; Croce& & Hein 2020
Madry et al. 2018
Zhang et al. 2019b Kurakin et al. |2116: Xu et al. 2019a c). PGD attack is one of the most popular adversarial attack methods (Madry et al. 2018 Croce & Hein 2020), which makes use of the first- order gradient to generate perturbations on the inputs and has achieved great success in attacking CV models with low computational cost. Besides, PGD is also powerful method to generate transfer attacks against unseen victim models (Szegedy et al. [013] Liu et al. [0166] Moosavi-Dezfore et al.) 2017). Even in the black-box scenario (i.e., without having access to model parameters), PGD is a principled framework to generate black-box attacks by leveraging function query- -ueey-based gradient estimates (Cheng et al. 2018) or gradients of surrogate models (Cheng et al. [019 Dong et al.) 2018
Xie et al.] 2019: Zou et al. 2020: Dong et al. 2019).
Adversarial attacks in NLP. Different from attacks against CV models, gradient-based attack generation methods are less popular in the NLP domain. HOTFLIP (Ebrahimi et al.) (018)) is one of the most representative gradient-basee attack methods by leveraging gradients to estimate the impact of character and/or word-level substitutions on NLP models. However, HOTFLIP neglects the optimality of site selection in the discrete character/token space and ignores the constraint on the post-attacking text fluency (to preserve readability) (Ren et al.) 2019). By contrast, this work co- optimizes the selections of perturbation sites and tokens, and leverages a perplexity-guine maintain the fluency of adversarial texts. Another attack method GBDA (Guo et al. 2021) models the token replacement operation as a probability distribution which is optimized using gradients. However, acquiring this probability distribution is accompanied with high computation and memory costs. By contrast, our work can achieve comparable or better performance with higher efficiency.
loss to
The mainstream of robustness evaluation in NLP in fact belongs to query-based methods (Li et al., 2020 Jin et al. 2020 Ren et al. 2019 Garg & Ramakrishnan 2020 Li et al. 2019: Wang et al,, 2021
Li et al. 2021b) Many current word-level query-based attack methods adopt two-phase
framework (Zang et al. 2020) including (1) generating candidate substitutions for each word in the
input sentence and (2) replacing original words with the found candidates for attack generation.
candidate substitutions for each word in the original sentence. Genetic Attack (GA) (Alzantot et al. 2018) uses the word embedding distance to select the candidate substitutes and filter out the candidates with an extra language model. Such strategy is also adopted in many other attack methods like et al. (2020) Fnr/hhiii al. (2018). PWWS (Ren et al.] 2019) adopts WordNet (Miller) 1995) for candidate substitution generation. Similarly, PSO Attack (Zang et al. 2020) employs a knowledge base known as HowNet (Qi et al.) 2019) to craft the candidate substitutions. Along with the development of the large pre-trained language models, Garg & Ramakrishnan (2020) and et al. (2.00))) propose to utilize mask language models such as BERT (Devlin et al. (219)) to predict the candidate substitutions. In TEXTGRAD, we also adopt the pre-trained language models to generate candidate substitutions.
Phase (1) aims at generating semantically-preservvii
Phase (2) requires the adversary to find substitution combination from the candidates obtained in
phase (1) to fool the victim model. A widely-used searching strategy is greedy search (Li et al. 2020 Jin et al. 2020: Ren et al.; 2019: Garg & Ramakrishnan 2020: Li et al.; 2019), where each candidate is first ranked based on its impact on model prediction, and then the top candidate for each word is selected as the substitution. Another popular searching strategy is leveraging population- based methods, such as genetic algorithms and particle swarm optimization algorithms (Kennedy) & Eberhart [1995). to determine substitutions (Zang et al., 2020 Alzzntoo et al.) 2018). Despite effectiveness, these query-based attack are prone to generating invalid or ambiguous adversarial examples (Wang et al. 2021) which change the original semantics and could even mislead humans. To overcome these problems, we propose TEXTGRAD, which leverages gradient information to co-optimize the perturbation position and token selection subject to a cattence-fluency constraint. We will empirically show that TEXTGRAD yields better attack success rate with lower sentence perplexity compared to the state-of-the-an query-based attack methods.
Adversarial training. Adversarial training (AT) (Goodfellow et al., 2014 Madry et al., 2018)
has been shown as an effective solution to improving robustness of deep models against adversarial attacks (Athalye et al. 2018). AT, built upon min-max optimization, has also inspired a large number of robust training approaches, ranging from supervised learning (Wong et al. 2020 Zhang et al. submitted to NeurIPS, 2021) to semi-supervised learning (Zhang et al.) 2019b] C Carmon al. 2019), and further to sulf-supervised learning (Chen et al. 2020). Yet, the aforementioned literature focuses on AT for vision applications. Despite some explorations, AT' for NLP models is generally under-
3

---

Published as a conference paper at ICLR 2023
explored. In JJin et al. 2020; Zang et al. 2020 Wattenhofer 2020: Li et al.
Alzantot et al. 2018 Zhang et al. 2019c Meng & 202 la). ad adversarial data are generated offine and then integrated with the vanilla model training. In Li et al. (2021b); Zhu et al. (2019): Dong et al. (202I): D: Wang et al. (2000), the min-max based AT is adopted, but the adversarial attack generation step (i.e. the inner maximization step) is conducted both methods are not effective to defend against strong NLP attacks like TEXTGRAD.
the embedding space rather than the input space. As result,
MATHEMATICAL FORMULATION OF NLP ATTACKS
In this section, we start with a formal setup of NLP attack generation by considering two opti- mization tasks simultancously: (a) (token-wise) attack site selection, and (b) textual perturbation via token replacement. Based on this setup, we will then propose a generic discrete optimization framework that allows for first-order gradient-based optimization.
Problem setup. Throughout the paper, we will focus on the task of text classification, where M(x) a victim model targeted by an adversary to perturb its input x. Let x [x], T2,..,,...... -rr, E N be the input sequence, where I; E (o.1....., IV| 1} is the index of ith token, V is the vocabulary table, and V refers to the size of the vocabulary table.
From the prior knowledge perspective, we assume that adversary has access to the victim model (i.e., white-box attack), similar to many existing adversarial attack generation setups in both CV and NLP domains (Carlini & Wagner 2017; Croce & Hein 2020 Madry et al.. 2018 SZegedy al. 2013). Besides, the adversary has prior knowledge on a set of token candidates for substitution at each position, denoted by 8, {sKI 8 ... ,im J at site i, where 8tj (0, 1,...; |V| .
denotes
the index of the jth candidate token that the adversary can be used to replace the ith token in x. Here m is the maximum number of candidate tokens.
From the attack manipulation perspective, the adversary has two goals: determining the optimal attack site as well as seeking out the optimal substitute for the original token. Given this, we intro- duce site selection variables z = [a....., _L] to encode the optimized attack site, where 2; E {0, 1} becomes if the token site selected and 0 otherwise. In this regard, an attack budger given by the number of modified token sites, 1 S ki where k is the upper bound of the budget. We next introduce token-wise replacement variables u, = [[st,.... Wim), associated with candidates in s;, where Uaj {0, 1}, and 1 candidate expressed by &, (., = E, (uag Sky). Please note that there is only one candidate token
1f 2, 1. Then, the ith input token 2; will be replaced by the
will be selected (constrained through I'u = 1). For ease of presentation, we will use é to denote the replacement-enablee perturbed input with the same length of x.
In a nutshell, an NLP attack can be described as the perturbed input example together with site selection and replacement constraints (cstr):
Perturbed input: xddv (z, x;x,8) (1--))xx+20
Discrete variables: z E 10,1)' u. E [0, Vi
(L)
Site selection estr: 1
Replacement estr: 1"u, = 1, Vi,
where denotes the element-wist wise multiplication. For ease of notation, u and are introduced to denote the sets of }u,} and {s,} respectively, and the adversarial example addv is a function of site selection and token replacement variables (z and u) as well as the prior knowledge on the input example x and the inventory of candidate tokens s. Based on (.. we will next formulate the optimization problem for generating NLP attacks.
Discrete optimization problem with convex constraints. The main difficulty of formulating the NLP attack problem suited for efficient optimization is the presence of discrete mno-cconve) con- straints. To circumvent this difficulty, common way is to relax discrete constraints into their convex counterparts (Boyd et al. (204)). However, this leads to an inexact problem formulation. To close this gap, we propose the following problem formulation with continuous convex constraints and an attack loss built upon the discrete projection operation:
minimize a subject
(xxx(x (B(2),
(2)
C:: [0.1) 1 < k, Ca Ca:uE lu = 1, Vi,
4

---

Published as a conference paper at ICLR 2023
where most notations consistent with (I) B is the projection operation that projects the continu- ous variables onto the Boolean set, i.e., 2 E {0, 13" and u; E {0, (0,1)m in Q and we use 2 and û, as the continuous relaxation of z and u, As will be evident later, efficient projection operation can be achieved by randomized sampling. The graceful feature of problem (2) is that the constraint sets C1 and C2 are convex, given by the intersection of a continuous box and affine inequality/equalitis..
4 OPTIMIZATION FRAMEWORK OF NLP ATTACKS
In this section, we present the details of gradient-driven first-order optimization that can be success- fully applied to generating NLP attacks. Similar to the attack senchmark-poojectee gradient descent (PGD) attack Madry et al. (2018) for generating adversarial perturbations of imagery data, we propose the PGD attack framework for NLP models. We will illustrate the design of PGD-based NLP attack framework from four dimensions: 1) acquisition of prior knowledge on inventory of can- didate tokens, 2) attack loss type, 3) regularization scheme to minimize the perplexity of perturbed texts, and 4) input gradient computation.
candidate tokens for substitution. We first tackle how to generate candidate tokens used for input perturbation. Inspired by BERT-ATTACK et al. (020) and BAE- R (Garg & Ramakrishnan 2020), we employ pre-trained masked language models (Devlin al.) 2019 Liu et J 2019, Lan et al., 2020), denoted as g, to generate the candidate substitutions. Specifically, given the input sequence x, first feed it into to get the token prediction probability at each position without masking the input. Then we take the top-m tokens at each position as the candidates. Please note that getting the token predictions at each position does not require masking the input. Using the original sentence as the input can make the computation more efficient (only one forward pass to get predictions for all positions). With similar approach, it has been shown in (Li et al., 2020) that the generated candidates are more semantically consistent with the original tokens. one, compared to the approach using actual " "[mask]" tokens......................... Note that TEXTGRAD AD compatible with any other candidate tokens generating method, making it general for practical usage.
Prior knowledge acquisition:
Determination of attack loss. Most existing NLP attack generation methods (Li et al.) 2020
Jin et al., 2020 Ren et af. 2019| ATzantot et al. 2018 Li et al. 2021a) use the negative cross- entropy (CE) loss as the attack objective. However, the CE loss hardly tells whether or not the attack succeeds. And it would hamper the optimization efficiency when the attack objective is regularized by another textual fluency objective (which will be introduced later). Our rationale is that intuitively a sentence with more aggressive textual perturbations typically yields a higher attack success rate but a larger deviation from its original format. Thus, is more likely to be less fluent.
A desirable loss for designing NLP attacks should be able to indicate the attack status (failure vs. success) and can automatically adjust the optimization focus between the success of an attack and the promotion of perturbed texts fluency. Spurred by the above, we choose the C& C-yyty attack loss (Carlini & Wagner, 2017):
Eask (xexxx max (xx (xxdy) max Zx(x ),
(3)
where xadv was introduced in (2). la denotes the predicted class of the victim model against the original input x, and 2, denotes the prediction logit of class i. In (3). the difference Z. (xady) max,4to (x(xdr) characterizes the confidence gap between the original prediction and the incorrect prediction induced by adversarial perturbations. The key advantages of using (3) for NLP attack generation are two-fold. First, the success of x (whose prediction is distinct from the original model prediction) is precisely reflected by its zero loss value, i.e., latk (xxdv) = O. Second, the attack loss (3) has the self-assessmeent ability since it will be automatically de-activated only if the attack succeeds, i.e., Zro (xadv) I maxiştt, (xxdv)) Such an advantage facilitates us strike graceful balance between the attack success rate and the texts perplexity rate after perturbations.
Text fluency regularization. We next propose differentiable texts fluency regularizer to be jointly optimized with the C&W attack loss,
f1eg Σ, 24 E, (4n(ttm(((s))) lmim (x.)) = E, E, E, (nykmmm(@s))) E, (x.) where the last equality holds since E, u, 1. mmim (.) indicates the masked language modeling loss (Devlin et al.) 2019) which is widely used for measuring the contribution of a word to the sen- tence fluency. For example, Euim (si)) measures new sentence fluency after changing the ith position
(4)
5

---

Published as a conference paper at ICLR 2023
as its jthh candidate. Smaller mm (s:z) indicates better sentence fluency after the replacement. We compute the masked language model loss (mim(()) for ith token and minimize the increment of masked language model loss after replacement.
Input gradient calculation. The availability of the gradient of the attack objective function is the precondition for establishing the PGD-based attack framework. However, the presence of the Boolean operation B()) in (2) prevents us from gradient calculation. To overcome this challenge, we prepend a randomized sampling step to the gradient computation. The rationale is that the continuously relaxed variables z and ú in (2) can be viewed as (element-wise) site selection and token substitution probabilities. In this regard, given the continuous values ž 2 and û ü obtained at the last PGD iteration (t - 1), for the current iteration & we can achieve (-) through the following Monte Carlo sampling step:
(2 {
with probability 31- with probability 1 2.-1,1
(5)
where [B(r) (žg-1)]s denotes the ith element realization of () at the r-th random trial. We use R to denote the total number of sampling rounds. The above sampling strategy can be similarly defined to achieve (ū,-)) It is worth noting that a large R reduces the variance of the random realizations of B1) and can further help reduce the gradient variance. Our empirical experiments show that R 20 suffices to warrant satisfactory attack performance. Based on (5). the gradient of the attack objective function in (2) is given by
g1. == 1 Σ
Uzlack (xadv
adv (z! u ; s)), 82.
1
Σ
V Valau((xxe (x!"), u ; s)),,
(6)
R
R
where (r)) = (2--1) and u (r) = B (ū, (ú,-1), and g1,+ and g2, corresponds to the variables 2 and ú, respectively. Our gradient estimation over the discrete space also has the spirit similar to straight-through estimator (Bengio et al. 2013) and Gumbel Softmax method (Jang et al. 2016).
Projected gradient descent (PGD) framework. Based on the C&W attack loss (3) the texts fluency regularization 4 and the input gradient formula (6). we are then able to develop the PGD optimization method to solve problem (2). At the t-th iteration, PGD is given by
2. Ie, (ā.-1 T)g,,,), u, = Ic, (ū. 7/g2,1),
where Ilc denotes the Euclidean projection onto the constraint set C, i.e., IIc(a) = arg minxec ||x x| all2 and the constraint sets C1 and C2 have been defined in 2 Due to the special structures of these constraints, the closed forms of the projection operations Ilc, and IIc, are attainable and illustrated in Proposition[A.I] in the appendix. Using the optimization framework above, the empirical con- vergence of the PGD is relatively fast. It will be shown later, 5-step PGD is sufficient to make our algorithm outperforming all other baseline methods.
EXPERIMENTS
5.1 EXPERIMENT SETUP
Datasets and attack baselines. We mainly consider the following tasks [T SST-2 Socher et al,
2013) for sentiment analysis, MNLI (Williams et al., 2018), RTE (Wagg al. 2018), ,, and QNLi
WWang et al., 2018) for natural language inference and AG News (Zhang et al. 2015) for text classification. We compare our proposed TEXTGRAD method with the following state-of-the-an white box and black-box NLP attack baselines: TEXTFOOLER (Jin et al. 2020), BERT-ATTACE (Li al. 2020), BAE-R (Garg Ramakrishnan 2020), HOTFLIP (Ebrahimi al. 2018). BBA (Lee et al. 2022) and GBDA (Guo et al.) 2021). We also include a greedy search- based method termed GREEDY, which combines the candidate genration method used in ours and the Greedy- -IR search strategy in Morris et al. (2020). In this regard, since the candidate substitute set is the same for GREEDY and TEXTGRAN AD, we can better demonstrate the advantage of TEXTGRAD over baselines that use greedy search to craft adversarial examples. We follow the benchmark attack setting in (Wang et al. 2021 Li et al. 2021b). The attack budget is set to 25% of the total word numbers in sentence for baselines and T TEXTGRAD to ensure the fair comparison. More details about the attack implementations could be seen in Appendix
Codes available at httptttt///// http://ghhhhhbccco//////aaaa////

---

Published as a conference paper at ICLR 2023
Victim models. We consider two classes of vie- tim models in experiments, namely convention- ally trained standard models and robustly trained models with awareness of adversarial attacks. In the robust training paradigm, consider Adver- sarial Data Augmentation (ADA), sixup-based Adversarial Data Augmentation (MIXADA) (Si
Table 2: Performance of proposed TEXTGRAD tack method and baseline methods against normally trained victim models. The performance is measured by attack success rate (ASR) as well as perplexity (PPI.) across different datasets and model architec- tures. more powerful attack method is expected have a higher (t) ASR and lower (J) PPI.. The best results under each metric are highlighted in bold and second best are underlined.
et al.
2021). PGD-AT (Madry et al., 2018).
FREELB (Zhu et al. 2019), INFOBERT Wang
et al. 2020), and ASCC (Dong et al.,
2021).
Notably, except ADA and MIXADA, other ro- bust training methods impose adversarial per- turbations at the (continuous) embedding space.
Danset
Araak Method
ASR
BERT
P01.
ASR
RoBERT 105.
ASR
ALBERT
PVS.
TENTPOOO
144
68%.22
69.77
S4
BERT-ATTACK
56.44
435.29
32:73
851-
Following (Li et al. 2021b), we remove the le perturbation constraint when training with PGD- AT and FREELB.
HorFLL
56.07
7g
15.30
18:35
BBA
65.06
421.00 00
81.51
532.69
$1.19
451
GHDA
15.70
314000
TENTERAD
90.31
266.41
96.85
276.
95.51
315.4
TENTFOOLLS
314.11
66.02
MERIIAATTAU
19.77
241:25
All the victim models are fine-tuned based on three popular NLP encoders, i.e., BERT-base (Devlin et al. 2019), RoBaRTa--arge (Liu et al. 2019). and ALBERT-xxlarge22 (Lan al. 2020). When attacking these models, TEXTGRAD use the corresponding masked language model
MNL
GREEDY
265.53
35.66
272.94
HorFLUP
54.46
31
76.46
204
29.14
91
BBA
$2366
346.60
78.44
329,63
77.84
423.60
GBDA
50.17
250.4]
TEXTGRAD
9408
193.42
95.44
211.51
94.44
254 17
TENTFoor
$19.4
74 M1
ATTAC-
6461
30
279.6
70.17
343 62
GuEEUY
0067
78.11
32.52
517.07
HorFLIP
45.51
318.13
70.55
194.07
34367
901 32
generate candidate substitutes. We also follow the best training settings in (Li al. 202Tb).
BBA
60.67
361.30
69.16
237.41
70.62
418.07
GHIDA
50.20
471 20
TEXTGRAD
71.1
202.96
83.90
148.51
87.41
378.07
TEXTFOOLLE
53.55
359.90
398-65
36-34
451 02
QNLI
GRREDY
BAE-R
62.31
67.55
500.55
6281
6321
324.4
570.64
Evaluation metries. First, attack success rate (ASR) measures the attack effectiveness, given by the number of examples that are successfully attacked over the total number of attacked ex- amples. Second, perplexity (PPL) measures the quality of the generated adversarial texts. We use the pre-trained GPT-XL (Radford et al. 2019) language model for PPL evaluation.
HorFUP
48.07
301 35
49.91
$13.0
BBA
60.31
498.74
59.12
429.77
5574
451.55
GRDA
63.52
1473.15
TEXTGRAD
70,48
297.59
68.00
297.16
73.83
335.24
TEXIPPOOLER
427.19
64.37
475
News
00:35
52364
02.35
579 81
S27.68
25344
12.99
534 35
HorPLiP
49.27
307.60
52.08
465.66
35:41
431.36
BBB
T4.70
147.31
05.15
157.87
55.01
33571
GBDA
70:26
456 60
TEXTGRAD
74.51
303.21
75.19
203.01
85.12
397.93
The official impurmenttionn of GRDD eveentl oupoor stocking RoRFRTyAl BERT See Appendx 0 0z detailed explanations.
5.2 EXPERIMENT RESULTS
standard models. In Table we compare the at- tack performance (in terms of ASR and PPL.) of TEXTGRAD with 7 NL. attack baselines across 5 datasets and 3 victim models that are normally trained. As we can see, TEXTGRAD yields better ASR than all the other baselines, leading to at least 3% improvement in nearly all settings. Except our method, there does not exist any baseline that can win either across dataset types or across model types. From the perspective of PPL, TEXTGRAD nearly outperforms all the baselines when attack- ing BERT and RoBERTa. For the victim model ALBERT, TEXTGRAD yields the second or the third best PPL. with a small performance gap to the best PPL result. Additionally, the ASR improvement gained by TEXTGRAD remains significant in all settings when attacking ALBERT, which shows a good adversarial robustness in several past robustness evaluations (Wang et al. 202T).
Attack performance
Attack performance on robustly-trained models. of TEXTGRAD against robustly- trained models. To make a thorough assessment, attack methods are compared under 6 robust BERT models obtained using 6 robust training methods on SST-2 and RTE datasets. As lines, BBA and GBDA seem outperforming the others when attacking robust models. However, compared TEXTGRAD, there remains over 4% ASR gap in most of cases. From the PPL per- spective, TEXTGRAD achieves at least the second best performance. It is worth noting that the con- sidered test-time attacks (including TEXTGRAD and baselines) are not seen during robust training. Therefore, all the robust models are not truly robust when facing unforeseen attacks. In particular, TEXTGRAD can easily break these defenses on SST-2, as evidenced by achieving at least 89% ASR.
demonstratts the attack effectiveness
see, TEXTGRAD yields the best ASR in all settings. Among base-
7

---

Published as a conference paper at ICLR 2023
Table 3: Performance of attack methods against rohustly-trained victim models. Different robustified versions
BERT are obtained using different adversarial defenses including ADA, MIXADA (Si et al.) (021), 1 PGD- AT Madry et al. 2018), FREE-LB (Zhu ct al. 2019), INFOBERT (Wang et al. 2020). and ASCC DDng et al.
2021 J. Two datasets including SST-2 and RTE are considered. The attack performance is measured by ASR and PPL. The best results under each metric (corresponding each column) highlighted in bold and the
second best results are underlined.
Dataset
Attack Method
ASR
|
PPL
ASR
MIXADA
PPL
ASR
PGD.AT
PPL
ASR
FRIE--LB
PPL
ASR
INFOBERT
PPL
ASR
ASCC
PPL
TEXTFOOLER
81.20
550.91
81.56
531.
74.30
481.40
71.99
474.86
$0.19
440.09
83.46
554.42
BERT-AFTACK
84.77
455.21
84.47
459.7
408.57
79.45
407.13
86.
443.85
83,33
431
881-2
GREEDY
BAE-R
86.20
85,90
324.66
447.78
85.30
$6.90
291.64
433.93
83.22
80.79
411.48
288.92
79.86
78.57
440.61
341.96
87.40
86.22
429.34
324.51
80.52
83.46
296.65
406.83
HOTFLIP
47.11
28
49.26
40.87
353.18
23.
66
355.40
61.55
314-29
BBA
90.00
$10.83
$8.37
479 33
78.94
46.1.6
83.61
163.33
84.86
479.11
91.69
455.72
GBDA
63.40
321.75
$6.12
346.70
83.87
368.14
67.35
325,05
83.50
274.11
27.71
318.85
TEXTGRAD
93.81
316.85
93.72
271.
92.20
272.11
93.00
2700.37
92.00
TEXTFOOLER
62.42
300.29
65.62
384.39
56.22
214.87
63.40
270.23
68.36
405.06
-19.41
278.88
BERT-ATTACK
68.48
296.96
71.88
264,
60.54
287.78
64.95
313,94
72.88
365,47
55,88
313.11
RTE
GREEDY
BAE-R
63.86
66.67
378.57
231 8D
T2.50
73.46
432.57
2.0.12
58.60
61.62
392.67
199.79
68.
67.55
274.13
163.37
69 (99449
71.35
293.98
372.42
55.03
52.35
216.94
435.63
HOTFLIP
30.12
479.65
25.31
379.64
29.57
405 00
46
368.25
28.65
345 80
33.73
290.61
BRA
69.69
340 60
70.65
418.07 (7
271
66
419,29
35%.07
52.35
237
GBDA
81.81
986.35 36
75.30
473.80 80
67.02
973.50
20.10
1461.07
84.18
1331.00
34,70
483. 38
TEXTGRAD
80.
219.81
87.04
254,
71.51
223 56
78.06
302.81
82.55
204.64
67.46
226 54
34
14
91.
64
20
91
2010
Torliol
TerOred
Todliod
Croady
Bar-Allchh
54C
Gready
Anock nocation surtaer
AT3CA
ATACE Rearon number
Atdck meradon Humber
Abbect Hamaton Mumbbe
(a) Dataset: SST-2
(b) Dataset: MNLI-m
(c) Dataset: RTE
(d) Dataset: QNLI
(e) Dataset: AG News
Figure 1: ASR of TEXTGRAD with different attack iteration numbers. We attack the standard BERT model with TEXTGRAD different datasets. For each dataset, we show the curve of ASR of TEXTGRAD wett different iteration numbers (the orange curves) well as the ASR of the best query-based baseline each corresponding dataset from Table (the dashed lines). TEXTGRAD can beat the best baseline with only iterations on all datasets.
TEXTGRAD versus attack strengths. More- over, evaluate the attack performance of TEXTGRAD from two different attack strength setups: (a) the number of optimization steps for attack generation, and (b) the maximum num- ber of words modifications (that we call at- lack budget), i.e., k in (2). Results associated with above setups (a) and (b) are presented in [ and Table In both cases, we com- pare the performance of TEXTGRAD with that of baselines when attacking a normally trained BERT model. As shown in Figure[] ASR of TEXTGRAD increases the number of attack iterations increases. Moreover, TEXTGRAD achieves a substantial ASR improvement over the best baseline by merely taking very small number of iterations (less than 8 iterations in all cases). As shown in Table ASR of TEXTGRAD increases as the attack budget (k) increases. Additionally, even if k is small (e.g., k 5% of the number of words), TEXTGRAD still significantly outperforms the baselines in ASR.
Effect of attack budget k on ASR of TEXTGRAD. Evaluation is performed under the dard BERT model on SST. Recall that the attack budget constraints the ratio of the words allowed to be modi- fied in sentence. Here k = x% is short for k = x% of the number of words in sentence.
Table 4:
Atlack Method
k- 57%
k- 110%
k 135%
k- 20%
k- 25%
k- -10%
TEXTFOOLER
14.56
42.69
62.79
75.41
82.84
85.67
BEST-ATTACK
19.40
43.99
72.35
$4.55
36.44
92.03
BAE R
20.11
53.60
70.65
$1.31
36.62
$9.03
GREEDY
15.04
89.23
63.45
$0.54
87.79
90007
HoTFLIP
9.26
27.48
39,00
49.65
56.07
59.96
BBA
17.45
48.3 29
68.0 66.16
79.36
85 56
89.59
TEXTGRAD
31.56
56.78
77.73
$8.74
93.51
95.1
Other experiment results. In Appendix tion study, and human evaluation. In nutshell, we show that TEXTGRAD achieves graceful attack transferability. The optimization techniques including random restarting and randomized sampling help boost the attack performance. For human evaluation, TEXTGRAD outperforms BAE-R, R, which performs the best in automatic quality evaluation.
B we further include the attack transferability, abla-
8

---

Published as a conference paper at ICLR 2023
Table 5: Robustmess evaluation of different adversarial training methods SST-2 dataset. The performance measured by clean accuracy (%) and robust accuracy (%) under different attack types. We also report the average robust accuracy (Avg RA) over different attack types.
Method BERT- AERT-ATTACK
Model
Clean Accuracy
TEXTGRAD
TEXTFOOLER
BAE-R
AdvGLUE
Avg RA
6.04
15.98
12.63
12.47
30.55
15.53
PGD-AT
92.31
10.15
23.72
12.52
15.49
36.80
19.73
INFOBERT
92.86
6.47
18.40
9.28
12.80
28.47
15.08
TENTFOOLER-AN
88.16
16.03
49.70
20.81
19.82
54.86
32.24
TEXTGRAD-AT
27.62
41.13
53.47
41.34
TEXTGRAD-TRAN TRAADS
81.49
55,91
35.53
32,02
39.43
51.38
42,85
6 ADVERSARIAL TRAINING WITH TEXTGRAD
In this section, we empirically show that TEXTGRAD-basee AT (termed TEXTGRAD-AT) provides an effective adversarial defense comparing to other AT baselines in NLP. We focus on robustifying BERT model on SST-2 dataset, and set the train-time iteration number of TEXTGRAD to 5 and restart time to During evaluation, we set TEXTGRAD with 20 attack iterations and 10 restarts.
Table makes detailed comparison between TEXTGRAD-AT with 6 baselines including ① stan- dard training (ST), PGD-AT (Madry et al.] 2018), @ FREE-LBB (Zhu et al.] 2019), + IN- $ ASCC (Dong et al. 2021), and @ "EETTFOOLFR-XT We re-
FOBERT (Wang et al. mark that the AT variants 2 generate adversarial perturbations against the continuous feature representations of input texts rather than the raw inputs at the training phase. By contrast, ours and TEXTFOOLER-AT generate adversarial examples in the discrete input space. As will be evident later, TEXTFOOLER-AT is also the most competitive baseline to ours. Besides TEXTGRAD-ATI propose TEXTGRAD- -RADEE by integrating TEXTGRAD with TRADES (Zhang et al. 2019b), which is commonly used optimize the tradeoff between accuracy and robustness. See more implementation details in Appendix B At testing time, four types of NLP attacks (TEXTGRAD TEXTFOOLER, BERT-ATTACK, and BAE-R) are used evaluate the robust accuracy of models acquired by the aforementioned training methods.
2020).
also
Our key observations from Table summarized below. First, the models trained by TEXTGRAD- AT and SEETGGRAD-TRADER achieve the best robust accuracy in nearly all settings. The only ex- ception is the case of TEXTFOOLER unseen attack for our approaches but it is known for TEXTFOOLER-A trast, if TEn-TEXA and non- TEXLFO attacks are used for robustness evaluation, then our methods outperform TEXTFOOLER FR-T- by a substantial margin (e.g., robust enhancement un- der BAE-R). Second, we evaluate the performance of different robust training methods under the AdvGLUE (Wang et al.) 202T) benchmark. We observe that TEXTGRAD-AT and TEXTGRAD- TRADES perform better than the baselines 2-3, while TEXTFOOLER-A" is slightly better than However, this is predictable since AdvGlue uses TEXTFOOLER to generate the adversarial dataset (Wang et al. 2021). As shown by the average robust accuracy in T methods are the only ones defend against wide spectrum of attack types. Third, our methods trade off an accuracy drop for robustness improvement. However, the latter is much more significant than the former. For example, none of baselines can defend against TEXTGRAD. while our improvement in robust accuracy is over 35% compared to TEXTFOOLER-AT We further show that robust accuracy converges quickly in two epoch training in Appendix F demonstrating that TEXTGRAD-AV can be used to enhance the robustness of downstream tasks in an efficient manner.
vs. the TEXTFOOLER attack, since TEXTFOOLER is an
during training. By con-
of the attack methods
7 CONCLUSION
In this paper, we propose TEXTGRAD, effective attack method based on gradient-driven opti- mization in NLP. TEXTGRAD not only achieves remarkable improvements in robustness evaluation but also boosts the robustness of NLP models with adversarial training. In the future, will consider how to generalize TEXTGRAD to other types of perturbations, such as word insertion or deletion, further improve the attack performance.

---

Published as a conference paper at ICLR 2023
8 ACKNOWLEDGMENT
The work of Bairu Hou, Jinghan Jia, Yihua Zhang, Sijia Liu, and Shiyu Chang was partially sup- ported by National Science Foundation (NSF) Grant IIS 220-220 The computing resources used in this work were partially supported by the MIT-IBM Watson AI Lab.
REFERENCES
Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai- Chang. Generating natural language adversarial examples. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 2890-2896, 2018.
Anish Athalye, Nicholas Carlini, and David Wagner. Obfuscated gradients give false sense of security: Circumventing defenses to adversarial examples. arXiv preprint arXiv: 1xXiv:88020044000 2018.
Yoshua Bengio, Nicholas Léonard, and Aaron Courville. Estimating propagating gradients through stochastic neurons for conditional computation. arXiv preprint arXiv: 1wXiv:1308.34. 338.3432, 2013.
Stephen Boyd, Stephen Boyd, and Lieven Vandenberghe. Convex optimization. Cambridge uni- versity press, 2004.
Nicholas Carlini and David Wagner. Towards evaluating the robustness of neural networks. In JEEE Symposium Security and Privacy (S&P), pp. 39-57 57. IEEE, 2017.
Yair Carmon, Aditi Raghunathan, Ludwig Schmidt, John C Duchi, and Percy S Liang, Unlabeled data improves udversarial robustness, In Advances in Neural Information Processing Systems, pp. 11190-1 1201, 2019.
Tianlong Chen, Sijia Liu,, Shiyu Chang, Yu Cheng, Lisa Amini, and Zhangyang Wang. Adversarial robustness: From supervup pre-training to fine-tuning. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp. 699-708, 2020.
Minhao Cheng, Thong Le, Pin-Yu Chen, Jinfeng Yi, Huan Zhang, and Cho-Jui Hsieh. Query- efficient hard-label black-box attack: An optimization-basee approach. arXiv preprint 1rXiv:1807.04457, 2018
Shuyu Cheng, Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Improving black-box adversarial attacks with transfer-based prior. arXiv preprint arXiv: 1006:9900000 19919, 2019.
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free free attacks. In International Conference on Machine Learning, pp. 2206- 2216. PMLR, 2020.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171-4186, 2019.
Xinshuai Dong, Anh Tuan Luu, Rongrong Ji, and Hong Liu. Towards robustness against natural language word substitutions. In 9th International Conference Learning Representations, ICLR 1021, OpenReview.net, 2021.
Yinpeng Dong. Fangzhou Liao, Tianyu Pang. Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li. Boost- ing adversarial attacks with momentum. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 9185-9193. 2018.
Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu. Evading defenses to transferable adversar- ial examples by translationiivaaiinl attacks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4312-4321, 2019.
10

---

Published as a conference paper at ICLR 2023
Yinpeng Dong, Qi-An Fu, Xiao Yang, Tianyu Pang, Hang Su, Zihao Xiao, and Jun Zhu. Benchmark- ing adversarial robustness on image classification. Proceedings of the IEEE/CVE Conference Computer Viston and Pattern Recognition, pp. 321-331, 2020.
Javid Ebrahimi, Anyi Rao, Daniel Lowd, and Dejing Dou. Hotflip: White-bor box adversarial exam- ples for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguisties (Volume 2: Short Papers), pp. 31-36, 2018.
Siddhant Garg and Goutham Ramakrishnan. Bae: Bert-based adversarial examples for text clas- sification. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) P), pp. 6174-6181, 2020.
lan J Goodfellow, Jonathon Shlens, and Christian Szegedy. Explaining and harnessing adversarial examples. arXiv preprint arXiv: 142.6557,,, 2014.
Chuan Guo, Alexandre Sablayrolles, Hervé Jégou, and Douwe Kiela. Gradient-basel adversarial attacks against text transformers. In Proceedings the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 5747-5757, 2021.
Eric Jang, Shixiang Gu. and Ben Poole. Categorical reparameterizzatoon with gumbel-softmax, arXiv preprint arXiv:1611 111001114 144, 2016.
Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. Is bert really robust? a strong baseline for natural language attack on text classification and entailment. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 8018-8025, 2020.
James Kennedy and Russell Eberhart. Particle swarm optimization. In Proceedings of ICNN'95- international conference on neural networks, volume 4, pp. 1942-1948. IEEE, 1995.
Alexey Kurakin, Ian Goodfellow, and Samy Bengio. Adversarial examples in the physical world. arXiv preprint arXiv: .02533, 2016.
Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Sori- cut. Albert: A lite bert for self-supervised learning of language representations. In International Conference on Learning Representations, 2020.
Deokjae Lee, Seungyong Moon, Junhyeok Lee, and Hyun Oh Song. Query-efficient and scalable black-box adversarial attacks on discrete sequential data via bayesian optimization. In Interna- tional Conference on Machine Learning, pp. 12478-12497. PMLR, 2022.
Dianqi Li, Yizhe Zhang, Hao Peng, Liqun Chen, Chris Brockett, Ming-Ting Sun, and William B Dolan. Contextualized perturbation for textual adversarial attack. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 5053-5069, 2021a.
J Li, S Ji, T Du, B Li, and T Wang. Textbugger: Generating adversarial text against real-world applications. 26th Annual Network and Distributed System Security Symposium, 2019.
Linyang Li, Ruotian Ma. Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. Bert-attack: Adversarial attack against bert using bert. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 6193 6202, 2020.
Zongyi Li, Jianhan Xu, Jiehang Zeng, Linyang Li, Xiaoqing Zheng, Qi Zhang, Kai-Wei Chang, and Cho-Jui Hsich. Searching for effective defender: Benchmarking defense against adversarial word substitution. arXiv preprint arXiv:2108. 12777, 202
Y. Liu, X. Chen, C. Liu, and D. Song. Delving into transferable adversarial examples and black-bor attacks. arXiv preprint arXiv:1611.02777 2016.
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv: 1907. / 2019.

---

Published as a conference paper at ICLR 2023
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. In International Conference on Learning Representations, 2018.
Zhao Meng and Roger Wattenhofer. A geometry- attack for generating natural language adversarial examples. In Proceedings of the 28th International Conference Computational Linguistics, pp. 6679-6689, 2020.
George A Miller. Wordnet: lexical database for english. Communications of the ACM, 38(11):
39 39-41, 1995.
Seyed- Moyee- Moosavi-Dezfooli, Alhussein Fawzi, Omar Fawzi, and Pascal Frossard. Universal adversarial perturbations. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1765-1773, 2017.
John Morris, Eli Lifland, Jin Yong Yoo, Jake Grigsby, Di Jin, and Yanjun Qi. Textattack: A frame- work for adversarial attacks, data augmentation. and adversarial training in nlp. In Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing: System Demon- strations, pp. 119-11991100 119-126, 2020.
Nicolas Papernot, Patrick McDaniel, Somesh Jha, Matt Fredrikson, Z Berkay Celik, and Ananthram Swami. The limitations of deep learning in adversarial settings. In IEEE European Symposium on Security and Privacy ( (EuroS& P), pp. 372-387. IEEE, 2016.
Fanchao Qi, Chenghao Yang, Zhiyuan Liu, Qiang Dong, Maosong Sun, and Zhendong Dong. Open- hownet: An open sememe- based lexical knowledge base. arXiv preprint arXiv: htt::/900 /901_09957 2019.
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. Generating natural language adversarial examples through probability weighted word saliency. In ACL, 2019.
Ali Shafahi, Mahyar Najibi, Amin Ghiasi, Zheng Xu, John Dickerson, Christoph Studer, Larry S Davis, Gavin Taylor, and Tom Goldstein. Adversarial training for free! arXiv preprint 19X1::1904.12433 2019.
Chenglei Si, Zhengyan Zhang, Fanchao Qi, Zhiyuan Liu, Yasheng Wang, Qun Liu, and Maosong Sun. Better robustness by more coverage: Adversarial and mixup data augmentation for ro- bust finetuning. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 1569-1576. Association for Computational Linguistics, 2021. doi: 1/2221251//2221 madings-acl.115 137.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 conference empirical methods in natural language pro- cessing, pp. 1631-1642. 2013.
Christian Szegedy, Wojeiech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. Intriguing properties of neural networks. arXiv preprint 1rXiv:1312.6199
2013.
Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman. Glue: A multi-task benchmark and analysis platform for natural language understanding. In International Conference Learning Representations, 2018.
Boxin Wang, Shuohang Wang, Yu Cheng, Zhe Gan, Ruoxi Jia, Bo Li, and Jingjing Liu. Infobert: Improving robustness of language models from information theoretic perspective. In Interna- tional Conference on Learning Representations, 2020.
Boxin Wang, Chejian Xu, Shuohang Wang, Zhe Gan, Yu Cheng, Jianfeng Gao, Ahmed Hassan Awadallah, and Bo Li. Adversarial glue: A multi-task benchmark for robustness evaluation of language models. ArXiv, abs/2111.00 1.02840, 2021.
12

---

Published as a conference paper at ICLR 2023
Adina Williams, Nikita Nangia, and Samuel Bowman. A broad-coverage challenge corpus for sen- tence understanding through inference. In Proceedings of the 2018 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies,
Volume (Long Papers), pp- 11 1112-1122, 2018.
Eric Wong, Leslie Rice, and J. Zico Kolter. Fast is better than free: Revisiting adversarial training. In International Conference on Learning Representations, 2020.
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang. Zhou Ren, and Alan L Yuille.
Improving transferability of adversarial examples with input diversity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2730-2739, 2019.
Han Xu, Yao Ma, Haochen Liu, Debayan Deb, Hui Liu, Jiliang Tang, and Anil Jain. Adversarial attacks and defenses in images, graphs and text: review. arXiv preprint arXiv: vrXiv:190.008072 2019a.
Kaidi Xu, Hongge Chen, Sijia Liu, Pin-Ye Yu Chen, Tsui-Wei Weng, Mingyi Hong, and Xue Lin. Topology attack and defense for graph neural networks: An optimization perspective. In Interna- tional Joint Conference on Artificial Intelligence (IJCAI), 2019b.
Kaidi Xu, Sijia Liu, Pu Zhao, Pin-Yo Yu Chen, Huan Zhang, Quanfu Fan, Deniz Erdogmus, Yanzhi Wang, and Xue Lin. Structured adversarial attack: Towards general implementation and better interpretabilitty. In International Conference on Learning Representations, 2019c.
Yuan Zang, Fanchao Qi, Chenghao Yang. Zhiyuan Liu, Meng Zhang, Qun Liu, and Maosong Sun. Word-level textual ad adversarial attacking as combinatorial optimization. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6066-6080, 2020.
Dinghuai Zhang, Tianyuan Zhang, Yiping Lu, Zhanxing Zhu, and Bin Dong. You only propagate once: Accelerating adversarial training via maximal principle. arXiv preprint arXiv: 1905:::905000088
2019a.
Hongyang Zhang, Yaodong Yu, Jiantao Jiao, Eric Xing. Laurent El Ghaoui, and Michael Jordan. Theoretically principled trade-off between robustness and accuracy. International Conference on Machine Learning, 2019b.
Huangzhao Zhang, Hao Zhou, Ning Miao, and Lei Li. Generating fluent adversarial examples for natural languages. In Proceedings of the 57th Annual Meeting of the Association for Computa- 2019c.
tional Linguistics, pp.
Xiang Zhang, Junbo Zhao, and Yann LeCun. Character-level convolutional networks for text clas- sification. Advances in neural information processing systems, 28:649-657,220 657,
Yihua Zhang, Guanhua Zhang, Mingyi Hong, Shiyu Chang, and Sijia Liu. Revisiting and advancing adversarial training through the lens of bi-level optimization, submitted NeurlPS, 2021.
Chen Zhu, Yu Cheng, Zhe Gan, Sigi Sun, Tom Goldstein, and Jingjing Liu. Freelb: Enhanced adversarial training for natural language understanding. In International Conference on Learning Representations, 2019.
Junhua Zou, Zhisong Pan, Junyang Qiu, Xin Liu, Ting Rui, and Wei Li. Improving the transfcrability of adversarial examples with resized fiiizedddivereeiinpiee diversity-ensembbl and region fitting. In European Conference on Computer Vision, pp. 563-579. Springer, 2020.
13

---

Published as a conference paper at ICLR 2023
A PROJECTION OPERATIONS IN PGD FRAMEWORK
The projection operations lIc and Nc, in Equation 4 are demonstrated below:
Proposition A.1. Given C₁ {2) 12 I k 2 E [0, 3 the project operation for a data point z' with respect to C1 is
P [x]
if1" P Plo1) Sk: k; Po..1 [z' [1] if > and 1 Plo.1|
Ile, (z')
(8)
111 k,
and for C2 = {ü, ú; = 1. úr E [0, 11m the project operation for a data point ui with respect to C2 is:
cc. [ū]] = P(o,a [ū] -1
(9)
where v is the roof of 12t P Po.1 - i v1 = 1, (. is an element-wise projection operation,
P(o.1(2)== = x i E [0, 1). 0 if < 0, and if >
similar derivation of the projection has been shown (Xu al. 2019b).
B IMPLEMENTATION DETAILS
In this section, we include the implementation details of victim models, hyper-parameters for base- lines, and training details.
Training of victim models We run our experiments on the Tesla V100 GPU with 16GB memory. We fine-tune the pre-trained BERT-base-uncased model on each dataset with batch size of 32, a learning rate of 2e-5 for 5 epochs. For RoBERTa-large and ALBERT-xXlarge2,, we use batch size of 16 and learning rate of 1e-5. For the robust models, we use the implementation of (Li et al.) 2021b). Each model is trained for 10 epochs with a learning rate of 2e-5 and batch size of 32. Specifically, for ADA and MIXADA, we perturb the whole training dataset for augmentation. For MIXADA, we use a 2.0 in the beta distribution and mix the hidden representations sampled from layer {7.95 9. 12}. For PGD-AT, we use step size = 3e-2 and number of steps m 5 for both SST and RTE dataset. For FREE-LB, we use step size a le-1 and number of steps m on SST-2 dataset and a = 3e-2, m = 3 on RTE dataset. For INFOBERT, the step size is 4e-2 and number of steps is for both two datasets. Finally, we use a 10, B = and run for 5 steps to generate perturbation for ASCC. For datasets that the labels of testing dataset are not available (MNLL, RTE, QNLI), we randomly sample 10% of training dataset as validation dataset and use the original validation dataset for testing. For the AG News dataset where the validation set is not available, we use the same way to generate the validation dataset.
Attack Implementation Regarding the hyper-parameters of TEXTGRAD, utilize 20-step PGD for optimization and fix the number of sampling R in each iteration to be 20. We adopt learning rate of 0.8 for both ž and ú, and normalize the gradient gı. and g2 to unit norm before the descent step. After PGD optimization, we sample 20 different ă and ú pairs for a single input To determine the final adversarial example of x, we select the one with the minimum PPI. measured by the GPT-2 language model (Radford et al. 2019) among all successful attacks. Although such a post-processing approach via multiple sampling introduces additional computational overhead, ensures the high quality of generated attacks. If all 20 sampled attacks fail to fool the victim model, we allow TEXTGRAD to restart the attack process with a different initialization of 2 and u. Restart with different initializations is standard setting used in white-box PGD attacks for imagery data. Here, we set the maximum number of restarts to 10. However, empirically we find that most of the attacks will be successful with single without restart. The average number of restarts in our experiment is around 1.6-1.8. To ensure query- based basclines approaches with a large enough query budget, we the maximum number queries for them be 2000.
Attack parameters For baselines, the attack budget is set 25% of the total word numbers in a sentence. Since TEXTGRAD modifies the sentence in teken- we set the maximum to- kens that TEXTGRAD modify be 25% of the total word numbers in sentence. By doing
14

---

Published as a conference paper at ICLR 2023
so, TEXTGRAD uses the same or less attack budget than baselines, and ensures the fair compari- son. We use the implementation of the TextAttack (Morris et al. 2020 library for TEXTFOOLER, BERT-ATTACK, BAE-R and BBA. The number of candidate substitutions for each word is 50 in TEXTFOOLER and BAE-R. For MERRT-ATACK, we set this value to 48 following the default set- ting. For BBA is consistent with the original paper. We use our candidate substitute generation method in HoT- FLIP and GREEDY and the number of candidate substitution tokens for the two baselines also 50. For natural language inference datasets (MNLI-m, QNLI, RTE), we restrict all the attackers to only perturb the hypothesis.
the same candidate substitution generation method TEXTFOOLER which
GBDA purely rely on soft constraints during attack instead of hard attack budgets (i.e., the maxi- mum perturbation which 25% of the total word numbers sentence for all the other methods including TEXTGRAD). Furthermore, the soft constraints in GBDA rely on an extra causal language model (such as GPT-2 (Radford et al.) (019)) during attacking. When attacking masked language models such as BERT, RoBERTa, and ALBERT, one needs to train a causal language model that shares the same vocabulary with the victim model, making their method less flexible. Since the offi- cial implementation of GBDA only provides causal language model that has the same vocabulary table as BERT, it only support attacking BERT and cannot be used to evaluation the robustmess of ROBERTa/ALLBER
in our experiments in Table 2 Therefore, we only report the experiment results when the victim model is BERT. We use the default hyper-parameters of GBDA in their original paper (100 attack iterations with batch size 10 and learning rate 0.3: Aperp 1). For Xaims we follow the paper's setting to cross-validate it in range [20, 200). Finally, for SST-2 and MNLI-m datasets, we use Asm = 50; for other dataset we find the attack example quality very low given a small Asim- Therefore we use Agim 200 for the other datasets.
For TEXTGRAD, we also consider more attack constraints. Firstly, we conduct part-of-speech checking before attacking and only nouns, verbs, adjectives, and adverbs can be replaced. Sec- ondly, after generating candidate substitutions, we use WordNet (Miller) 1995) to filter antonyms of original words to avoid invalid substitutions. Thirdly, stop-words will not be considered during attacking. which means we will not substitute original words that are nal words with stop-words. Finally, considering the tokenization operation in pre-trained language models, we filter out sub-words in the generated candidate substitutions before attacking to further improve the quality of adversarial examples.
or replace origi-
C ATTACK TRANSFERABILITY
We compare the attack transferability of various attack methods in this section. cobpeessompares the attack transferability of various attack methods given different pairs of source victim model used for attack generation and a targeted model used for transfer attack evaluation, where the considered models (BERT, RoBERTa, and ALBERT) are normally trained on SST-2. As we can see, transfer attacks commonly lead to the ASR degradation. However, compared to baselines, TEXTGRAD yields better transfer attack performance in nearly all the cases. It is also worth noting that there exists large ASR drop when NLP attacks transfer to an unseen model. Thus, it requires more in-depth future studies to tackle the question of how to improve transferability of NLP attacks.
D ABLATION STUDIES
Table demonstrates the usefulness of proposed optimization tricks: random restarting and random- ized sampling, where the former has been commonly used in generating adversarial images (Madry ct al. 2018), and the latter critical calculate input gradients. As we can see, both optimization tricks play positive roles in boosting the attack performance of TEXTGRAD. However, the use of randomized sampling seems more vital, as evidenced by the larger ASR drop (6%-11%% when using TEXTGRAD w/o randomized sampling.
We also study the performance of variants of baselines. Specifically, BBA can be combined with different substitution generation methods. In our main experiments, we evaluate the attack perfor- mance of BBA using the substitution generation method of TEXTFOOLER, which adopts the word embedding similarity to generate substitutions. In Table we test the performance of BBA when using the substitution generation method of BAE-R.
15

---

Published as a conference paper at ICLR 2023
Table 6: NLP attack transferabilitty Attacks generated from source victim models (BERT, RoBERTa, and ALBERT) are evaluated the same set models. The experiment is conducted on the SST-2 dataset and all the models are normally trained.
Surrogate Model
Attack Method
Victim Model
BERT
RoBERTa
ALBERT
BERT
82.84
37.83
37.84
TEXTFOOLER
RoBERTa
25,47
69.38
31.07
ALBERT
20.69
23.46
69.77
BERT
86,44
39.38
42.78
BERT-ATTACK
RoBERTa
36.50
79.34
42.84
ALBERT
32.02
33.14
82.73
BERT
86.62
55.86
$7.10
BAE-R
RoBERTs
55.54
85,92
57.55
ALBERT
53.36
58.15
85,80
BERT
87.79
59.70
$7.48
Greedy
RoBERTa
47.52
90,43
57.77
ALBERT
44.45
55.59
88.66
BERT
93.51
64.92
55.13
TEXTGRAD
RoBERTa
54.10
96.45
$7.39
ALBERT
49.68
58.10
93.51
Table 7: Sensitivity analysis of random restarts and sampling. trained BERT model.
TEXTGRAD is conducted over normally
Attack Method
SST-2
MNL.Im
RTF
QNLI
AG News
TextGrad
93.51
94.08
71.91
70.48
74.51
restart
90.27
91.90
67.42
65.26
68.87
n.O. restart & sampling
78.30
85.44
58.99
55.05
$4,23
From the experiment results, we highlight the following conclusions. First, advanced attack method, BBA outperforms TEXTFOOLER and BAE-R that use greedy search during attack generation when using the same substitution generation methods. The substitution generation method of BAE- also provides more advantages for attack- ing. Secondly, TEXTGRAD consistently out- performs black-box basclines, indicating the superiority of the gradient-driven optimiza- tion method when generating adversarial ex- amples.
Table 8: Performance proposed TEXTGRAD and base- line methods against normally trained victim models on SST-2 and MNLI-m datasets evaluated by attack success rate. The performance of BBA is evaluated when us- ing substitution generation methods of TEXTFOOL and BAF.R R respectively.
SST-2
MNLI-m RoBERTa
Atlack Method
RoBERTa
TEXTFOOLER
82.84
69.38
74.82
67.33
BBA + TEXTFOOLER
85.96
81.51
82.86
78.44
BAE-R BBA + BAE-R
86.62
85.92
87.00
84,56
90.80
91.2
93.61
93.77
TEXTGRAD
93.51
96.45
94.08
95.44
E PRELIMINARY HUMAN EVALUATION
Besides automatic evaluation, we also conduct human evaluations for justifying the validity of ad- versarial examples. Here an adversarial example regarded as valid when its label annotated by human annotator is consistent with the ground-truth label of the corresponding original example. During the evaluation, each annotator is asked to classify the sentiment labels (positive/negative) of the given sentences, thus requiring domain knowledge. Note that the ground-truth label is not provided. The following instruction is displayed to the annotator during annotation:
Please classify the sentiment of the movie review displayed above. Answer Qf you think the sentiment in that movie review is negative and answer I if positive.
Your answer (0/1):
16

---

Published as a conference paper at ICLR 2023
Our method is compared with BAE, the baseline with the best attack quality according to Table and Specifically, We randomly sample 100 examples from the SST-2 dataset that are successfully attacked by both TEXTGRAD and BAE-R, resulting in 200 adversarial examples in total. These adversarial examples are randomly shuffled and annotated by four annotators. We compute the validity ratios according the annotations of each annotator well the average validity ratio. Finally, the average validity ratios are 43.5% for TEXTGRAD and 39.75% for BAE-R, showing that the quality of adversarial examples generated by TEXTGRAD is slightly better than the baseline. We will also conduct more large-scale human evaluations in the next step.
F CONVERGENCE ANALYSIS
90
In Figure 2 we further show the convergence trajectory of us- ing TEXTGRAD-AT fine-tune pre-trained BERT, given by clean accuracy and robust accuracy vs. the training epoch number. We observe that the test-time clean accuracy de- creases as the training epoch number increases, implying the accuracy-robustnee tradcoff. We also note that the test-time robust accuracy quickly converges in two epochs. This sug- gests that benefiting from a large-scale pre-trained model, TEXTGRAD-AT be used to enhance the robustness of downstream tasks in an efficient manner.
80-
70
60
50
Training Epoch
Figure 2: Clean accuracy and robust accuracy SST-2 train/test dataset during adversarial training.
G LIMITATION AND SOCIETAL IMPACT
While the proposed TEXTGRAD can both improve the robustness evaluation and boost the adver- sarial robustness of NLP models, acknowledge that there are still some limitations that need to improve in the future. Firstly, TEXTGRAD crafts adversarial examples by synonym substitution. cannot handle other perturbations such as word insertion, word deletion, and sentence-level modi- fication. We hope to extend our convex relaxation optimization to more perturbations in the future further promote the performance. Secondly, how to ensemble TEXTGRAD with other types of attacks (for example, black-box baselines) to form a more powerful attack is not explored. Given success of AutoAttack (Croce & Hein 2020) computer vision, is also plausible to build an ensemble attack in NLP.
With the application of large pre-trained NLP models, the vulnerability of pre-trained NL.P models also raises concerns. We admit that TEXTGRAD may be employed design textual adversarial examples in real life. thus resulting in security concerns and negative outcomes. However, one can still adopt TEXTGRAD for robustness evaluation and adversarial training as to improve the security and trustworthiness of NLP systems.
17


Reviews:
[{"review_id": "UQmKDx_lly3", "reviewer": "Reviewer_iGRm", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a white-box attack method in NLP that can achieve a high attack success rate and high quality. According to the results in the paper, the proposed method is efficient and effective.", "summary_of_the_review": "In summary, the proposed method is novel and effective. \nOn the other hand, I have raised some concerns about the experimental design and results, which may hurt the paper's solidness. The detailed author response has addressed my concerns partly.\n", "strength_and_weaknesses": "Strengths:\n1)\tThe method is based on an effective convex relaxation method, which is a novel approach to co-optimize the continuously-relaxed site selection and perturbation variables.\n2)\tAccording to the results in the paper, the proposed method is able to generate adversarial examples with high quality and high attack success rate in an efficient way.\n\nWeaknesses:\n1)\tHow to maintain a balance between C&W-type attack loss and text fluency regularization?\n2)\tIn Table 5, TEXTGRAD-AT performs much worse than TEXTFOOLER-AT in terms of clean accuracy, while TEXTGRAD has a lower PPL than TEXTFOOLER. I think it is a bit strange why high-quality adversarial examples cannot achieve better clean accuracy.\n3)\tIn Eq (5), how to satisfy the constraints in Eq (2)? For example, 1^T z ≤ k, 1^T u = 1.\n4)\tThe ablation experiments are inadequate and the role of text fluency regularization and C&W-type attack loss was not analyzed.\n5)\tIn Figure 1, why does TEXTGRAD with 1-step attack iteration perform better than the majority of baseline attacks, and is it caused by the preprocessing of substitution words? \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "EaZLHnySJ6", "reviewer": "Reviewer_eerx", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Unlike the computer vision domain, it is hard to generate text adversarial examples efficiently by using the first-order projected gradient descent due to the discrete nature of texts. In this study, a new attack generator, named TextGrad, was proposed to co-optimize the site selection (where to replace tokens) and the substitute choice. To circumvent the difficulty caused by the non-differentiability of discrete optimization, a convex relation method and the corresponding sampling-based solving algorithm has been proposed to map from the continuous optimization space to the discrete token perturbations. The experimental results show that TextGrad achieved some improvements in the attack success rate while the generated adversarial examples have lower perplexity scores.", "summary_of_the_review": "A text adversarial example generation method, named TextGrad, was proposed in this study, in which the site selection (where to replace tokens) and the substitute choice can be co-optimized in a discrete optimization problem. A sampling-based method was also proposed to solve this discrete optimization problem by relaxing discrete constraints into their convex counterparts and using gradient-driven optimization. However, the experimental results show that the models trained with TextGrad suffer a significant drop in clean accuracy and TextGrad requires about three times the computation time than existing greedy-based methods. The similarities and differences between this work and (Zhou et al., 2021, Dong et al., 2021) also need to be discussed.", "strength_and_weaknesses": "Strength:\n\n(1) It is nice to formulate the problems of both the site selection (where to replace tokens) and token modification (which tokens should be selected to replace the original ones) for text adversarial example generation as a discrete optimization problem with convex constrains. \n\n(2) A sampling-based method was also proposed to solve the optimization problem by relaxing discrete constrains into their convex counterparts and using gradient-driven optimization.\n\n(3) They experimentally show that TextGrad achieves improvements in both the attack success rate and the perplexity score over exiting adversarial example generation methods.\n\nWeakness:\n\n(1) A similar idea has been investigated in (Zhou et al., 2021) and (Dong et al., 2021), which relaxes a set of discrete points (a word and its synonyms) to a convex hull spanned, and uses a convex hull formed by a word and its synonyms to capture word substitutions although the goal of their studies is to improve the adversarial robustness of text classification models instead of generating adversarial examples. It would be good to add some text to discuss the similarities and explain the differences.\n\n(2) Although the experimental results show that the TextGrad can be used in adversarial training to improve the robustness of NLP models, the models robustly trained with TextGrad suffer a significant drop in clean accuracy (see table 5). \n\n(3) Many random trials are required to solve the optimization problem via importance sampling, which make the problem method takes about three times the computation time than the greedy-based algorithms.\n\n[1] Zhou, Yi, et al. Defense against synonym substitution-based adversarial attacks via Dirichlet neighborhood ensemble. ACL, 2021.\n\n[2] Dong, Xinshuai, et al. Towards robustness against natural language word substitutions. ICLR, 2021.\n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "DFU1wvU3Qs", "reviewer": "Reviewer_J6Pr", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "In contrast to the computer vision (CV) domain where the first-order projected gradient descent (PGD) is used as the benchmark approach to generate adversarial examples for robustness evaluation, there lacks a principled first-order gradient-based robustness evaluation framework in NLP. To bridge the gap, this paper proposes TEXTGRAD, a new attack generator using gradient-driven optimization, supporting high-accuracy and high-quality assessment of adversarial robustness in NLP.", "summary_of_the_review": "The motivation of this paper is very clear. The proposed method sounds and the paper is in a good shape. Experiment part is solid.", "strength_and_weaknesses": "Strength:\n1. The motivation of this paper is very clear. There are 2 challenges lie for bridging the gap between CV PGD and NLP PGD. (1) the discrete nature of textual inputs together with the strong coupling between the perturbation location and the actual content. (2) the additional constraint that the perturbed text should be fluent and achieve a low perplexity under a language model. These challenges make the development of PGD-like NLP attacks difficult.\n2. The proposed method sounds and the paper is in a good shape. This paper develops an effective convex relaxation method to co-optimize the continuously-relaxed site selection and perturbation variables, and leverage an effective sampling method to establish an accurate mapping from the continuous optimization variables to the discrete textual perturbations.\n3. Experiment part is solid which shows that the proposed method not only achieves remarkable improvements in robustness evaluation\nbut also boosts the robustness of NLP models with adversarial training. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "9btDtv9ZCDu", "reviewer": "Reviewer_7WNo", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper presents a method of white-box adversarial attacks against text classification systems. Given a classification model, the algorithm involves taking an input example and transforming it to construct an adversarial example by (1) selecting a subset of positions to replace, (2) for each selection position, selecting a word from a given set of words for that position while constraining these selections to be lead to a fluent sentence. This work executed this idea by formulating this search problem as continuous optimization with the primary goal to flip the model label for x with a constraint on fluency of the output. On several text classification tasks with 3 different models, the authors report better performance in terms of generating sentences that are more successful at flipping the label compared to that of original x and have lower perplexity. On a small scale human evaluation, the generated adversarial samples seem slightly better than baselines at preserving the original sentiment.", "summary_of_the_review": "The presented algorithm has interesting ideas for generating adversarial examples for text classification systems and shows improvement on attack metrics, however one major criterion for evaluating the adversarial examples is label preservation on which the authors provide a very small scale analysis which is not entirely convincing of the utility of the method. Finally, I would add that I am not an expert in this space and provided a rebuttal, willing to revise my score. ", "strength_and_weaknesses": "Strengths:\n1. The proposed algorithm clearly outperforms baselines on attack success rate as well as perplexity. Ablations show the importance of each setup. \n2. The paper is written well and easy to follow in most parts. \n\n\nWeaknesses:\n1. The algorithm seems to be lacking a crucial aspect of preserving the original label. The proposed algorithm replaces a subset of words in the original sentence with some candidates provided by MLMs. This has glaring issues wrt to the definition of an adversarial example. For any position in the sentence, an MLM can easily generate antonyms as candidates which will result in a fluent sentence but flip the label. So it would not really be an adversarial example. The human evaluation does show a slight improvement in label preservation over baselines but it is still low at 43%. Without label preservation guarantees, the evaluation of the attack success rate is meaningless. The evaluation setup which shows: \"of the adversarial examples that preserve label, how many led to an attack success\" would be more meaningful.\n2. The fluency criterion used as part of the loss is conceptually not convincing either. The author chose to use MLM probabilities as a way to evaluate fluency. MLMs by design are not suited to measure the plausibility of full sequences. Since the final evaluation is done by perplexity, a fluency criterion defined by an autogressive (aka causal) LM would be better suited as an objective. Additionally, human evaluation should also involve evaluating fluency.", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: poster