Adversarial Text Generation by Search and Learning
Authors: Guoyi Li
Bingkang Shi
Zongzhen Liu
Dehan Kong
Yulei Wu
Xiaodan Zhang
Longtao Huang
Honglei Lyu
Venue: EMNLP 2023 Findings (2023)
Keywords: Adversarial robustness
Adversarial training
Unsupervised Text Generation

Abstract: Recent research has shown that evaluating the robustness of natural language processing models using textual attack methods is significant. However, most existing text attack methods only use heuristic replacement strategies or language models to generate replacement words at the word level. The blind pursuit of high attack success rates makes it difficult to ensure the quality of the generated adversarial text. As a result, adversarial text is often difficult for humans to understand. In fact, many methods that perform well in terms of text attacks often generate adversarial text with poor quality. To address this important gap, our work treats black-box text attack as an unsupervised text generation problem and proposes a search and learning framework for Adversarial Text Generation by Search and Learning (ATGSL) and develops three adversarial attack methods (ATGSL-SA, ATGSL-BM, ATGSL-FUSION) for black box text attacks. We first apply a heuristic search attack algorithm (ATGSL-SA) and a linguistic thesaurus to generate adversarial samples with high semantic similarity. After this process, we train a conditional generative model to learn from the search results while smoothing out search noise. Moreover, we design an efficient ATGSL-BM attack algorithm based on the text generator. Furthermore, we propose a hybrid attack method (ATGSL-FUSION) that integrates the advantages of ATGSL-SA and ATGSL-BM to enhance attack effectiveness. Our proposed attack algorithms are significantly superior to the most advanced methods in terms of attack efficiency and adversarial text quality.

Adversarial Text Generation by Search and Learning
Guoyi Li', Bingkang Shi', Zongzhen Liu', Dehan Kong², Yulei Wu',
Xiaodan Zhang', Longtao Huang', Honglei Lyu' 1 Chinese Academy of Sciences, Institute of Information Engineering, Beijing, China
2 Alibaba Group, Alibaba Artificial Intelligence Governance Laboratory, Beijing, China
3 University of Bristol, Department of Electrical and Electronic Engineering, Bristol, UK
[liguoyi, shibingkang, zhangxiaodan @iie.ac.cn
Abstract
text do affect human judgment but can deceive deep learning models. (Bender and Koller, 2020) pointed out that deep neutral models have defects understanding the meaning conveyed by language. Thus, generating adversarial samples has become a common method for evaluating the weakness and robustness of DNNs.
Recent research has shown that evaluating the robustness natural language processing mod- els using textual attack methods is significant. However, most existing text attack methods only use hcuristic replacement strategies or lan- guage models generate replacement words at the word level. The blind pursuit of high attack success makes it difficult to quality of the generated adversarial text. As result, adversarial text is often difficult for hu- understand. In fact, many methods that perform well in terms of text attacks often gen- erate adversarial text with poor quality. ad- dress this important gap, work treats black- box text attack as an unsupervised text genera- tion problem and proposes search and learn- ing framework for Adversarial Text Generation by Search and Learning (ATGSL) and develops three adversarial attack methods (ATGSL-SA, ATGSL-BM, ATGSL-FUSION) for black-box text attacks. first apply a heuristic search attack algorithm (ATGSL-SA) and linguistic thesaurus to generate adversarial samples with high semantic similarity, After this process, we train conditional generative model to learn from the search results while smoothing out search noise. Moreover, we design an efficient ATGSL-BM attack algorithm based on the text generator. Furthermore, we propose a hybrid at- tack method (ATGSL-FUSION) that integrates the advantages of ATGSL-SA and ATGSL-BM enhance attack effectiveness. Our proposed attack algorithms are significantly superior to the most advanced methods in terms of attack efficiency and adversarial text quality.
Existing malicious text generation algorithms can be classified into character-level attacks, sentence-level attacks, and word-level attacks. Character-level attacks (Belinkov and Bisk. 2017; Ebrahimi al., 2017) include the addition, dele- tion, replacement, and order exchange of charac- ters, which sacrifices the readability of the gen- crated text in exchange for the attack's success rate. Sentence-level attacks (Jia and Liang, 2017; Inui et al., 2019) regard the original input of the whole sentence a perturbation object, which of- ten makes a considerable difference between the generated text and the original input. is not easy to guarantee the quality of the generated text. Therefore, many studies focus on improving the attack success rate and the quality of the attack text generated by word replacement, hence word-level attacks.
the
Previous works mainly generate textual attacks on word replacement according to specific rules (Alzantot al., 2018; Ren et al., 2019; Jin et al., 2020; Bender and Koller, 2020; Zang al., 2020b; Yang et al., 2021a). Most of these models show good attack performance by using multiple lin- guistic constraints (e.g.. NER tagging and POS tagging) and a well-organized linguistic thesaurus (e.g., WordNet and HowNet). However, these works require extensive preprocessing, and the sub- stitution words selection heavily relics tags and cannot guarantee the fluency and grammaticality of adversarial samples. The other attack methods based on language models, such as BERT, can gen- crate contextual perturbations (Garg and Ramakr- ishnan, 2020; Li et al., 2020b; Yang et al., 2021b;
1 Introduction
Recent research has demonstrated that deep neu- ral networks (DNNs) vulnerable maliciously crafted adversarial text examples that can fool vic- tim models into making wrong predictions (Wang, 2018; Papernot et al., 2016a,b), These text exam- ples that add malicious perturbation to the original

---

Malik et al., 2021). These models ensure the pre- dicted token fits the sentence well but cannot pre- serve the semantic similarity (Yang et al., 2021a). For example, in the sentence "I feel [MASK]", pre- dicting the [MASK] as happy or sad is equally sensible but results in a sentiment analysis task. In order to improve these issues, recent research has focused on learning- based methods (Zang et al., 2020a; Lee et al., 2022; Sabir et al., 2021), aiming utilize model learning to improve the balance between the efficiency attack algorithms and the quality of adversarial lexts from the attack evalua- tion history.
attack effectiveness and the quality of adver- sarial texts. In addition, ATGSL-FUSION has the best attack success rate because it im- proves the impact of the initial conditions on the ATGSL-SA search.
Extensive experimental results show that our model significantly improves the existing state-of-the-ar models in adversarial text gen- eration in terms of attack efficiency and text quality.
Related Work
2.1 Textual Adversarial Attack
Inspired by these works, adversarial text genera- tion can be seen as an unsupervised text generation problem. Thus, we propose a new framework for Adversarial Text Generation by Search and Learn- ing (ATGSL). This framework includes a search module that uses strong search algorithms (c.g.. Simulated Annealing) to search synonym spaces and a learning module (e.g.. BERT-MLM) that learns from search results. We first use the Sim- ulated Annealing (SA) optimization algorithm in each step to determine token replacement priority. Then we accept or reject suggestions based on heuristic-defined scoring function and save success- ful attack adversarial samples (ATGSL-SA). Since ATGSL-SA requires large number of iterations and attack effectiveness is easily affected by ini- tial conditions, we utilize search results as pseudo references for training condition generators. We design an efficient ATGSL-BM attack algorithm based on generators. In addition, we propose a hy- brid attack method (ATGSL-FUSION) enhance attack effectiveness.
Existing textual adversarial attack methods can mainly be classified into character-level, sentence- level, and word-level attacks based on the gran- ularity of perturbations. Character-level attacks mainly operate on the addition, deletion, substitu- tion, and exchange order of characters in the orig- inal input, and typical substitution methods such as random substitution (Belinkov and Bisk, 2017), character similarity substitution (Eger et al., 2019), etc. However, character-level attacks often produce low-quality adversarial samples that can violate grammar rules and be resisted by grammar-based defence methods (Pruthi et al., 2019) 9).. Sentence- level attacks treat the original input of the whole sentence as an object of perturbation. Typical exam- plcs of such attacks include paraphrasing (Ribeiro et al., 201 8; lyyer et al., 2018), encoding-decoding (Zhao et al., 2017), but the generated text can cause a significant discrepancy with the original text. Word-level attacks perturb words in the original input, and word replacement is the primary method. Common word replacement methods can be cate- gorized into rule-based attacks and learning-based attacks.
We propose black-box attack framework based on a search and learning framework to improve the balance of the attack efficiency and the quality of adversarial samples. To the best of our knowledge, this is the first of its kind to propose search and learning methods to generate adversarial samples.
Rule-based attacks mainly select candidate words based on the preset strategy. (Jin al., 2020) developed methods to search for replace- ment words relying on cosine distance to find ad- jacent word vectors in Glove's embedding space, which may lead to the opposite meaning of original words. To avoid this limitation, (Ren et al., 2019) selected candidate words from well-organized lin- guistic thesaurus (c.g., WordNet (Miller, 1998) and HowNet (Dong and Dong, 2006)) and chose ap- propriate replacement words with the optimal strat- egy. However, this work focuses on fixed WIS (Weight Important Score) order, leading to local se- lection and excessive word replacement. To reduce
ATGSL-SA generates word substitutions from both synonym candidates and sememe candi- dates. It integrates label score, replacement word rate, and semantic similarity into the design of the SA algorithm to generate adver- sarial texts with higher semantic similarity.
Our attack method ATGSL-BM based on fine- tuned pre-trained language models, improves

---

the effect of static WIS on word selection order, BESA (Yang et al., 2021b) optimized the step size of Simulated Annealing to choose the best token replacement combination. However, this method is easily influenced by the initial condition and generates many queries for candidate sets. Recent language model-based attacks are mainly based on BERT Masked Language Model (BERT-MLM). (Garg and Ramakrishnan, 2020; Li et al., 2020b) used BERT-MLM to score replacement words. Al- though these methods consider the semantic rele- vance of context, they still cause ambiguity in tasks such as rumor detection and emotion analysis. This is because the candidate words generated by these pre-trained models do not consider the role of the original words that were replaced by [MASK].
in several ways: 1) Our search aims obtain ad- versarial texts with higher semantic similarity and quality; 2) We use search and learning to design three attack algorithms: ATGSL-SA, ATGSL-BM, and ATGSL--UUSOO. To the best of our knowl- edge, we are the first to address adversarial text generation using the search-and-leame method.
3 Problem Statement
This paper focuses on black-box attacks, in which attackers do not know the internal structure and parameters of the target model, and can only query the target model to obtain its oulput relative to a given imput. In this study, attackers can query the target model's output labels and confidence scores. Formally, let X (21, -2,,..,, be the input dataset including samples, and each ax(8) corre- sponds a ground-trune truth label y
Learning- attacks learn from evaluation his- tory to improve learning model parameters (Zang et al., 2020a; Lee et al., 2022; Sabir et al., 2021). (Zang et al., 2020b) utilized the probability feed- back of the target model modify the parameters of particle swarm optimization (PSO) in order to select the replacement better. BBA (Lee et al., 2022) utilized Bayesian algorithms to learn query history and optimized the selection of replacement locations. But these methods of fitting the target model in the evaluation history have high compu- tational complexity and uncertainty. On the other hand, ReinforeeBug (Sabir et al., 2021) designed feedback generation method including grammar evaluation based on a reinforcement learning frame- work, which improves the quality of the generated samples, but also has the drawbacks of the above WIS basee method, which can affect attack performance. Inspired by this, our work aims to improve the balance between attack efficiency and the quality of adversarial texts by attempting to build the framework by virtue of both search and learning.
Y. Let X - Y be a well-trained model that classifies input samples into labels. This attack can generally be modelled as an optimization problem, which aims to mislead the target model by the adversar- ial sample Xadv with better quality of adversarial texts:
F(Xadv) Y.
An adversarial text example X is defined as the original input X that has been subjected to slight perturbations AX, i.e. Xado X AX.
Methodology
Our attack algorithms are summarized in Algo- rithm 1. In our approach, we first employ the ATGSL-SA algorithm to search for suitable replace- ment words in synonym and sememe word lists, suring semantic similarity while identifying appro- priate replacement word combinations for attacks. To address the issue of high iteration time cost and the tendency to get trapped in local optima due to its initial condition in SA (Henderson et al., 2003), we designed ATGSL-BA BM and ATGSL-FUSION In ATGSL-BM, we fine-tune pre-trained lan- guage model (LM) to learn the search patterns of the ATGSL-SA algorithm and serve the foun- dation for the attack algorithm. The large model capacity and extensive pre-training of LMs enable the generation of high-quality adversarial texts. ATGSL-FUSION, we generate intermediate solu- tions through ATGSL-BM to improve the sensitiv- ity of the initial condition in ATGSL-SA.
2.2 Unsupervised Text Generation
Neural unsupervised text generation has made sig- nificant progress, with variational autoencoders (Kingma and Welling, 2013) being well-known approach. Search-based methods have also been developed for various text generation tasks (Kumar al., 2020; Schumann al., 2020; Miao al., 2019; Liu et al., 2019a), but they are not learnable. (Li et al., 2020a) proposed a search-and-learning approach to improve performance and inference ef- ficiency. Our paper adopts this approach but differs

---

Algorithm 1: Our Proposed Algorithms
This is consistent with the research by (Niven and Kao, 2019), who found that the language model focuses on statistical clues of some words. There- fore, we preprocess all candidate words of each token and design a priority mechanism to select the best replacement word. Since the attack method based on linguistic thesaurus has been proved to have higher semantic similarity between the gen- erated text and the original text (Zang al., 2020b; Yang et al., 2021a), our work gets initial candidate words from synonym-base basedd and sememe-based substitution C; = W U HI, as shown in lines 3-4 of Algorithm For each potential candidate w E C who replaces the original word wi, we define X W1, u2..... w/, ...., Um and candidate importance score Iuf as the true score probability reduction:
Input:
Original text X. initial search state X.ni
target model F. well-trained conditional generative model B
Output: Adversarial sample Xudi
Initialization: The initial temperature
Tinst 0. internal simulation steps
MaxStep 20, S initially empty set, the initial
adversarial example X.dv X
for each loken wy C do
Candidate C, sampled from synonym space
W (WordNet) and sememe space (HowNet);
for t L, M.xtee.
method == ATGSL-SA or ATGSL-FUSION then
Randomly choose an edit position & utilize Cs to replace ze in Eq. 3 to craft Xnewi
Compute the objective value s(*) by Eq.
if method == A7GSL-BM then
Randomly choose an edit position & replace position set and use the well-trained generative model B to generate new word combination as Xnew in Eq. 3;
1 P(Y true X) trre X
if a( sY|| new) - ado) then X.do Xnewi
In every search for replacement words, we choose the word wy with the highest (',, as the best replace- ment word wy. The synonym candidate selection function given below:
12
else
13
Compute the probability p' by Eq. 9; method == ATGSL-SA then With probability p. a = Xnewi
14
18
L
: = R(wx, C.) = arg max L
16.
if method == A7GSL-BM then With probability 1-p, discard edit position k;
w/EC;
Search Process. the search process, we regard each molecule in the search space corresponds to word in the sentence, and the replacement of each word corresponds to the movement of the molecular position. During the simulation process, any molecular motion that can reduce the objec- tive function will be accepted, and there is also conversion probability of increasing the objec- tive function. Additionally, this approach can re- duce the impact of word replacement order because sometimes replacing two words {lop1, lop3) can be even better than changing the top-3 WIS words [top], top2, top3).
if method == ATGSL-FUSION then
With probability p. discard the replaced
Xadu and utilize B to generate new Xodv change the initial condition;
20
Calculate TV by Eq. 10, To = maa (Ty, 0.01);
21
F(Xad)) # Y then Return: Xado
Return:
Adversarial sample Xade
4.1 The ATGSL-SA Algorithm
In ATGSL-SA, we regard determining the best word replacement order as combinatorial opti- mization problem, and use SA to decide word re- placement priority. Simulated Annealing (SA) is effective heuristic search algorithm, suitable for searching large discrete or continuous spaces (Kirkpatrick et al., 1983; Granville al., 1994). Therefore, we choose SA as our search text attack algorithm to search synonym and sememe space and generate adversarial samples with high seman- tic similarity.
Our goal in the ATGSL-SA is to find new adver- sarial samples that minimize the probability of true labels while maintaining high semantic sim- ilarity. The heuristic-based objective function line 11 of Algorithm is:
(new) Perse (Xnew) + a x Dis(X, Xnew) + 8x Semm-Semm..p. X.ow)),
(4)
where Dis(X, Xnew) represents the number of different words between X and Xnows Sem(X, E [o, 1] (the higher, the better) is calculated by Universal Sense Encoder (USE)
Candidate Word List. In a sentence composed of m words X = {ws, W2, ..., Wm }. only some key- words have an impact on the prediction model F.

---

(Cer et al., 2018), and a, 33 are parameters make a tradeoff between the attack efficiency and the semantic similarity. In our implementation, we em- pirically set {a, } - {0.01,0, 0.1). Please refer to Appendix A for more details of ATGSL-SA.
Table Statistics the datasets
Task
Dataset
Train
Test
Classes
Avg
ACTNews
27K
9K
4
43
Chesiffication
IMDB
25K
25K
2
227
MR
7K
3K
2
30
MNLI
430k
10K
3
15
Kntailmenn
4.2 The ATGSL-BM Algorithm.
SNLI
500%
10K
3
12
The ATGSL-BM algorithm consists of two stages: training and attack. During the training stage, the text conditional generator learns the attack patterns of ATGSL-SA. The training procedure is summa- rized in Algorithm 2. During the attack stage, use the trained conditional generator generate text candidates. Fig. and Algorithm 2 in Ap- pendix B provides a diagram of the training stage. Training Process. The local search algorithm has low computational efficiency during inference. requiring several hundred steps of editing and re- evaluation for each sample. Due to the state-of-the art performance of the encoder-decodes framework of BERT-MLM (BM) (Vaswani al., 2017) in text generation, our intuition is to fine-tune BERT- MLM based on ATGSL-SA's search results. On the other hand, BERT-MLM provides a new way of selecting candidate words, which utilizes a bidi- rectional language model to determine two can- didate words (top1, }op2] through context, unlike ATGSL-SA, which determines topl and then calcu- lates P(lop2[top1). This method further reduces the impact of word replacement orders. Specifi- cally, we use [MASK] to fill in the replacement word for Xædv obtained from ATGSL-SA to obtain the mmakk = ? p.aaks ;..,, } In the train- ing process, we construct question-answer pairs (X, x as input for the encoder and (X, X
In short, due to the reduced cross-entropy loss, Eq. 8 is equivalent to reducing (L()(SA) || p; (MM)) Minimizing the KL-term makes the the slot p; (BM) more wide-spreading than the (( because of asymme- try nature, which explains why CE-trained BERT- MLM can smooth out the noise of the stochastic SA search (Li et al., 2020a).
A Attack Process. Our following insight is to use trained BM to design an ATGSL-BM algorithm for generating adversarial texts. Similar to ATGSL-SA search for adversarial texts, in each round, this al- gorithm determines the new replacement position randomly in line 10 of Algorithm We mask out m corresponding positions and obtain the top-t can- didate words for each replacement position from the trained BERT-MLM, we list all possible candi- dates sentence E t x m. Which is m' candidate. We use to calculate the semantic similarity between all candidates and original text and select top-k can- didates. And choose the one with the best attack effect X Xnew-
4.3 The ATGSL-FUSION Algorithm.
ATGSL-FUSSON a hybrid algorithm of ATGSL- SA and ATGSL-BM. For instance, in each step of ATGSL-SA, set of positions be replaced is selected during the initial stage. After selecting several replacement words [w), W22 wyy) if mul- tiple attack attempts Xnew do not decrease score s (Yiruel X this local optima problem caused by the initial conditions will waste considerable computational time. In ATGSL-FUSION, by al- tering the initial conditions and using the results generated by ATGSL-BM as an intermediate solu- tion Xnow for further search, we can avoid local optima in line 20 of Algorithm 1. And proceed with the next iteration based on the modified Xadv in the space of synonyms and sememe.
the label.
by EEcoaae)
(ho (imaak Eimask J.maak)
(6)
faee (S (hn ... N manek r... ken)),
3. = argmaz((so so ftmaal Bour ( (mmask)))
(7)
where Jene and Sdee the encoder and decoder. Given a source sequence x, the objective is the word-by-word cross-entropy (CE) loss, given by
N
JCE ΣΣ
(SA) (BM) Mie log Pie
(8)
5 Experiments
where (SA) is a binary value, indicating whether the ith [MASK] is or not in the ATGSL-SA's out- put for this data sample, and log Pi,r = Pr =
5.1 Experiment Settings
(BM)
Datasets. We evaluate the proposed ATGSL and its variants on five public datasets, including IMDB
ol (SA) masky, which the BERT-MLM predicts.

---

refer to Appendix C.2 for more details of these attacks. Table 2: The attack success rate (ASR) of various attack algorithms on text datasets.
Datasel
Model
PWWS
TFRO
PSO
Reintorce
RESA
ATGSL
-BM
-FUSION
CNN
91.8%
92.1%
93.1%
90,7%
93.1%
93.8%
95.1%
96.8%
97.9%
99.5%
LSTM
90.1%
91.3%
92.8%
94,2%
99.6%
MR
BERT
85.7%
83.9%
88.4%
81.6%%
82.8%
92.
93.2%
94.1%
97.4%
98.9%
RoBERTa
82.8%
87.2%
41.2%
918.7%
CNN
91.1%%
96.6%
98.5%
96.7%
98, 2.6
98.11%
98.4%
98.7%
96.1%%
99.5 %
1STM
94.3%
95.88%
97.698
92,25%
96645%
95.55 598
97. 396
97. 8%
95,4%
98.9%%
IMDB
RoBERTa
74.2%
78.95
82.1%
65.6%
86.8%
92.458
94.25
93.55
97.6%
CNN
82.3%
81.7%
83.9% 9%
81.5%
88.4%
90.2%%
$8.6%
91. 6%
91.9%
93.2%
AG's
ISTM
78.75%
80.8%
77.7%
85.8
86 56.45%
84.
92.33%
94.1
BERT
73.6%
73.2%%
77.88%
74.8%
833.3%
82.7%
86.3%
888.5%
89.3%
92,88 %
RoBERTa
72.5%
81..3%
83.65
81.55
87.8%
4%
InferSent
85.7%
85.2%%
86.6%
84.3%
87.8%
90 69%
91.5%
92.6%
94.7%%
97.5
MNLI
ESIM
8D.35
822669
83.5%
81.55%
86,3%
85 66%
85.7%
87 66.6
900.3%
9/2.4%
BERT
92.1%%
82.8%
78.4%%
94.7%
85.44%
92.4%
RoBERTa
80.2%
81.41%
8.1.9%
80.1%
83.1%%
8.1.5%
83.6%
84.7%
92.9%
96.2%
InferSent
97.9%
ESIM
87.66%
89.5%
85.5%
90.39
90.25
90.75%
91.59
93.22%
95.3%
SNLI
BERT
90.3%
89.88%
92.3%
90.4%
94.3%
92.7%
93.5%
95.7%
96.8%
98.6%
RoBERTa
91.5%
97.3%%
(Maas et al., 1), MR (Pang and Lee, 2005), AG's News (Zhang et al., 2015), MNLI matched (Williams et al., 2017) and SNLI (Bowman et al.. 2015). The AG's News, IMDB, and MR are used for classification tasks, whereas MNLI and SNLI are used for textual entailment. Statistical details of these datasets are summarized in Table Target models. We apply our attack algorithm to popular target models, i..., CNN (Kim, 2014), LSTM (Hochreiter and Schmidhuber, 1997), BERT (Devlin et al., 2018) and RoBERTa (Liu et al., 2019b) sentence classification tasks, and the standard InferSent (Conneau et al., 2017), ESIM (Chen al., 2016), BERT and RoBERTa on tex- tual entailment tasks. CNN stacked by a word embedding layer with 50 embedding dimensions, a convolutional layer with 250 filters, and each kernel size of 3. LSTM passes the input sequence through a 100-dimension embedding layer, concatenating a 128-unit long short-term memory layer, and fol- lowing dropout of 0.5. We download BERT (bert- base-uncased),, RoBERTa (roberta- base) from the Transformers model hub HuggingFace². The origi- nal test results are listed in Table 7. Please refer to Table 7 in Appendix D.I to obtain the test success rates of each model on datasets.
For all datasets, we evaluate the attack success rate, average word substitution rate, semantic simi- larity score, grammatical Errors score and cost time as shown in Tables 2, 3 and 4. ASR is defined the misclassification rate of the target model. In our experiment, semantic similarity and grammar (grammatical errors) are calculated by Universal Sense Encoder (USE) 3 and LanguageTTool 4 re- spectively.
5.2 Experimental Results
Main Results. Tables 2 and 3 show the perfor- mance of our proposed method, ATGSL, and all compared methods five datasets. The results demonstrate that ATGSL outperforms all state-of- the-art models. Compared to fixed WIS algorithms such as PWWS and TEFO, other algorithms such as BESA that use SA to optimize token replace- ment combinations have better attack performance because they consider the impact of substitution word selection order on WIS. Although PSO uti- lizes heuristic methods to optimize substitution word selection, it takes too much time attack very deep models (BERT, RoBERTa) with long text input (such as IMDB). While heuristic-based strong search algorithms have significant effects, a large number of iterations can result in expensive at- tack costs. On the other hand, algorithms based on language models, such as BEAT and BESA, have fewer grammatical errors but cause redundancy of
Baselines. We compare our method with these baselines such as PWWS, TEFO, PSO, Reinforce- Bug. BEAT, BESA. PWWS, TEFO, BEAT are rule- based attacks and the PSO and BESA are Learning- based attacks (Ren al., 2019; Jin al., 2020; Zang al., 2020b; Sabir et al., 2021; Li et al., 2020b; Yang et al., 202 lb; Lee et al., 2022). Please
'hats://tthubb.dee/goog// universal-entenne-nncodde "htnps//langgaggccoloorgg

---

Table 3: Automatic evaluation results of adversarial example quality. "%M", "&I" and "%S" indicate the modification rate, the semantic similarity, grammatical error increase rate, respectively.
Table 4: Time (in seconds) needed in attacking the BERT.
Datuset
TFEO
Reiefurce
MR
6424
18.30
4532
3890
5230
Mintod
MR
IMDB
AD's Nows
NG's News
IMDB
18377
7857
8850
9532
18531
9352
6883
9872
7533
MILLI
3173
2261
2327
CNN
13.3
1.4
0.80
1.5
3,5
0.57
6.3
7.5
0.72
13.5
8.3
3.5
0.49
8:1
0.63
Dutoset
BEAT
RESA
ATUSL
AECISL
ATUIL
CNN
13.3
9.4
0.23
2.8
32
0.54
7.5
7-3
0.59
8.7
9.6
0.54
AS% News
9691
17513
16545
8385
19351
21.3
13.4
0.76
0.52
MILI
SNLI
2753
1638
3450
2630
3213
4392
2232
1357
2573
3342
13.6
6.8
2.6
0.91
5.0
0.55
RoBEXKI
12.3
8.4
0.70
9.3
0.81
13:3
7.8
2.5
0.91
6.5
0.7
smooth out noise in the heuristic-defined search tar- get and generate high-quality adversarial texts with lower attack costs. Additionally, ATGSL-FUSION has the highest attack success rate since it has the acceptance probability to use the adversarial texts generated by ATGSL-BM as the intermediate solu- tion to avoid being trapped by initial conditions in local optima.
1948
DIKT
16.5
0.1
0.85
1.9
0.00
RoBERKI
17.3
9.5
0.75
5.1
4.1
0.55
9,0
7.5
0.82
D0AT
DIKK
15.8
27
0.58
0.60
0.5
0.56
15.5
8.5
0162
6.3
2.
0.65
9.8
73
0.2
BBA
BERT
14.5
100
0.63
6.5
27
0.56
0.51
1.3
9.7
0.57
0.61
ROBEXTA
14.9
13.3
0.61
6.3
27
0.51
7.6
102
0.53
BESA
DOKT
LSTSE
104
13:3
13.5
9.5
11:22
3.5
2.9
0.91
0.92
6.1
5.3
88
0356
ROBERKT
11.5
10.9
01:1
8.3
299
0.30
6.9
0.53
0.56
ATGEL
LSTM
10.5
9.4
0.91
22
0.96
4.2
0.89
Ablation study. In addition, the bottom three rows of Tables 3 and 5 show the effects of ATGSL-SA and ATGSL-BM. These results demonstrate that our algorithms play important roles in semantic similarity and the quality of generated texts, re- spectively. We also conduct supplementary ex- periments with HowNet (ATGSL-SA without H), BERT-base (ATGSL-BM without fine-tuning), and other variants to analyze their performance in at- tack success rate, semantic similarity, runtime, and qrs. It is noticeable that the adversarial samples generated by ATGSL-SA and ATGSL-FUSION have higher semantic similarity, while those gen- erated by ATGSL-BM have fewer grammar errors and higher fluency. During the attack phase, the time consumption and queries of ATGSL- are less than other variants. Compared to ATGSL-BM without fine-tuning our ATGSL-BM has a higher ASR and semantic similarity. Additionally, we in- vestigate how the amount of training data affects the ASR of ATGSL-BM on different datasets. As result, short text datasets (c.g.. MR, SNLI, MNLI) require less data to achieve high ASR than long text datasets (e.g., IMDB, AG's News). Please refer to Fig. 2 in Appendix D.2 for more details.
-A
DOKT
13:3
10.5
11:05
2.6
3.1
0.97
5.3
86
12.3
13.4
322
0.96
3/4
0.3
CNN
12.5
8.5
01:2
1,5
0.92
1/4
ATGEL
LSTM
12.3
7.5
0.54
3.2
1.2
0.91
533
3,0
0.
BERT
14.5
8.8
0.75
17
0.30
6.3
0.8
14.9
9.2
18
0.9
6.7
0.5
CNN
0.6
0.2
2.5
0.95
3.1
0.8
ATOSI
LSTM
9.4
9.5
0188
1:7
25
0.04
12
6.8
036
BERT
10.7
11.
0.3
31
0.93
4.4
0.3
replacement words due to the semantic uncertainty of generated words and lower semantic similarity. Learning model-based attacks such as Reinforce- Bug and BBA show that using evaluation history to train the model has advantages in sentence quality and efficiency. ReinforceBug considers the text quality evaluation score in the reward to generate higher quality adversarial texts but cannot guar- antee high attack success rate due to its high variance. BBA reduces time cost but has difficulty in fitting complex target models using evaluation history (BERT, RoBERTa).
Our proposed attack algorithms have shown ex- cellent performance in various aspects. Compared to BESA, which also uses the SA algorithm, our ATGSL-SA algorithm has a higher attack success rate and semantic similarity due to its ability to uti- lize a well- rell-organized linguistic thesaurus and solve the problem of being trapped in local optimal solu- tions by using acceptance probability p and variable temperature T. We also evaluate semantic consis- tency and grammar errors to ensure that readers do not change their initial predictions and main- tain reading fluency. Our ATGSL-BM algorithm uses a well-trained conditional generative model to
5.3 Human Evaluation
To validate and assess the quality of adversarial samples, we randomly sample 200 adversarial ex- amples targeting LSTT on the MR dataset and tar- geting BERT on the SNLI dataset. Human judges were asked to evaluate the text similarity and gram-

---

Table 5: The analysis for all variants to attack the BERT model on MR. Qrs is the average number of queries.
Table 6: Transfer-1: The ASR of transferred adversarial examples on the AG's New. Transfer-2: Transfer results of ATGSL-BM model with the same emotion classifica- tion task. Higher ASR reflects higher transferabilityy
Datsect
Mathods
ASIL
1.5
Time
Qn
ATCSI
ATGSL-SA
94.2%
0.35
6755
72
MB
ATGSL RM Owlo thee tuno)
$6.35
0.63
4818
63
Transfer 1
PSO
BEAT
BESA
FUSION
ctrining process)
19653
-BERT
68.5%
72.3%
74.9%
procnec)
DERT-+CNN
72.4%
75.5%
76.3%
84.4%
AESSI-FUSION
98.9%
0.33
8327
87
Transfer-2
CNN
LSTM
VERT
RoBERTT
IMDB -M
92.3%
89.5%
85.4%
84,7%
HIMDB
90.8%
87.7%
84.4%
85.3%
1.-
14-
maintains high attack success rate, demonstrat- ing that our attack algorithm using fine-tuned pre- trained models has strong cross-dataset transfer- ability.
344
M4
02
Figure 1: As more adversarial samples increase, the improvement accuracy after ATGSL-SA and ATGSI. FUSION attacks is demonstrated.
6 Case Study
Tables 13 and 14 in Appendix D.5 show examples generated by BERT-ATTACK, PSO, ATGSL-SA, ATGSL-BM, and ATGSL-FUSION on the IMDB and SNLI datasets. The results indicate that our approaches exhibits better performance in terms of attack efficiency and text quality. For more analysis of the results, please refer to Appendix D.5.
matical correctness of the adversarial text generated by our method. As shown in Table 10, our method achieves higher scores in both text similarity and grammatical correctness. For more analysis of the results, please refer to Appendix D.4.
Conclusion
5.4 Adversarial Training
In this paper. we introduced ATGSL, a new at- tack framework with three algorithms that balance attack efficiency and adversarial sample quality. ATGSL-SA used Simulated Annealing to search for similar adversarial texts. ATGSL-BM fine- tuned pre-trained language model (BERT-MLM) to improve attack effectiveness and text quality. ATGSL-FUSSON addressed the ATGSL-SA algo- rithm's 's susceptibility initial conditions by using trained language model to generate intermediate solutions. Our experiments show superior attack performance compared to baseline methods while maintaining a balance between attack performance and text quality.
For adversarial training, we generate adversarial examples using 10% of samples from the IMDB and SNLI datasets' training sets. We then com- bine the generated adversarial examples with the original training sets of our respective datasets and retrain BERT. We then use our attack strategy to attack BERT again. Results are shown in Fig. 1. After the attack, the accuracy rate is increased by 15% to 35%. This indicates that adding adversarial samples to the training data makes the target model more robust to attacks.
5.5 Transferability
If an adversarial example is generated for a specific target model but also successfully attacks other target models, it is called a transferable example. We evaluate the transferability of adversarial at- tacks generated on the Ag's News dataset (Kurakin et al., 2018). The results are shown in Table 6. In Transfer-1, 1, compared to previous attacks, our at- tack produces adversarial examples with higher attack success rate, demonstrating better transfer- ability. In addition, we conduct our experiments on the same binary sentiment classification datasets (MR, IMDB) in Transfer-2. Our ATGSL-BM still
Limitations
A Although we can see that ATGSL-BM has achieved new state-of-the-ard level and performed well on short text datasets (MR, SNLI, MNLI) while main- taining high attack efficiency at low cost, its perfor- mance on long text datasets (IMDB, AG's News) is not as good as ATGSL-SA. As shown in Figure 3, this is due to insufficient training data. Even if we can generate enough training samples from lim- ited test data (multiple attacks), we cannot enrich

---

the variety of training samples. Our future work is further learn more knowledge from successful or failed adversarial samples using self-supervised learning. On the other hand, the fine-tuning of the pre-trained text generator model used the typical BERT-MLM. On the other hand, the fine-tuming of the pre-trained text generator model used the typi- cal BERT-MLM. In future work, we will continue explore the expandability of our proposed attack framework by trying to integrate it with more popu- lar pre-trained models, which is key focus of our future work.
Qian Chen, Xiaodan Zhu, Zhenhua Ling, Si Wei. Hui Jiang, and Diana Inkpen. 2016. Enhanced Istm for natural language inference. arXiv preprint arXiv:1609.0666
Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault, and Antoine Bordes. 2017. Supervised learning of universal sentence representations from natural language inference data. arXiv preprint UTXiv:1705.02364
Jacob Devlin, Ming-Was Wei Chang. Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training deep bidirectional transformers for language understand- ing, arXiv preprint arXiv:1810.0880)
Zhendong Dong and Qiang Dong. 2006. Hownet and the computation of meaning (with Cd-rom), World Scientific.
Broader Ethical Impact
Our research focuses the important problem of adversarial vulnerabilities of classification models on discrete sequential data. Even though it is pos- sible that a malicious adversary misusing ATGSL to attack public text classification APIs, we believe our research can be a basis for the improvement in defending against adversarial attacks on discrete sequential data.
Javid Ebrahimi. Anyi Rao, Daniel Lowd, and De- jing Dou. 2017. Hotflip: White-box adversarial examples for text classification. arXiv preprint arXiv:1712.06751.
Steffen Eger, Gözde Güi Sahin, Andreas Rücklé, Ji- Ji-Ung Lee, Claudia Schulz, Mohsen Mesgar, Krishnkant Swarnkar, Edwin Simpson, and Iryna Gurevych. 2019. Text processing like humans do: Visually at- tacking and shielding nlp systems. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:
ACKNOWLEDGEMENT
Human Language Technologies, Volume (Long and Short Papers), pages 1634-1647.
This work is funded by the National Key R&D Program of China (2022YFB3103700, 2022YFB3103704)
Jules Gagnon-Marchand, Hamed Sadeghi, Md Akmal Haidar, and Mehdi Rezagholizadeh. 2019. Salsa- text: self attentive latent space based adversarial text generation. In Advances in Artificial Intelligence: 32nd Canadian Conference on Artificial Intelligence, Canadian AI 2019. Kingston, ON, Canada, May 28 31, 2019, Proceedings 32, pages 119-131. Springer.
References
Moustafa Alzantot, Yash Sharma, Ahmed Elgohary, Bo-Jhang Ho, Mani Srivastava, and Kai-Wei Chang. 2018. Generating natural language adversarial exam- ples. arXiv preprint arXiv: 1EdY:110040799
Gao, Jack Lanchantin, Mary Lou Soffa, and Yanjun Qi. 2018. Black-box generation of adversarial text sequences to evade deep learning classifiers. In 2018 IEEE Security and Privacy Workshops (SPW), pages 50-56. IEEE
Yonatan Belinkov and Yonatan Bisk. 2017. Synthetic and natural noise both break neural machine transla- tion. arx preprint arXiv:1711.0000
Emily M Bender and Alexander Koller. 2020. Climb- ing towards nlu: On meaning. form, and under- standing in the age of data. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 5185-5198.
Siddhant Garg and Goutham Ramakrishnan. 2020. Bac: Bert-based adversarial examples for text classifica- tion. arXiv preprint 2XXv:20044001970
Vincent Granville, Mirko Krivánek, and J-P Rasson. 1994. Simulated annealing: A proof of convergence. IEEE transactions on pattern analysis and machine intelligence, 16(6):52-655
Samuel R Bowman, Gabor Angeli, Christopher Potts, and Christopher D Manning. 2015. A large annotated corpus for learning natural language inference. arXiv preprint arXiv: 108.0666...
Darrall Henderson. Sheldon H Jacobson, and Alan Johnson. 2003. The theory and practice of simulated anncaling. Handbook of metaheuristics, pages 287- 319.
Daniel Cer. Yinfei Yang. Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St John, Noah Constant, Mario Gunjardo-Cespedees Steve Yuan, Chris Tar, 2018. Universal sentence encoder. arXiv preprint arXiv: arXiv:1803.111 117.5
Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 988:11735 1780.

---

Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan. 2019. Proceedings of the 2019 conference on empiri- cal methods in natural language processing and the 9th international joint conference on natural language processing (emnlp-ijenlp). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-UCCNPP).
Xianggen Liu, Lili Mou, Fandong Meng, Hao Zhou, Jie Zhou, and Sen Song. 2019a. Unsupervised para- phrasing by simulated annealing. arXiv preprint arXiv:1909.0356
Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy. Mike Lewis, Luke 7ettlemoyer, and Veselin Stoyanov. 2019b. Roberta: A robustly optimized bert pretraining ap- proach.
Mohit lyyer, John Wieting, Kevin Gimpel, and Luke 7ettlemoyer. 2018. Adversarial example generation with syntactically controlled paraphrase networks. arXiv preprint 1eXiv:1804.06033
Andrew Maas, Raymond E Daly. Peter T Pham, Dan Huang. Andrew Y Ng. and Christopher Potts. 2011. Learning word vectors for sentiment analy- sis. In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pages 142-150.
Robin Jia and Percy Liang. 2017. Adversarial examples for evaluating reading comprehension systems. arXiv preprint ar Xiv: I 777 07328.
Vijit Malik, Ashwani Bhat, and Ashutosh Modi. 2021. Adv-olm: Generating textual adversaries via olm. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 841 141-
Di Jin, Zhijing Jin, Joey Tianyi Zhou, and Peter Szolovits. 2020. Is bert really robust? strong base- line for natural language attack on text classifica- tion and entailment. In Proceedings of the AAAT conference on artificial intelligence, volume 34, pages 8025-8025.
Ning Miao, Hao Zhou, Lili Mou, Rui Yan, and Lei Li. 2019. Cgmh: Constrained sentence generation by eetropolis-hasting sampling. In Proceedings of the AAAI Conference Artificial Intelligence, volume 33, pages 6834-6842.
Yoon Kim. 2014, Convolutional neural networks for sentence classification. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1751, Doha, Qatar. Association for Computational Linguis tics.
George A Miller. 1998. WordNet: An electronic lexical database. MIT press.
Diederik P Kingma and Max Welling. 2013. Auto- encoding variational bayes. arXiv preprint arXiv:1312.6114.
Timothy Niven and Hung- Yu Kao. 2019. Probing neu- ral network comprehension of natural language argu- ments. arXiv preprint 110iv:199700775500
Scott Kirkpatrick, C Daniel Gelatt Jr, and Mario P Vecchi. 1983. Optimization by simulated annealing. science, 220(4598):671-660
Pang and Lillin Lee. 2005. Seeing stars: Exploiting class relationships for sentiment categorization with respect rating scales. arXiv preprint cs/0506075.
Dhruv Kumar, Lili Mou, Lukasz Golab, and Olga Vechtomova. 2020. Iterative edit-based unsu- pervised sentence simplification. arXiv preprint 099699
Nicolas Papernot, Patrick McDaniel, Somesh Jha. Matt Fredrikson. Berkay Celik, and Ananthram Swami. 2016a. The limitations of deep learning in adver- sarial settings. In 2016 IEEE European symposium on security and privacy (EuroS&P), pages 372-387. IEEE.
Alexey Kurakin, Ian J Goodfellow, and Samy Bengio. 2018. Adversarial examples in the physical world. In Artificial intelligence safety and security, pages 99-112. 12. Chapman and Hall/CRC.
Nicolas Papernot, Patrick McDaniel, Ananthram Swami, and Richard Harang. 2016b. Crafting adver- sarial input sequences for recurrent neural net- works. In MILCOM 2016-2016 IEEE Military Communications Conference, pages 49 49-54. IEEE.
Deokjae Lee, Seungyong Moon, Junhyeok Lee, and Hyun Oh Song. 2022. Query-efficient and scal- able black-box adversarial attacks on discrete sequen- tial data via bayesian optimization. In International Conference Machine Learning, pages 12478- 12497. PMLR.
Danish Pruthi, Bhuwan Dhingra, and Zachary C Lip- ton. 2019. Combating adversarial misspellings with robust word recognition. arXiv preprint arXivv:905.I 11268.
Jingjing L., Zichao L., Lili Mou, Xin Jiang, Michael Lyu, and Irwin King. 2020a. Unsupervised text gen- eration by learning from search. Advances in Neural Information Processing Systems, 10800080888111..
Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. 2019. Generating natural language adversar- ial examples through probability weighted word saliency. In Proceedings of the 57th annual meeting of the association for computational linguistics. pages 1085-1097.
Linyang Li, Ruotian Ma, Qipeng Guo, Xiangyang Xue, and Xipeng Qiu. 2020b. Bert-attack: Adversar- ial attack against bert using bert. arXiv preprint arXiv:2004.099) .09944

---

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Semantically equivalent adversarial rules for debugging nlp models. In Annual Meeting of the Association for Computational Linguistics
Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character- convolutional networks for text classi- fication. Advances in neural information processing systems, 28.
(ACL).
Zhengli Zhao, Dheeru Dua, and Sameer Singh. 2017. Generating natural adversarial examples. arXiv preprint arXiv:
Bushra Sabir, Muhammad Ali Babar, and Raj Gaire.
2021. Reinforcebug: A framework to generate adver- sarial textual examples. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5954-5964.
A The Details of ATGSL-SA Algorithm.
The search process of ATGSL-SA can be divided into the following steps:
Raphael Schumann, Lili Mou, Yao Lu, Olga Vech- tomova, and Katja Markert. 2020. Discrete op- timization for unsupervised sentence summariza- tion with word-level extraction. arXiv preprint arXiv:2005.01991
(1) At first, the algorithm randomly selects a replacement word and calculates new true |
(2) Judging based on Metropolis criteria: when
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Eukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30.
s( (Yruel X), update X to Xnew- When s(Y true (new) >
8( Y turre| Xnew) <
(Y truel (ado); caleulate the probability p in the line 14 of Algorithm 1:
Zheng Wang. 2018. Deep learning-based intrusion detection with adversaries. IEEE Access, 6:38367- 38384.
p = e (x(Y trud torYYruelXNenn ( /aao (9)
take random number (0 < 1), up- date the Xndv the Xnew when r < p. It is not difficult to find that the probability of accepting Xnew increases as temperature in- creases. After modification, the classifier F is misled, we obtain the successfully attacked adversarial sample.
Adina Williams, Nikita Nangia, and Samuel R Bow- man. 2017. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05422
Xinghao Yang, Weifeng Liu, James Bailey, Dacheng Tao, and Wei Liu, 2021a. Bigram and unigram based text attack via adaptive monotonic heuristic search. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 706-714.
(3) Conduct a cooling:
Xinghao Yang. Weifeng Liu, Dacheng Tao, and Wei Liu. 2021b. Besa: Bert-based simulated annealing for adversarial text attacks. In International Joint Conference on Artificial Intelligence.
To Timse + (Sem((Xun Sem((new)))) Cxt.
(10)
If the Xnew semantic similarity score is higher, then temperature increases. This de- sign can prevent the SA algorithm from en- tering the optimal local solution early. Then, turn to step 1, repeat multiple times, and the result tends be stabilized.
Yuan Zang, Bairu Hou, Fanchao Qi, Zhiyuan Liu, Xi- aojun Meng, and Maosong Sun. 2020a. Learning to attack: Towards textual adversarial attacking in real- world situations. arXiv preprint ar xiv::22009.091
Yuan Zang, Fanchao Qi, Chenghao Yang. Zhiyuan Liu, Meng Zhang, Qun Liu, and Maosong Sun. 2020b. Word-level textual adversarial attacking as combinatorial optimization.
B Details of ATGS-BM Training Process
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6066 6080.
Fig. 2 and Algorithm 2 illustrates how the con- dition generator of ATGSL-BM learns from the search results of ATGSL-SA.
Guoyang Zeng, Fanchao Qi. Qianrui Zhou, Tingii Zhang, Zixian Ma. Bairu Hou, Yuan Zang. Zhiyuan Liu, and Maosong Sun. 2021. Openattack: An open-source textual adversarial attack toolkit. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguisties and the 11th International Joint Conference on Natural Language Processing: System Demonstrations,
C Experiment Implementation Details
C.1 Datasets
To show the wide applicability of ATGSL,, we eval- uate ATGSL on various datasets for classification tasks textual entailment.
pages 363-371.

---

ATGSL-SA
thing
acteally releasing thosters
The sapelling sune alverilo Eliil
The www.erarillllae..o thay daronii
mutwepping wideeiiii
About
rapelday tibe
the unaily mitragghilas Bookc
Duat they anerraal reloning fin aaniiiiate
Buy
Xas
RRISO
Fineduuning
Queliun
matterptinn this paie
rour caporblngtee abarraa
anat supibtig enrprien Hing
plea thy weaatt releaigg
Ban acauly risaia ttan (tols))
Bay atall be
the mixi merting stou
the ceppeeertttttee
the murerti ey bla
X
X
RR
Figure 2: The diagram of ATGS-BM training process.
Algorithm 2: The Trained BERT-MLM
pairs from transcribed speeches, popular fic- tion, and government reports. The task is to determine the relationship between pair of concepts, premises, and hypotheses. The test set and training set are derived from the same sources.
Input: Sample text larget model F Output: fine-tuned BERT-MLM model (S(SA) - {}: // Pseudo-reference set is
initially an empty set;
11 Generate adversarial samples as pseudo-reference for training;
for an inpat X do
(SA) = ATGSL-SA((m,,,, C., F);
// ATGSL-SA is detailed Algorithm %
SNLI (Bowman et al., 2015): A dataset com- posed of 570K sentence pairs derived from image captions. The task is to determine the relationship between two sentences: whether the second sentence can be derived from the first sentence's implication, contradiction, or neutral relationship.
1;
(SA) + RR (SA) U U
(SA):
for all epochs do tine-tune BERT-MLM by cross-entropy loss
with pseudo-reference set RR (SA) and its masked R.R. mask: conditioned on origin X;
Return: Resulting the well-trained BERT-MLM
C.2 Baselines
We compare the performance of ATGSL against the state-of-the-art methods as follow: (1) PWWS (Ren et al.. 2019): A method chooses candidate words from WordNet and sorts word attack order by multiplying the word saliency and probability variation. (2) TextFooler (TEFO) (Jin et al., 2020): method obtains synonyms close to original word from the Glove space, and the word selection priority determined by iteratively deleting input words and calculating the DNNs score changes. (3) PSO (Zang et al., 2020b): A method selects word candidates from HowNet and employs the PSO to find adversarial texts. (4) ReinforeeBug (Sabir et al., 2021): reinforced model that di- rectly utilizes the prediction confidence score of the adversarial text the target model and the sen- tence quality score as feedback to optimize the
AG's News (Zhang et al., 2015): A sentence- level dataset for classifying news-type sen- tences into 4 topics: World, Sports, Business, and Science.
Movie Reviews (Pang and Lee, 2005): A sentence-level sentiment classification dataset composed of positive and negative movie re- views from Rotten Tomatoes.
IMDB Polarity (Maas al., 2011): A document-level dataset for binary sentiment classification composed of polar movie re- views from IMDB.
MNLI matched (Williams et al., 2017): A tex- tual entailment dataset composed of sentence

---

fect of the attack. (5) BERT-ATTACK (BEAT) (Li al., 2020b): A method utilizes the BERT-MLM to generate candidate words and attack words in descending order with the static word importance score. (6) BESA (Yang et al., 2021b): A method leverages the BERT to generate candidate words and employs Simulated Annealing (SA) to deter- mine the word substitution order adaptively. (7) BBA (Lee et al., 2022): A Bayesian optimized black-box attack method that dynamically calcu- lates replacement positions based on query history dynamics and automatic correlation determination (ARD) classification rules.
1.00
0.95
0.90
MILI
0.80
SNLI
AG's Nows
0.75
IMDE
0.70
0
30
35
40
ATGGLLBMI adversarial sample count
Figure 3: Efffect of different amounts of training data the ATGSL-BM.
We implement PWWS, PSO, TEFO and BEAT models using Open Source Framework (Zeng et al., 2021), and ReinforceBug, BESA, BBA and our method with Pytorch. To make a fair comparison, we set the upper bound of the number of replacing words as M = 20. Our method gives the parameter settings in line of Algorithm and Algorithm 2. ATGSL-BM, we set kt {2, 10}. More- over, in each dataset, we randomly select 5k test samples for multiple iterations to craft 40k attack successful adversarial samples for the training pro- cess. Finally, 1k test samples that the model has not seen before were selected as test data for the attack experiment.
D.3 Character-level adversarial text generation
While our method is based on word-level adver- sarial text generation techniques, it's worth noting that our search- based learning framework is equally applicable to character-level adversarial text gen- eration. Hence, we have incorporated comparative experiments involving character-level adversarial text generation. The baselines we refer to are:
DeepWordBug(Gao et al., 2018): A character- level adversarial attack algorithm based on the differential evolution technique, introducing subtle character- level perturbations to gener- ate adversarial text that leads to erroneous out- puts from natural language processing mod- els.
D Additional Experiment Results
D.1 Original Accuracy of Various Datasets.
PWWS(Ren et al., 2019): An attack algorithm utilizing white-box strategy, generating adver- sarial text through subtle character replace- ments, insertions, and deletions to mislead the classification output of natural language processing models.
For each dataset, train four state-of-the-ee mod- els on the training set and obtain test set accuracy scores similar to the original implementations, as shown in Table 7.
Table 7: Test accuracy of five datasets before attacks.
Model
MR
IMDB
AG's News
Model
MNLI
SNI.I
Table 8: Classification accuracy disturbed datasets using different attack methods. The third column rep- resents the classification accuracy of the model the original samples. Lower classification accuracy corre- sponds to more effective attack methods.
CNN
78.3%
83.2%
90.9%
InferSent
70.6%
84.3%
LSTM
79.3%
84.5%
89.3%
ESIM
78.3%
85.6%
86.5%
92.3%
Roberta
87.1%
93.5%
94.1%
RoBERTa
86.7%
89.5%
Dagauss
Origial
FWWS5
ATGSL-SA
ATGSL-3M
ATGEL-/ESSTV
D.2 Efffet of Training Amount
CharhN:
77.79
27.270
23.45
22.3%
TERT
$6.5%
18 1/.55
93.3%
31.8%
01.38
R/01255
$7.15
11.55
11.6%
33.1%
33338
19
As shown in Fig. 3, we found that short text datasets (e.g., MR. SNLI, MNLI) require less data to achieve high ASR, while long text datasets (e.g.. IMDB, AG's News) need more data for the same purpose.
Ag's Noos
ESTM
BERT
01.52
15.00
01.58
55.8%
17.1%
35.65
3586
5555
33.16
18.0
96.15
13.65
13.9%
From Table 8 and 9, is evident that at the character-level, our model consistently exhibits re-

---

Table 9: Word replacement rate of each attacking method the selected models.
line methods within the context of human assess- ments, as well as emphasize the label-preserving proficiency of ATGSL.
Datactts
Mecki
TWWS
ATTIN-SAA
ATONL-RM
ATCSI-PUSION
For each adversarial example, we have solicited human evaluators to assign scores based on three distinct aspects: label correctness, syntactic accu- racy, and semantic similarity when compared the original exemplar. The abbreviation "Ace" indi- cates the conformity of the adversarial sample its original classification. Additionally, grammatical correctness score is evaluated on scale from to 5, where:
15TM
BERT
RoBEXTa
CocNNN
22.8%
19.4%
18.3%
15.75
1184
AgS Nows
15TM
BERT
11.35
31.65
10.25
76.75
21.76
18.76
22 P5
11.25
19:75
Ba5EXT
26.5%
2636
23.5%
22.85
18.2%
Table 10: Human- Results.
Dataset
Method
Semantic
Grammar
ATGSL-SA
0.96
4,23
MR
ATGSL-BM
0.94
4.68
Level 1: Text contains severe grammatical errors, rendering comprehension difficult.
ATGSL-FUSION
0.93
4.38
ATGSL-SA
0.91
4.43
SNLI
ATGSL-BM
0.88
4.78
Level 2: Text exhibits multiple grammatical errors, impacting understanding.
ATGSI-FUSION 0.85
4.57
Level 3: Text features minor grammatical er- rors, but remains intelligible overall.
markable attack efficiency and reduced word re- placement rate. This observation highlights the adaptability and flexibility of our generative strat- egy across diverse text granularities.
Level 4: Text is essentially devoid of gram- matical errors, flowing smoothly and compre- hensibly.
D.4 Details of Human Evaluation
order to evaluate the quality of adversarial ex- amples, we randomly select 200 samples that tar- get LSTM on the MR dataset and BERT on the SNLI dataset. The true class labels of these sam- ples are kept hidden and evaluators are asked to classify them. We find that 95% of adversarial ex- amples in MR and 94% in SNLI have the same classification label as their original samples. In addition, five graduate students majoring in lin- guistics are provisionally recruited to annotate all the users according to their expertise experience. Human judges are asked to score each adversarial example on grammatical correctness and semantic similarity with the original example. They are in- structed to score each example from 1 to 5 based on grammatical correctness and assign a score of 0, 0.5, or 1 for semantic similarity, following the prac- tice of (Gagnon-Marchane et al., 2019; Jin et al., 2020). The results are shown in Table 10. Clearly, human judges find that the adversarial texts gener- ated by ATGSL-BB have higher grammatical cor- rectness. Additionally, they find that ATGSL-SA and ATGSL-BM, which utilize linguistic thesaurus (e.g., WordNet and HowNet), have higher semantic similarity.
Level 5: Text is devoid of conspicuous gram- matical errors, demonstrating exceptionally high grammatical precision.
Furthermore, semantic similarity is gauged through the allocation of scores, with values 0, 0.5, and 1, as follows:
0: Generated text diverges significantly from the original text, with minimal shared seman- ties or themes.
0.5: Some semblance of similarity exists be- tween the generated text and the original text, yet notable differences persist.
1: The generated text closely mirrors the orig- inal text, exhibiting a high degree of semantic and thematic consistency.
As shown in Table 11, it is evident that human evaluators have observed higher degree of syn- tactic accuracy in the adversarial texts generated by our approach, particularly ATGSL-BM, com- pared to alternative methods (PWWS. PSO), Addi- tionally, the utilization of linguistic lexicons such as WordNet and HowNet in ATGSL-SA has been found to result in enhanced semantic similarity.
Additionally, in Table 11 we will augment the comparative results between ATGSL and the base-

---

Table 11: Human-Evaan Evaluation Results.
Table 12: The attack success rate (ASR) in high/low
semantic similarity
Datasets
Method
Accuracy
Semantic
Grammar
PWWS
0.79
0.81
3.81
Datusets
Mothod
ASRiHligh Semmatic)
LSR Semaniic)
BEAT
0.74
0.72
4.39
PWWS
0.81
0.73
MR
ATGSL-SA
0.90
0.96
4,23
BEAT
0.83
0.84
ATGSL-BM
0.93
0.94
4.68
MR
STGSL-SA
0.96
0.80
ATGSL-DM
0.96
ATGSL-FUSION
0.96
0.93
4.38
0.97
0.93
PWWS
0.73
0.83
4.14
0.84
BEAT
0.71
0.67
4.54
BEAT
0.88
0.86
SNLI
ATGSL-SA
0.89
0.91
4.43
SNLI
ATGSL SA
0.94
0.87
ATGSL-BM
0.86
0.86
4,78
ATCSL DM
0.95
0.95
0.91
0.91
4.57
ATGSLI-PUSION
0.98
0.93
As observed from the Table 12, our approach demonstrates outstanding performance in both high and low similarity categories. For heuristic-basse algorithms like PWWS and ATGS-SA, they excel in generating adversarial samples with high simi- larity, primarily relying synonym replacement. Consequently, the attack effectiveness is better in high similarity cases. This can be attributed to the inclusion of similarity scores in the objective and cooling functions of ATGS-SA. The objective is to slightly sacrifice similarity to achieve a broader search space in cases of continuous attack failures. This explains the drop in similarity in lower simi- larity categories, where ASR may decrease.
Moreover, our method demonstrates superior accu- racy in correctly categorizing texts, highlighting its strengthened label-preserving capability.
We also provide the attack success rate (ASR) under different similarities and label consistencies. We classify adversarial examples generated by our proposed method as follows: high similarity and human judgment of the same category (HS), high similarity and inconsistent label (HI), low similarity and human judgment of the same category (LS), low similarity and inconsistent labels (L!).
For our ATGSL-SA the MR dataset, targeting the BERT model, achieved Attack Success Rate (ASR) of 94.1 14,, with a total of 814 success- fully attacked samples. The distribution across our four categories is as follows: HS:HI:LS:LI= 773:32:24:6.
On the other hand, ATGS ATGS-BM and BEAT are lan- guage model-based methods and are less sensitive to the similarity of adversarial samples compared to heuristic algorithms. Therefore, they perform well in both categories. However, ATGS-BM generates significantly higher number of high similarity adversarial samples compared to BEAT.
Taking the MR dataset and BERT the target model as an example, the attack success rate (ASR)
for ATGSL-SA in high-similarity adversarial exam- ples is calculated as follows: ASR HS/ / (HS HI)
= 773 / (773 + 32) RJ 0.960. In low-similarity ad- versarial examples, the ASR is calculated as ASR = LS / (LS + LI)= 24 / (24 6) = 0.8.
D.5 Details of Case Study
Tables 13 and 14 show examples generated by BERT-ATTACK, PSO, ATGSL-SA, ATGSL-BM, and ATGSL-FUSION on the IMDB and SNLI datasets. Compared to ATGSL, although PSO searches for adversarial samples in synonym and semantic spaces, it does not consider the text sim- ilarity generated during the iteration process, re- sulting in lower adversarial text similarity. In con- trast, the ATGSL algorithm has a certain probabil- ity of accepting suboptimal solutions, making it easier to find the optimal solution. Additionally, ATGSL-BM has fewer syntax errors and higher at- tack efficiency than algorithms generated by heuris- tic methods. Compared BERT-ATTACK, which is also based on pre-trained models, ATGSL-BM has higher text similarity and ASR. Furthermore, FUGSLN is relatively stable and maintains
For our ATGSL-BM on the MR dataset, tar- geting the BERT model, we achieved an ASR of 97.4%, with a total of 842 successfully attacked samples. The distribution across the four categories is as follows: HS:HI:LS:LI 744:28:58:12 12. For ATGSL-BM, in high-similarity adversarial exam- ples, the ASR is calculated as ASR HS (HS + HI) 744/ (744 + 28) 0.963. In low-similarity adversarial examples, the ASR is calculated as
ASR = L.S (LS + L.) = 24/ (24 + 6) z 0.967.
These results illustrate the effectiveness of the ad- versarial attack methods under consideration, with slightly higher ASR values for ATGSI-BM both high and sim-saiilar adversarial examples com- pared to ATGSL-SA.

---

good balance between attack efficiency and ad- versarial text quality.

---

Table 13: Adversarial examples by attacking BERT model on MR dataset.
BEAT-ATTACK (Successful attack. True lahel score 24,91%, semantic similarity 0.36, qrs=164, grammaticality score 2)
An theomppovertible theontrovendibillly contemporary french psychotogient grief drama examining tragedy the stendeff lationship of an aloof father and his freeze after 20 years apart.
PSO (Successful attack. True label score 33.26%. semantic similarity score 0.56, gus=150, grammaticality score 3)
incontrovertible french psychological examining analyse the standoff aloof father father begetter freeze freezing son after years apart aside.
dramatic
ATGSL-SA (Successful attack. True label score = 44, 86%, semantic similarity score 0.75, qrs=84, grammaticalityy score 2)
An ircomirovertible inarguable french psychelogienl unworldly drama examining the standoff of an aloof father begetter and his freeze son boy after 20 years apart asunder.
ATGSL-BM (Successful attack. True label score = 86%, semantic similarity score 0.84, qrs 56, grammaticality score 1)
An incontrovertbble french russian psychologicul psychiatric druma examining question the standoff of an aloof father and his freeze son after 20 years apart.
ATGSL-FUSION (Successful attack. True lahel 31.28%, semantic similarity 0 grs 94, grammaticality score 2)
incontrovertible french psychological drume seriocomedy examining the stundoff of aloof father and his freese trammel after 20 years apart.
Table 14: Adversarial examples by attacking BERT model on SNLI dataset.
Premise:A young woman with brown hair and elderly man with gray hair and a sweater jump the air on snowy hill with snowshoes on their feet.
An old dead woman women and a young man are erossing the street
Method: BERT-ATTACK (Successful attack. Truc lahel score 19.78%, qrs 43, semantic similarity score 0.33, grammaticality score
eld abandoned woman and young man enosting frustrate the route.
Method: PSO (Successful attack. True label score 22.28%, semantic similarity score 0.54, qrs = 68, grammaticality score = 2)
An old woman female and a young man brother are crossing the street.
Method: ATGSL-SA (Successful attack. True label score 24.67%, semantic similarity score 0.73, qrs 43, grammaticality score =
old woman and young man are crossing the street trajectory
Method: ATGSL-BMM (Successful attack. True label score 28.78%, semantic similarity score = 0.88, grs 17, grammaticality score = I)
An old woman and young tender mer husband are crossing the street
Method: ATGSL-FUSSON (Successful attack. True label score 20.58%, semantic similarity score = 0.79, qrs 52, grammaticality score


Reviews:
[{"review_id": "3UxrsZdwoY", "reviewer": "Reviewer_UFnh", "summary": "", "questions": "Q1. For ATGSL-SA, I wonder how hyper-parameter values (alpha/beta) are selected.\nQ2. Yo mentioned grammar errors are evaluated via GPT-2. Why did you use GPT-2 when there are more modern models?\n", "limitations": "", "rating": 4, "confidence": 2, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Here are several reasons to accept:\n1. The paper contains a thorough literature review, which further highlights the novelty of their approach.\n2. The paper’s method is unique in a sense that it proposes search/learning methods and leverages pre-trained LM to improve attack performance/text quality.\n3. The authors evaluated their approach very thoroughly across multiple datasets and baselines. ", "reasons_to_reject": "One weakness I can think of is, given the limitations of training data size, pre-trained LM does not outperform the ATGSL-SA for long text datasets. It would've been nice if all results remained consistent regardless of the length of datasets.", "questions_to_authors": "Q1. For ATGSL-SA, I wonder how hyper-parameter values (alpha/beta) are selected.\nQ2. Yo mentioned grammar errors are evaluated via GPT-2. Why did you use GPT-2 when there are more modern models?\n", "paper_topic_and_main_contributions": "This paper introduces a novel attack framework named ATGSL, consisting of three components balancing attack performance and text quality. Specifically, it starts with the ATGSL-SA algorithm to efficiently search for replacement words. To improve iteration time cost, they authors propose ATGSL-BM and ATGSL-FUSION algorithms. While ATGSL-BM leverages a finetuned LM to find optimal search patterns,  ATGSL-FUSION uses search results of ATGSL-BM as intermediate solutions. The proposed framework outperforms existing methods in terms of both attack success rate and text quality maintenance.\n\nOverall, this work’s contributions are in twofold: 1) their proposed algorithms are novel in a sense that it proposes search/learning methods and leverages pre-trained LM to improve attack performance/text quality; 2) their proposed algorithms outperform all baselines and successfully passed the evaluation from humans’ lens.\n", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "Not that I know of.", "Typos_Grammar_Style_and_Presentation_Improvements": "In line 412, it's said \"we use use use to calculate\". "}, "extra_text": {}}, {"review_id": "WEcwt0s9lt", "reviewer": "Reviewer_9DV4", "summary": "", "questions": "A. Could ATGSL be applied to more granularities, like character-level and sentence-level?\nB. Have you done some human evaluations regarding 1) comparing ATGSL to previous baselines; 2) the label-preserving ability of ATGSL?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "### Answer a:\n\nI comprehend your viewpoint. While our method is based on word-level adversarial text generation techniques, it's worth noting that our search-based learning framework is equally applicable to character-level adversarial text generation. Hence, we have incorporated comparative experiments involving character-level adversarial text generation.\n\nBaselines:\nDeepWordBug([1]): A character-level adversarial attack algorithm based on the differential evolution technique, introducing subtle character-level perturbations to generate adversarial text that leads to erroneous outputs from natural language processing models.\n\nPWWS([2]): An attack algorithm utilizing white-box strategy, generating adversarial text through subtle character replacements, insertions, and deletions to mislead the classification output of natural language processing models.\n                                                 \n\n#### Table 1: Classification accuracy on disturbed datasets using different attack methods. The third column represents the classification accuracy of the model for the original samples. Lower classification accuracy corresponds to more effective attack methods.\n\n| Datasets | Model | Original | DeepWordBug | PWWS | ATGSL-SA | ATGSL-BM | ATGSL-FUSION |\n| -------- |:--------:| --------:| :--------:| :--------:| --------:| :--------:| :--------:| \n|MR      |CharCNN |             77.9%|     27.8%        |        25.4%  |  20.8%     |  19.7%     |          17.8%     |\n|        |    LSTM       |     77.3%    | 28.6%      |          25.2% |   21.7%  |     18.9%      |          17.2%    |\n|      |      BERT        |              86.5%|     38.3%      |          30.2%     |23.8%     |  21.3%    |           19.3%|\n|      |      RoBERTa |               87.1% |    37.8%   |             31.6% |    23.1%  |     22.3%  |             19.6%|\n|Ag’s News|    CharCNN |            89.3%  |  32.8% |               25.8%  |  23.1%    |   22.5%            |     16.8% |\n||             LSTM     |       89.3% |    35.6% |               23.9%|   22.8%  |     20.4% |               18.9% |   \n||BERT             |         93.5%| 41.3%    |            32.7%  |   25.8%   |    23.5%       |        22.1%|\n|| RoBERTa     |           94.1% |43.8%   |             33.3%|     25.1%  |     23.1%     |          22.9%      |\n\n\n\n\n\n\n\n#### Table 2:  Word replacement rate of each attacking method on the selected models.\n|Datasets  |Model      |             DeepWordBug | PWWS| ATGSL-SA| ATGSL-BM |ATGSL-FUSION|\n| :--------: |:--------:| :--------:| -------- |:--------:| :--------:| -------- |\n|MR |     Char CNN    |             18.8%        |       13.4%  |  8.1%  |     7.7%   |            7.3%    |           \n|      |        LSTM            |          17.3%       |        14.2%  |  8.4%    |   6.9%   |             7.5%   |\n||              BERT            |           22.1%       |       20.7% |    14.7%   |    11.4%      |         9.3%|\n|  |            RoBERTa       |         22.4%         |      19.8%     | 15.3%   |   12.8%           |    9.5%|\n|Ag’s News|  Char CNN |         22.8% |              19.4% |   18.3%  |     15.7%     |          17.8%     |         \n|   |          LSTM         |             21.6%    |           20.2%  |  18.7%    |   16.3%   |             17.2%|   \n|  |            BERT     |                  27.3% |             26.7%   |  24.7%     |  22.4%        |       19.3%|\n| |         RoBERTa     |           26.5%         |      26.8%   |   23.6%     | 22.8%         |      18.2%|\n\nFrom Tables 1 and 2, it is evident that at the character-level, our model consistently exhibits remarkable attack efficiency and a reduced word replacement rate. This observation highlights the adaptability and flexibility of our generative strategy across diverse text granularities.\n\n\n\n###  Answer b: \nCertainly, we intend to provide a detailed elucidation of the score distribution in our manual evaluation process. Additionally, we will augment the comparative results between ATGSL and the baseline methods within the context of human assessments, as well as emphasize the label-preserving proficiency of ATGSL.\n\nFor each adversarial example, we have solicited human evaluators to assign scores based on three distinct aspects: label correctness, syntactic accuracy, and semantic similarity when compared to the original exemplar. The abbreviation \"Acc\" indicates the conformity of the adversarial sample to its original classification. Additionally, grammatical correctness score is evaluated on a scale from 1 to 5, where:\n\nLevel 1: Text contains severe grammatical errors, rendering comprehension difficult.\nLevel 2: Text exhibits multiple grammatical errors, impacting understanding.\nLevel 3: Text features minor grammatical errors, but remains intelligible overall.\nLevel 4: Text is essentially devoid of grammatical errors, flowing smoothly and comprehensibly.\nLevel 5: Text is devoid of conspicuous grammatical errors, demonstrating exceptionally high grammatical precision.\nFurthermore, semantic similarity is gauged through the allocation of scores, with values of 0, 0.5, and 1, as follows:\n\n0: Generated text diverges significantly from the original text, with minimal shared semantics or themes.\n0.5: Some semblance of similarity exists between the generated text and the original text, yet notable differences persist.\n1: The generated text closely mirrors the original text, exhibiting a high degree of semantic and thematic consistency.\n\n\n####   Table 3: Human-Evaluation Results.\n\n|Dataset| Method |Accuracy| Semantic |Grammar|\n| :--------: |:--------:| --------:| :--------: |:--------:|\n|MR|       PWWS | 0.79    |    0.81    |   3.81|\n ||             BEAT|    0.74  |      0.72   |    4.39|\n||            ATGSL-SA |0.90 |   0.96  |      4.23|\n||            ATGSL-BM| 0.93 |   0.94 |   4.68|\n||            ATGSL-FUSION| 0.96| 0.93 |4.38   |\n|SNLI|       PWWS|  0. 73|       0.83  |     4.14|\n||             BEAT   | 0.71  |      0.67     |  4.54|\n||            ATGSL-SA |0.89   | 0.91  |    4.43|\n||            ATGSL-BM |0.86|    0.86  |  4.78|\n||         ATGSL-FUSION| 0.91 |0.91| 4.57|   \n\n\nAs shown in Table 3, it is evident that human evaluators have observed a higher degree of syntactic accuracy in the adversarial texts generated by our approach, particularly ATGSL-BM, compared to alternative methods (PWWS, PSO). Additionally, the utilization of linguistic lexicons such as WordNet and HowNet in ATGSL-SA has been found to result in enhanced semantic similarity. Moreover, our method demonstrates superior accuracy in correctly categorizing texts, highlighting its strengthened label-preserving capability.\n\n###  Answer c: \nRegarding the innovation and limitations of our model, we leverage novel search and learning methods to enhance the attack efficiency and text quality of language models. It is worth noting that in the context of incremental experiments related to limitations, there are instances where ATGSL-BM's results do not surpass those of ATGSL-SA, which can be attributed to certain tricks employed within ATGSL-SA, such as the innovative cooling function. In essence, our ATGSL-BM significantly outperforms existing state-of-the-art methods, confirming the pronounced advantages and notable scalability of our approach. While further enhancing our ATGSL-BM through tricks is indeed feasible, such improvements could be pursued as a separate avenue of research. Additionally, the widespread popularity of large pre-trained models underscores that the substantial gains achieved in practical scenario applications far outweigh the costs associated with their training. We earnestly request reviewers to reconsider our work in light of these aspects.\n\n###  Notes: \nI greatly appreciate your review and feedback. Multiple reviewers have recognized the uniqueness and significance of our proposed search/learning framework, leveraging pre-trained language models to enhance attack performance and text quality. Additionally, our approach has been meticulously evaluated across multiple datasets and baselines. In response to the reviewers' feedback, we conducted supplementary experiments and achieved significant results. This submission holds immense importance to me as it offers a pivotal opportunity to showcase our extensive efforts and achievements. We earnestly request the reviewers to reevaluate our work, and we express our heartfelt gratitude for your consideration.\n\nReference:\n[1]Gao, J.; Lanchantin, J.; Soffa, M. L.; and Qi, Y. 2018. Black- box generation of adversarial text sequences to evade deep learning classifiers. In 2018 IEEE Security and Privacy Workshops (SPW), 50–56. IEEE.\n[2]Shuhuai Ren, Yihe Deng, Kun He, and Wanxiang Che. 2019. Generating natural language adversar- ial examples through probability weighted word saliency. In Proceedings of the 57th annual meeting of the association for computational linguistics, pages 1085–1097.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The proposed ATGSL framework is novel and effective, which is proven by the given experimental results. \n2. Experimental results show that ATGSL outperforms several black-box attack baselines on various classification datasets.", "reasons_to_reject": "1. It seems that ATGSL is still a word-substitution based adversarial attack. Could ATGSL be applied to more granularities, like character-level and sentence-level?\n2. Regarding the human evaluation section. Human evaluation is important when evaluating textual adversarial attacks since it is hard to infer whether the semantic meaning of an adversarial example has been changed by word substitutions from automatic evaluation metrics. However, the human evaluation section only provides the comparison between three variants of ATGSL. It also does not include the evaluation on label-preserving for ATGSL.", "questions_to_authors": "A. Could ATGSL be applied to more granularities, like character-level and sentence-level?\nB. Have you done some human evaluations regarding 1) comparing ATGSL to previous baselines; 2) the label-preserving ability of ATGSL?", "paper_topic_and_main_contributions": "This paper proposes a search and learning framework for Adversarial Text Generation by Search and Learning (ATGSL) and develops three novel black-box adversarial attack methods (ATGSL-SA, ATGSL-BM, ATGSL-FUSION). The core idea of ATGSL is regarding adversarial example generation as a unsupervised text generation problem, such that some search and learning modules can be adapted to craft high-quality adversarial examples.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "black box attack--balck-box attack\n\nLine 45: neutral--neural"}, "extra_text": {}}, {"review_id": "880DMCK6bb", "reviewer": "Reviewer_xqBr", "summary": "", "questions": "Question a: You say you're using GPT-2 to create a grammar score. How are you calculating a grammar score based on the outputs of GPT? I would assume some kind of perplexity or sequence probability. Is it normalized to sequence length? \n\nQuestion b: The output values for Sem() and Gram() aren't clear and without knowing what good scores and bad scores are and what their ranges are, it's hard for me to understand why you would want to divide semantic similarity by grammaticality. Could you maybe plot your score over the range of values they can take and show the cutoff value $\\epsilon$?\n\nQuestion c: Are all attacks using the same sem()/gram() score as their constraint? Results vary greatly depending on the constraints and if attacks use tighter or looser constraints that could put them at a (dis-)advantage. Also, do you know how many examples are left after filtering them out according to the constraints? If there's only one possible replacement then search may not be doing much.\n\nQuestion d: Section 3 and equation 2 seem to suggest that you're using hard constraints to filter out examples that don't have a score better than $\\epsilon$. Line 11 in Algorithm 1 and equation 5 seem to suggest that this is a soft constraint modulated by $\\alpha$ and $\\beta$, which is correct?\n\nQuestion e: Why are you measuring attacks in terms of time to attack instead of queries? If you're querying a black-box model by API, you pay per number of API calls. Previous work has also measured queries as it shows how many examples actually need to be tried out. With learned attacks, faster times could be a result of different model choices and not fewer queries.", "limitations": "", "rating": 3, "confidence": 2, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "I deeply apologize for any inconvenience that the formulaic expressions in the problem description might cause you. We greatly appreciate your understanding. We will continue to address your inquiries and subsequently revise the text concisely, aiming to enhance your reading efficiency.\n\n### Answer a: \nI understand your perspective. Indeed, the accurate situation is that we used LanguageTool(https://languagetool.org) to identify grammatical errors, rather than GPT-2, due to an oversight on my part. We deeply apologize for any inconvenience caused by this oversight. Please rest assured, we are committed to rectifying this issue and making the necessary corrections.\n\n### Answer b: \nTaking into consideration the potential for reader misunderstanding stemming from the presence of $Gram()/Sem()$ terms within the objective function, we are inclined to implement certain revisions. To elucidate, our primary objective lies in fostering the generation of high-quality adversarial texts by the model while simultaneously ensuring maximal preservation of textual coherence and fluency. As a matter of fact, we have not integrated stringent constraints such as $Gram()/Sem()$ into the model algorithm.To alleviate any potential misconceptions, in accordance with your feedback, we intend to make the following amendments to the final version of the problem description:\n\nThis attack can generally be modelled as an optimization problem, which aims to mislead the target model by the adversarial sample $\\mathbf{X}_{adv}$ with better quality of adversarial texts: \n\n\n$\\mathbf{F}(\\mathbf{X}_{adv})\\neq \\mathbf{Y}$,\n\n\nAn adversarial text example $\\mathbf{X}_{adv}$ is defined as the original input $\\mathbf{X}$ that has been subjected to slight perturbations $\\Delta \\mathbf{X}$, i.e., \n\n\n$\\mathbf{X}_{adv} = \\mathbf{X} + \\Delta \\mathbf{X}$ with the overarching goal of preserving a notable degree of semantic similarity and syntactic fluency.\n\n### Answer c: \nI understand your concerns. This issue should be similar to the previous one. In reality, our algorithmic procedure does not incorporate a stringent filtering mechanism based on $Sem()/Gram()$. Our ATGSL-SA method, akin to the approach adopted by Simulated Annealing (BESA as in [1]), indeed integrates the semantic score within the objective function formulation for the adversarial text generation search process (Eq. 11), utilizing a and b as balance parameters. However, the Gram score is not employed. We express our apologies once again.\n\n### Answer d: \nThe present inquiry bears resemblance to the preceding one. Our approach involves the utilization of the Sem() as a constituent of the objective function design. The objective is to effectively explore adversarial text instances characterized by elevated attack potency and substantial textual similarity, akin to the approach outlined in Eq. 10 of BESA. Indeed, ATGSL-SA represents an enhanced iteration of BESA, incorporating refinements such as the integration of synonym tables and the introduction of a temperature-cooling function within the objective function. The latter adjustment serves to mitigate the influence of initial conditions and the potential entrapment in local optima. In addition, our core innovation lies in utilizing novel search and learning methods to enhance the attack efficiency and text quality of language models. Moreover, the widespread popularity of large pre-trained models also attests to the substantial benefits our model achieves in practical scenario applications far outweighing its training costs.\n\n### Answer e: \nI appreciate your perspective. We have taken into full consideration the pivotal nature of both the average query count (Qrs) and the time cost as crucial metrics for model evaluation. It is noteworthy that certain studies, recognizing the inherent instability of Qrs due to factors such as uncertainties in model response, have accorded greater significance to time cost, as documented in references [1]~[3]. Conversely, while we consciously avoided direct Qrs comparisons within Tables 3 and 4 in our paper, it is imperative to underscore that in Experiment 5 of the ablation study, we indeed present Qrs metrics. Moreover, in the forthcoming final version, we intend to incorporate the Qrs comparison between ATGSL and alternative methods, as illustrated in the following Table 1. In contrast to other methodologies, our approach (especially ATGSL-BM) exhibits a proclivity for fewer Qrs, as substantiated by our supplementary experiments.\n\n#### Table 1: Qrs (average number of queries) needed in attacking the BERT.\n\n|Datasets| PWWS|TEFO|PSO|ReinforceBug|BBA|\n|:--------:| --------:| :--------: |:--------:|:--------:| :--------:|\n|MR|126|130|172|152|128|\n|SNLI|64|73|110|94|63|\n\n|Datasets| BEAT|BESA|ATGSL-SA|ATGSL-BM|ATGSL-FUSION|\n|:--------:| --------:| :--------: |:--------:|:--------:| :--------:|\n|MR|98|164|83|55|87|\n|SNLI|33|83|37|23|42|\n\n### Answer f: \nRegarding the matter of reproducibility, multiple reviewers have noted the commendable reproducibility score of our work. In actuality, our model features a limited number of hyperparameters, with their respective values being delineated in Sections 4.1 (The values of $\\alpha$ and $\\beta$ are outcomes derived from repeated experimentation.) and 5.1. Furthermore, we have provided a comprehensive codebase, training procedures, and outcomes within the supplementary materials. The meticulously trained model parameters are also slated for public dissemination online. This comprehensive approach underscores the robust reproducibility inherent in our model architecture.\n\n\n\n###  Notes: \nThank you very much for your review and feedback. Multiple reviewers have acknowledged the uniqueness and significance of our proposed search/learning framework, utilizing search outcomes to enhance the attack performance and text quality of language models. Additionally, our approach has undergone comprehensive evaluation across multiple datasets and baselines. In response to the reviewers' feedback, we conducted supplementary experiments at various levels of text granularity and achieved significant results, further refining the assessment of label-preserving capabilities in human evaluations. This submission holds immense importance to me, as it provides a pivotal opportunity to showcase our extensive efforts and accomplishments. We earnestly request the reviewers to reconsider our work, and we express our heartfelt gratitude for your consideration.\n\n\nReference:\n\n[1] Xinghao Yang, Weifeng Liu, Dacheng Tao, and Wei Liu. 2021b. Besa: Bert-based simulated annealing for adversarial text attacks. In International Joint Conference on Artificial Intelligence.\n\n[2] Xinghao Yang, Weifeng Liu, James Bailey, Dacheng Tao, and Wei Liu. 2021. Bigram and unigram based text attack via adaptive monotonic heuristic search. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pages 706–714.\n\n[3] Bushra Sabir, Muhammad Ali Babar, and Raj Gaire. 2021. Reinforcebug: A framework to generate adver- sarial textual examples. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5954–5964.re", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The method in this paper appears to improve upon other previous adversarial attacks by combining the strengths of both a search based method and a learned model based method.", "reasons_to_reject": "There are multiple inconsistencies throughout the paper on their methodology and what is actually implemented. One section seems to suggest that hard constraints are being used to filter examples but another section suggests that soft constraints are being used.\n\nI'm also not sure if the comparisons are fair due to the uncertainty over the constraints. Changing the constraints can wildly affect the ability of an attack method to find replacements or if there even replacements at all.\n\nThere are some inaccuracies such as referring to BERT as an encoder-decoder model and confusion from variables gaining/losing hats, primes, and super/subscripts.", "questions_to_authors": "Question a: You say you're using GPT-2 to create a grammar score. How are you calculating a grammar score based on the outputs of GPT? I would assume some kind of perplexity or sequence probability. Is it normalized to sequence length? \n\nQuestion b: The output values for Sem() and Gram() aren't clear and without knowing what good scores and bad scores are and what their ranges are, it's hard for me to understand why you would want to divide semantic similarity by grammaticality. Could you maybe plot your score over the range of values they can take and show the cutoff value $\\epsilon$?\n\nQuestion c: Are all attacks using the same sem()/gram() score as their constraint? Results vary greatly depending on the constraints and if attacks use tighter or looser constraints that could put them at a (dis-)advantage. Also, do you know how many examples are left after filtering them out according to the constraints? If there's only one possible replacement then search may not be doing much.\n\nQuestion d: Section 3 and equation 2 seem to suggest that you're using hard constraints to filter out examples that don't have a score better than $\\epsilon$. Line 11 in Algorithm 1 and equation 5 seem to suggest that this is a soft constraint modulated by $\\alpha$ and $\\beta$, which is correct?\n\nQuestion e: Why are you measuring attacks in terms of time to attack instead of queries? If you're querying a black-box model by API, you pay per number of API calls. Previous work has also measured queries as it shows how many examples actually need to be tried out. With learned attacks, faster times could be a result of different model choices and not fewer queries.", "paper_topic_and_main_contributions": "This paper combines heuristic search and a fine-tuned language model to generate adversarial attacks on black-box classifiers of natural language input. Search is performed over synonyms and sememes of words that appear in the input sentences and the replacement words are also used to fine-tune a BERT model to produce semantically similar words. The heuristic search and the model can be used separately or combined into a stronger attack.", "excitement": 3, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Algorithm 1 doesn't use $\\mathcal{B}$ anywhere in it. Addititionally, line 14 calculates $p'$ but the next few lines use $p$ and $1-p$.\n\nStarting on line 462 you write: \"We compare our method with these baselines such as PWWS, TEFO, PSO, ReinforceBug, BEAT, BESA, the PPWS, the PWWS, TEFO, BEAT are rule based attacks and the PSO, BESA and PWWS are Learning-based attacks\". PWWS is repeated four times here and it's included in both learning and rule based attacks."}}]

Author rebuttal:

Meta-review: This paper treats black-box text attack as an unsupervised text generation problem and proposes a search and learning framework (ATGSL) with three adversarial attack methods (ATGSL-SA, ATGSL-BM, ATGSL-FUSION). ATGSL is significantly superior to the most advanced methods in terms of attack efficiency and adversarial text quality. On the other hand, the comparisons may not be fair due to uncertainty over the constraints.
Decision: Accept-Findings