Adversarial Text Generation by Search and Learning (2023)

Abstract: Recent research has shown that evaluating the robustness of natural language processing models using textual attack methods is significant. However, most existing text attack methods only use heuristic replacement strategies or language models to generate replacement words at the word level. The blind pursuit of high attack success rates makes it difficult to ensure the quality of the generated adversarial text. As a result, adversarial text is often difficult for humans to understand. In fact, many methods that perform well in terms of text attacks often generate adversarial text with poor quality. To address this important gap, our work treats black-box text attack as an unsupervised text generation problem and proposes a search and learning framework for Adversarial Text Generation by Search and Learning (ATGSL) and develops three adversarial attack methods (ATGSL-SA, ATGSL-BM, ATGSL-FUSION) for black box text attacks. We first apply a heuristic search attack algorithm (ATGSL-SA) and a linguistic thesaurus to generate adversarial samples with high semantic similarity. After this process, we train a conditional generative model to learn from the search results while smoothing out search noise. Moreover, we design an efficient ATGSL-BM attack algorithm based on the text generator. Furthermore, we propose a hybrid attack method (ATGSL-FUSION) that integrates the advantages of ATGSL-SA and ATGSL-BM to enhance attack effectiveness. Our proposed attack algorithms are significantly superior to the most advanced methods in terms of attack efficiency and adversarial text quality.

## introduction recent research has demonstrated that deep neural networks (dnns) are vulnerable to maliciously crafted adversarial text examples that can fool victim models into making wrong predictions @xcite . these text examples that add malicious perturbation to the original text do not affect human judgment but can deceive deep learning models. (bender and koller, 2020) pointed out that deep neutral models have defects in understanding the meaning conveyed by language. thus, generating adversarial samples has become a common method for evaluating the weakness and robustness of dnns. existing malicious text generation algorithms can be classified into character-level attacks, sentence-level attacks, and word-level attacks. character-level attacks @xcite include the addition, deletion, replacement, and order exchange of characters, which sacrifices the readability of the generated text in exchange for the attack's success rate. sentence-level attacks @xcite regard the original input of the whole sentence as a perturbation object, which often makes a considerable difference between the generated text and the original input. it is not easy to guarantee the quality of the generated text. therefore, many studies focus on improving the attack success rate and the quality of the attack text generated by word replacement, hence word-level attacks. previous works mainly generate textual attacks on word replacement according to specific rules @xcite @xcite @xcite . most of these models show good attack performance by using multiple linguistic constraints (e.g., ner tagging and pos tagging) and a well-organized linguistic thesaurus (e.g., wordnet and hownet). however, these works require extensive preprocessing, and the substitution words selection heavily relies on tags and cannot guarantee the fluency and grammaticality of adversarial samples. the other attack methods based on language models, such as bert, can generate contextual perturbations @xcite @xcite . these models ensure the predicted token fits the sentence well but cannot preserve the semantic similarity @xcite . for example, in the sentence "i feel [mask]", predicting the @xcite as happy or sad is equally sensible but results in a sentiment analysis task. in order to improve these issues, recent research has focused on learning-based methods @xcite @xcite , aiming to utilize model learning to improve the balance between the efficiency of attack algorithms and the quality of adversarial texts from the attack evaluation history. inspired by these works, adversarial text generation can be seen as an unsupervised text generation problem. thus, we propose a new framework for adversarial text generation by @xcite . this framework includes a search module that uses strong search algorithms (e.g., simulated annealing) to search synonym spaces and a learning module (e.g., bert-mlm) that learns from search results. we first use the simulated annealing (sa) optimization algorithm in each step to determine token replacement priority. then we accept or reject suggestions based on a heuristic-defined scoring function and save successful attack adversarial samples (atgsl-sa). since atgsl-sa requires a large number of iterations and attack effectiveness is easily affected by initial conditions, we utilize search results as pseudo references for training condition generators. we design an efficient atgsl-bm attack algorithm based on generators. in addition, we propose a hybrid attack method (atgsl-fusion) to enhance attack effectiveness. ## problem statement this paper focuses on black-box attacks, in which attackers do not know the internal structure and parameters of the target model, and can only query the target model to obtain its output relative to a given input. in this study, attackers can query the target model's output labels and confidence scores. formally, let @xmath0, x 2 , ..., x n } be the input dataset including n samples, and each x (i) corresponds to a ground-truth label y true @xmath1. let f : @xmath2. this attack can generally be modelled as an optimization problem, which aims to mislead the target model by the adversarial sample x adv with better quality of adversarial texts: an adversarial text example x adv is defined as the original input x that has been subjected to slight perturbations ∆x, i.e., x @xmath3. ## methodology our attack algorithms are summarized in algorithm 1. in our approach, we first employ the atgsl-sa algorithm to search for suitable replacement words in synonym and sememe word lists, ensuring semantic similarity while identifying appropriate replacement word combinations for attacks. to address the issue of high iteration time cost and the tendency to get trapped in local optima due to its initial condition in sa @xcite , we designed atgsl-bm and atgsl-fusion 1 . in atgsl-bm, we fine-tune a pre-trained language model (lm) to learn the search patterns of the atgsl-sa algorithm and serve as the foundation for the attack algorithm. the large model capacity and extensive pre-training of lms enable the generation of high-quality adversarial texts. in atgsl-fusion, we generate intermediate solutions through atgsl-bm to improve the sensitivity of the initial condition in atgsl-sa. algorithm 1: our proposed algorithms input: original text x, initial search state xini, target model f, well-trained conditional generative model b output: adversarial sample x adv 1 initialization: the initial temperature @xmath4.1, internal simulation steps @xmath5, s is initially an empty set, the initial adversarial example x @xmath6; 4 for @xmath7, • • •, maxstep do 5 if @xmath8. 3 to craft xnew; 7 compute the objective value s( * ) by eq. 4; 8 if @xmath9. 3; 10 if s(y|xnew)s(y|x adv ) < 0 then 11 x @xmath10; 12 else 13 compute the probability p ′ by eq. 9; 14 if @xmath11, x @xmath12; 16 if @xmath13, discard edit position k; 18 if @xmath14, discard the replaced x adv and utilize b to generate new x adv to change the initial condition; 20 calculate t ′ 0 by eq. 10, @xmath15, 0.01}; 21 if f(x adv ) ̸ = y then return: x adv return: adversarial sample x adv 22 4.1 the atgsl-sa algorithm in atgsl-sa, we regard determining the best word replacement order as a combinatorial optimization problem, and use sa to decide word replacement priority. simulated annealing (sa) is an effective heuristic search algorithm, suitable for searching large discrete or continuous spaces @xcite . therefore, we choose sa as our search text attack algorithm to search synonym and sememe space and generate adversarial samples with high semantic similarity. candidate word list. in a sentence composed of m words @xmath16, w 2 , ..., w m }, only some keywords have an impact on the prediction model f . this is consistent with the research by @xcite , who found that the language model focuses on statistical clues of some words. therefore, we preprocess all candidate words of each token and design a priority mechanism to select the best replacement word. since the attack method based on linguistic thesaurus has been proved to have a higher semantic similarity between the generated text and the original text @xcite , our work gets initial candidate words from synonym-based and sememe-based substitution c @xmath17, as shown in lines 3~4 of algorithm 1. for each potential candidate w ′ @xmath18, we define x ′ w @xmath19, w 2 , ..., w ′ i , ..., w m and candidate importance score i w ′ i as the true score probability reduction: in every search for replacement words, we choose the word w i with the highest i ′ w i as the best replacement word w * i . the synonym candidate selection function is given below: in the search process, we regard each molecule in the search space corresponds to a word in the sentence, and the replacement of each word corresponds to the movement of the molecular position. during the simulation process, any molecular motion that can reduce the objective function will be accepted, and there is also a conversion probability of increasing the objective function. additionally, this approach can reduce the impact of word replacement order because sometimes replacing two words {top1, top3} can be even better than changing the top-3 wis words {top1, top2, top3}. our goal in the atgsl-sa is to find new adversarial samples that can minimize the probability of true labels while maintaining high semantic similarity. the heuristic-based objective function in line 11 of algorithm 1 is: ## case study tables 13 and 14 in appendix d.5 show examples generated by bert-attack, pso, atgsl-sa, atgsl-bm, and atgsl-fusion on the imdb and snli datasets. the results indicate that our approaches exhibits better performance in terms of attack efficiency and text quality. for more analysis of the results, please refer to appendix d.5. ## conclusion in this paper, we introduced atgsl, a new attack framework with three algorithms that balance attack efficiency and adversarial sample quality. atgsl-sa used simulated annealing to search for similar adversarial texts. atgsl-bm finetuned a pre-trained language model (bert-mlm) to improve attack effectiveness and text quality. atgsl-fusion addressed the atgsl-sa algorithm's susceptibility to initial conditions by using a trained language model to generate intermediate solutions. our experiments show superior attack performance compared to baseline methods while maintaining a balance between attack performance and text quality.