AdvCodec: Towards A Unified Framework for Adversarial Text Generation

1 INTRODUCTION . Recent studies have demonstrated that deep neural networks ( DNNs ) are vulnerable to carefully crafted adversarial examples ( Goodfellow et al. , 2015 ; Papernot et al. , 2016 ; Eykholt et al. , 2017 ; Moosavi-Dezfooli et al. , 2016 ) . While there are a lot of successful attacks proposed in the continuous data domain including images , audios , and videos , how to effectively generate adversarial examples in the discrete text domain still remains a hard problem . There are several challenges for generating adversarial text : 1 ) most existing gradient-based adversarial attack approaches are not directly applicable to the discrete structured data ; 2 ) it is less clear how to appropriately measure the naturalness of the generated text compared to the original ones ; 3 ) the manipulation space of text is limited , and it is unclear whether generating a new appended sentence or manipulating individual words will affect human judgements . So far , existing works on adversarial text generation either leverage heuristic solutions such as genetic algorithms ( Jin et al. , 2019 ) to search for potential adversarial sentences , or are limited to attacking specific NLP tasks ( Cheng et al. , 2018 ; Lei et al. , 2018 ) . In addition , effective targeted attacks have not been achieved by current attacks for any task . In this paper , we aim to provide more insights towards solving these challenges by proposing a unified optimization framework AdvCodec to generate adversarial text against general NLP tasks . In particular , the core component of AdvCodec is a tree based autoencoder which converts discrete text tokens into continuous semantic embedding , upon which the adversarial perturbation will be optimized regarding the chosen adversarial target . Finally , a tree based decoder will decode the generated adversarial continuous embedding vector back to the sentence level based on the tree grammar rules , aiming to both pre- serve the original semantic meaning and linguistic coherence . An iterative process can be applied here to ensure the attack success rate . In addition to the general adversarial text generation framework AdvCodec , this paper also aims to explore several scientific questions : 1 ) Since AdvCodec allows the flexibility of manipulating on different hierarchies of the tree structures , which is more attack effective and which way preserves better grammatical correctness ? 2 ) Is it possible to achieve targeted attack for general NLP tasks such as sentiment classification and QA , given the limited degree of freedom for manipulation ? 3 ) Is it possible to perform blackbox attack in general NLP tasks ? 4 ) Is BERT robust in practice ? 5 ) Do these adversarial examples affect human reader performances ? To address the above questions , we explore two types of tree based autoencoders on the word ( AdvCodec ( Word ) ) and sentence level ( AdvCodec ( Sent ) ) . For each encoding scenario , we generate adversarial text against different sentiment classification and QA models . Compared with the state-of-the-art adversarial text generation methods , our approach achieves significantly higher untargeted and targeted attack success rate . In addition , we perform both whitebox and blackbox settings for each attack to evaluate the model vulnerabilities . Within each attack setting , we evaluate attack strategies as appending an additional adversarial sentence or adding scatter of adversarial words to a paragraph , to evaluate the quantitative attack effectiveness . To provide thorough adversarial text quality assessment , we also perform 7 groups of human studies to evaluate the quality of generated adversarial text compared with the baselines methods , and whether human can still get the ground truth answers for these tasks based on adversarial text . We find that : 1 ) both word and sentence level attacks can achieve high attack success rate , while the sentence level manipulation can consider the global grammatical constraints and generate high quality adversarial sentences . 2 ) various targeted attacks on general NLP tasks are possible ( e.g . when attacking QA , we can ensure the target to be a specific answer or a specific location within a sentence ) ; 3 ) the transferability based blackbox attacks are successful in NLP tasks . Transferring adversarial text from stronger models ( in terms of performances ) to weaker ones is more successful ; 4 ) Although BERT has achieved state-ofthe-art performances , we observe the performance drops are also larger than other standard models when confronted with adversarial examples , which indicates BERT is not robust under the adversarial settings ; 5 ) Most human readers are not sensitive to our adversarial examples and can still answer the right answers when confronted with the adversary-injected paragraphs . In summary , our main contribution lies on : ( 1 ) We propose a general adversarial text generation framework AdvCodec that addresses the challenge of discrete text input to achieve targeted attacks against general NLP tasks ( e.g . sentiment classification and QA ) while preserving the semantic meaning and linguistic coherence ; ( 2 ) we propose a novel tree-based text autoencoder that ensures the grammar correctness of generated text ; ( 3 ) we conduct extensive experiments and successfully attack different sentiment classifiers and QA models with significant higher attack success rate than the state-of-the-art baseline methods ; ( 4 ) we also perform comprehensive ablation studies including evaluating the attack scenarios of appending an adversarial sentence or adding scatter of adversarial words , as well as appending the adversarial sentence at different positions within a paragraph , and draw several interesting conclusions ; ( 5 ) we leverage extensive human studies to show that the adversarial text generated by AdvCodec is natural and effective to attack neural models , while barely affecting human ’ s judgement . 2 RELATED WORK . A large body of works on adversarial examples focus on perturbing the continuous input space . Though some progress has been made on generating adversarial perturbations in the discrete space , several challenges still remain unsolved . For example , Zhao et al . ( 2017 ) exploit the generative adversarial network ( GAN ) to generate natural adversarial text . However , this approach can not explicitly control the quality of the generated instances . Most existing methods ( Liang et al. , 2017 ; Samanta & Mehta , 2017 ; Jia & Liang , 2017 ; Li et al. , 2018 ; Jin et al. , 2019 ) apply heuristic strategies to synthesize adversarial text : 1 ) first identify the features ( e.g . characters , words , and sentences ) that have the influence on the prediction , 2 ) follow different search strategies to perturb these features with the constructed perturbation candidates ( e.g . typos , synonyms , antonyms , frequent words ) . For instance , Liang et al . ( 2017 ) employ the loss gradient∇L to select important characters and phrases to perturb , while Samanta & Mehta ( 2017 ) use typos , synonyms , and important adverbs/adjectives as candidates for insertion and replacement . Once the influential features are obtained , the strategies to Perturbation . apply the perturbation generally include insertion , deletion , and replacement . Such adversarial text generation approaches can not guarantee the grammar correctness of generated text . For instance , text generated by Liang et al . ( 2017 ) are almost random stream of characters . To generate grammarly correct perturbation , Jia & Liang ( 2017 ) adopt another heuristic strategy which adds manually constructed legit distracting sentences to the paragraph to introduce fake information . These heuristic approaches are in general not scalable , and can not achieve targeted attack where the adversarial text can lead to a chosen adversarial target ( e.g . adversarial label in classification ) . Recent work searches for a universal trigger ( Wallace et al. , 2019 ) to be applied to arbitrary sentences to fool the learner , while the reported attack success rate is rather low . In contrast , with the tree based autoencoder , the proposed AdvCodec framework is able to generate grammarly correct adversarial text efficiently , achieving high attack success rates on different models . 3 THE ADVCODEC FRAMEWORK FOR ADVERSARIAL TEXT GENERATION . We describe the AdvCodec framework in this section . As illustrated in Figure 1 , the key component of the AdvCodec framework is a tree-based autoencoder . The hierarchical and discrete nature of language motivates us to make use of tree-based autoencoder to map discrete text into a high dimensional latent space , which empowers us to leverage the existing optimization based attacking method such as Carlini & Wagner ( 2016 ) to both efficiently and effectively generate adversarial text . Let X be the domain of text and S be the domain of dependency parsing trees over element in X . Formally , a tree-based autoencoder consists of an encoder E : X × S → Z that encodes text x ∈ X along with its dependency parsing tree s ∈ S into a high dimensional latent representation z ∈ Z and a decoder G : Z × S → X that generates the corresponding text x from the given context vector z and the expected dependency parsing tree s. Given a dependency tree s , E and G form an antoencoder . We thus have the following reconstruction loss to train our tree-based autoencoder : L = −Ex∼X [ log pG ( x|s , E ( x , s ) ] ( 1 ) As Figure 1 suggests , AdvCodec can operate on different granularity levels to generate either wordlevel or sentence-level contextual representation , and decode it into the adversarial text . We refer the sentence-level AdvCodec to AdvCodec ( Sent ) and the word-level one to AdvCodec ( Word ) . Both of them will be described in more details in the later part of this section . 3.1 OVERVIEW OF THE ADVCODEC FRAMEWORK Before diving into details , we provide a high level overview of AdvCodec according to the attack scenario and attack capability supported by this framework . Attack Scenario . Different from the previous adversarial text generation works ( Lei et al. , 2018 ; Cheng et al. , 2018 ; Papernot et al. , 2016 ; Miyato et al. , 2016 ; Alzantot et al. , 2018 ) that directly modify critical words in place and might risk changing the semantic meaning or editing the ground truth answers , we are generating the concatenative adversaries . First proposed by Jia & Liang ( 2017 ) , the concatenative adversary does not change any words in the original paragraph or question , but instead appends a new adversarial sentence to the paragraph to fool the model . However , the concatenative attack also needs to ensure the appended sentence is compatible ( Jia & Liang , 2017 ) with the original paragraph , which in other words means it should not contradict any stated facts in the paragraph , especially the correct answer . In our work , we further push the concept of concatenative adversaries further and propose a more general notion called scatter attack , which means we can inject adversarial words sporadically over the whole paragraph . The concatenative adversarial example falls into our case when those adversarial tokens form a sentence and on the same time the semantic meaning of the sentence does not contradict the original paragraph . Examples of concatenative attack and scatter attack is shown in table 1 . Attack Capability . AdvCodec is essentially an optimization based framework to find the adversarial texts with the optimization goal set to achieve targeted attack . For the sentiment classification task , AdvCodec can perform targeted attack to make the original positive reviews be classified as the most negative one , and vice versa . Particularly in the QA task , we design and implement two kinds of targeted attack : position targeted attack and answer targeted attack . A successful position targeted attack means the model can be fooled to output the answers at specific targeted positions in the paragraph , but the content on the targeted span can not be guaranteed . In contrast , a successful answer targeted attack is a stronger targeted attack , which refers to the situation when the model always outputs the preset targeted answer pair on the target no matter what the question looks like . An example of word targeted attack can be found in the table 1 . Although our framework is designed as a whitebox attack , our experimental results demonstrate our whitebox generated adversarial words can transfer to other blackbox models with high attack success rate . Finally , because AdvCodec is a unified adversarial text generation framework whose outputs are discrete tokens , it can be applied to different downstream NLP tasks . In this paper , we perform adversarial evaluation on sentiment classification and QA as examples to demonstrate how our framework is adapted to different works . 3.2 ADVCODEC ( SENT ) In this subsection , we describe AdvCodec ( Sent ) and explain how to utilize it to attack sentiment classification models and question answering systems . The main idea comes from the fact that tree structures sometimes have better performances than sequential recurrent models ( Li et al. , 2015 ; Iyyer et al. , 2014 ; 2018 ) and the fact that it is inherently flexible to add perturbations on hierarchical nodes of the tree structures . Motivated by this , we design a novel tree-based autoencoder to simultaneously preserve similar semantic meaning and original syntactic structure . Encoder . We adopt the Stanford Treestructured LSTM ( Tai et al. , 2015 ) as our tree encoder . In the encoding phase , features are extracted and summed from bottom ( leaf node , i.e . word ) to top ( root node ) along the dependency tree , extracted by Stanford CoreNLP Parser ( Manning et al. , 2014 ) . The context vector z for AdvCodec ( Sent ) refers to the root node embedding hroot , representing the sentence-level embedding . Decoder . Following the same dependency tree , we design the text decoder as illustrated in Figure 2 . In the decoding phase , we start from the root node and traverse along the dependency tree in level-order . The hidden state hj of the next node j comes from ( i ) the hidden state hi of the current tree node , ( ii ) current node predicted word embedding wi , and ( iii ) the dependency embedding dij between the current node i and the next node j based on the dependency tree . The next node ’ s corresponding word yj is generated based on the output of the LSTM Cell oj via a linear layer that maps from the hidden presentation oj to the logits that represent the probability distribution of the tree ’ s vocabulary . oj , hj = LSTM ( [ hi ; wi ; dij ] ) ( 2 ) yj = W · oj + b ( 3 )

Summary: Motivated by recent development of attack/defense methods addressing the vulnerability of deep CNN classifiers for images, this paper proposes an attack framework for adversarial text generation, in which an autoencoder is employed to map discrete text to a high-dimensional continuous latent space, standard iterative optimization based attack method is performed in the continuous latent space to generate adversarial latent embeddings, and a decoder generates adversarial text from the adversarial embeddings.  Different generation strategies of perturbing latent embeddings at sentence level or masked word level are both explored. Adversarial text generation can take either a form of appending an adversarial sentence or a form of scattering adversarial words into different specified positions. Experiments on both sentiment classification and question answering show that the proposed attack framework outperforms some baselines. Human evaluations are also conducted.