AdvCodec: Towards A Unified Framework for Adversarial Text Generation

1 INTRODUCTION . Recent studies have demonstrated that deep neural networks ( DNNs ) are vulnerable to carefully crafted adversarial examples ( Goodfellow et al. , 2015 ; Papernot et al. , 2016 ; Eykholt et al. , 2017 ; Moosavi-Dezfooli et al. , 2016 ) . While there are a lot of successful attacks proposed in the continuous data domain including images , audios , and videos , how to effectively generate adversarial examples in the discrete text domain still remains a hard problem . There are several challenges for generating adversarial text : 1 ) most existing gradient-based adversarial attack approaches are not directly applicable to the discrete structured data ; 2 ) it is less clear how to appropriately measure the naturalness of the generated text compared to the original ones ; 3 ) the manipulation space of text is limited , and it is unclear whether generating a new appended sentence or manipulating individual words will affect human judgements . So far , existing works on adversarial text generation either leverage heuristic solutions such as genetic algorithms ( Jin et al. , 2019 ) to search for potential adversarial sentences , or are limited to attacking specific NLP tasks ( Cheng et al. , 2018 ; Lei et al. , 2018 ) . In addition , effective targeted attacks have not been achieved by current attacks for any task . In this paper , we aim to provide more insights towards solving these challenges by proposing a unified optimization framework AdvCodec to generate adversarial text against general NLP tasks . In particular , the core component of AdvCodec is a tree based autoencoder which converts discrete text tokens into continuous semantic embedding , upon which the adversarial perturbation will be optimized regarding the chosen adversarial target . Finally , a tree based decoder will decode the generated adversarial continuous embedding vector back to the sentence level based on the tree grammar rules , aiming to both pre- serve the original semantic meaning and linguistic coherence . An iterative process can be applied here to ensure the attack success rate . In addition to the general adversarial text generation framework AdvCodec , this paper also aims to explore several scientific questions : 1 ) Since AdvCodec allows the flexibility of manipulating on different hierarchies of the tree structures , which is more attack effective and which way preserves better grammatical correctness ? 2 ) Is it possible to achieve targeted attack for general NLP tasks such as sentiment classification and QA , given the limited degree of freedom for manipulation ? 3 ) Is it possible to perform blackbox attack in general NLP tasks ? 4 ) Is BERT robust in practice ? 5 ) Do these adversarial examples affect human reader performances ? To address the above questions , we explore two types of tree based autoencoders on the word ( AdvCodec ( Word ) ) and sentence level ( AdvCodec ( Sent ) ) . For each encoding scenario , we generate adversarial text against different sentiment classification and QA models . Compared with the state-of-the-art adversarial text generation methods , our approach achieves significantly higher untargeted and targeted attack success rate . In addition , we perform both whitebox and blackbox settings for each attack to evaluate the model vulnerabilities . Within each attack setting , we evaluate attack strategies as appending an additional adversarial sentence or adding scatter of adversarial words to a paragraph , to evaluate the quantitative attack effectiveness . To provide thorough adversarial text quality assessment , we also perform 7 groups of human studies to evaluate the quality of generated adversarial text compared with the baselines methods , and whether human can still get the ground truth answers for these tasks based on adversarial text . We find that : 1 ) both word and sentence level attacks can achieve high attack success rate , while the sentence level manipulation can consider the global grammatical constraints and generate high quality adversarial sentences . 2 ) various targeted attacks on general NLP tasks are possible ( e.g . when attacking QA , we can ensure the target to be a specific answer or a specific location within a sentence ) ; 3 ) the transferability based blackbox attacks are successful in NLP tasks . Transferring adversarial text from stronger models ( in terms of performances ) to weaker ones is more successful ; 4 ) Although BERT has achieved state-ofthe-art performances , we observe the performance drops are also larger than other standard models when confronted with adversarial examples , which indicates BERT is not robust under the adversarial settings ; 5 ) Most human readers are not sensitive to our adversarial examples and can still answer the right answers when confronted with the adversary-injected paragraphs . In summary , our main contribution lies on : ( 1 ) We propose a general adversarial text generation framework AdvCodec that addresses the challenge of discrete text input to achieve targeted attacks against general NLP tasks ( e.g . sentiment classification and QA ) while preserving the semantic meaning and linguistic coherence ; ( 2 ) we propose a novel tree-based text autoencoder that ensures the grammar correctness of generated text ; ( 3 ) we conduct extensive experiments and successfully attack different sentiment classifiers and QA models with significant higher attack success rate than the state-of-the-art baseline methods ; ( 4 ) we also perform comprehensive ablation studies including evaluating the attack scenarios of appending an adversarial sentence or adding scatter of adversarial words , as well as appending the adversarial sentence at different positions within a paragraph , and draw several interesting conclusions ; ( 5 ) we leverage extensive human studies to show that the adversarial text generated by AdvCodec is natural and effective to attack neural models , while barely affecting human ’ s judgement . 2 RELATED WORK . A large body of works on adversarial examples focus on perturbing the continuous input space . Though some progress has been made on generating adversarial perturbations in the discrete space , several challenges still remain unsolved . For example , Zhao et al . ( 2017 ) exploit the generative adversarial network ( GAN ) to generate natural adversarial text . However , this approach can not explicitly control the quality of the generated instances . Most existing methods ( Liang et al. , 2017 ; Samanta & Mehta , 2017 ; Jia & Liang , 2017 ; Li et al. , 2018 ; Jin et al. , 2019 ) apply heuristic strategies to synthesize adversarial text : 1 ) first identify the features ( e.g . characters , words , and sentences ) that have the influence on the prediction , 2 ) follow different search strategies to perturb these features with the constructed perturbation candidates ( e.g . typos , synonyms , antonyms , frequent words ) . For instance , Liang et al . ( 2017 ) employ the loss gradient∇L to select important characters and phrases to perturb , while Samanta & Mehta ( 2017 ) use typos , synonyms , and important adverbs/adjectives as candidates for insertion and replacement . Once the influential features are obtained , the strategies to Perturbation . apply the perturbation generally include insertion , deletion , and replacement . Such adversarial text generation approaches can not guarantee the grammar correctness of generated text . For instance , text generated by Liang et al . ( 2017 ) are almost random stream of characters . To generate grammarly correct perturbation , Jia & Liang ( 2017 ) adopt another heuristic strategy which adds manually constructed legit distracting sentences to the paragraph to introduce fake information . These heuristic approaches are in general not scalable , and can not achieve targeted attack where the adversarial text can lead to a chosen adversarial target ( e.g . adversarial label in classification ) . Recent work searches for a universal trigger ( Wallace et al. , 2019 ) to be applied to arbitrary sentences to fool the learner , while the reported attack success rate is rather low . In contrast , with the tree based autoencoder , the proposed AdvCodec framework is able to generate grammarly correct adversarial text efficiently , achieving high attack success rates on different models . 3 THE ADVCODEC FRAMEWORK FOR ADVERSARIAL TEXT GENERATION . We describe the AdvCodec framework in this section . As illustrated in Figure 1 , the key component of the AdvCodec framework is a tree-based autoencoder . The hierarchical and discrete nature of language motivates us to make use of tree-based autoencoder to map discrete text into a high dimensional latent space , which empowers us to leverage the existing optimization based attacking method such as Carlini & Wagner ( 2016 ) to both efficiently and effectively generate adversarial text . Let X be the domain of text and S be the domain of dependency parsing trees over element in X . Formally , a tree-based autoencoder consists of an encoder E : X × S → Z that encodes text x ∈ X along with its dependency parsing tree s ∈ S into a high dimensional latent representation z ∈ Z and a decoder G : Z × S → X that generates the corresponding text x from the given context vector z and the expected dependency parsing tree s. Given a dependency tree s , E and G form an antoencoder . We thus have the following reconstruction loss to train our tree-based autoencoder : L = −Ex∼X [ log pG ( x|s , E ( x , s ) ] ( 1 ) As Figure 1 suggests , AdvCodec can operate on different granularity levels to generate either wordlevel or sentence-level contextual representation , and decode it into the adversarial text . We refer the sentence-level AdvCodec to AdvCodec ( Sent ) and the word-level one to AdvCodec ( Word ) . Both of them will be described in more details in the later part of this section . 3.1 OVERVIEW OF THE ADVCODEC FRAMEWORK Before diving into details , we provide a high level overview of AdvCodec according to the attack scenario and attack capability supported by this framework . Attack Scenario . Different from the previous adversarial text generation works ( Lei et al. , 2018 ; Cheng et al. , 2018 ; Papernot et al. , 2016 ; Miyato et al. , 2016 ; Alzantot et al. , 2018 ) that directly modify critical words in place and might risk changing the semantic meaning or editing the ground truth answers , we are generating the concatenative adversaries . First proposed by Jia & Liang ( 2017 ) , the concatenative adversary does not change any words in the original paragraph or question , but instead appends a new adversarial sentence to the paragraph to fool the model . However , the concatenative attack also needs to ensure the appended sentence is compatible ( Jia & Liang , 2017 ) with the original paragraph , which in other words means it should not contradict any stated facts in the paragraph , especially the correct answer . In our work , we further push the concept of concatenative adversaries further and propose a more general notion called scatter attack , which means we can inject adversarial words sporadically over the whole paragraph . The concatenative adversarial example falls into our case when those adversarial tokens form a sentence and on the same time the semantic meaning of the sentence does not contradict the original paragraph . Examples of concatenative attack and scatter attack is shown in table 1 . Attack Capability . AdvCodec is essentially an optimization based framework to find the adversarial texts with the optimization goal set to achieve targeted attack . For the sentiment classification task , AdvCodec can perform targeted attack to make the original positive reviews be classified as the most negative one , and vice versa . Particularly in the QA task , we design and implement two kinds of targeted attack : position targeted attack and answer targeted attack . A successful position targeted attack means the model can be fooled to output the answers at specific targeted positions in the paragraph , but the content on the targeted span can not be guaranteed . In contrast , a successful answer targeted attack is a stronger targeted attack , which refers to the situation when the model always outputs the preset targeted answer pair on the target no matter what the question looks like . An example of word targeted attack can be found in the table 1 . Although our framework is designed as a whitebox attack , our experimental results demonstrate our whitebox generated adversarial words can transfer to other blackbox models with high attack success rate . Finally , because AdvCodec is a unified adversarial text generation framework whose outputs are discrete tokens , it can be applied to different downstream NLP tasks . In this paper , we perform adversarial evaluation on sentiment classification and QA as examples to demonstrate how our framework is adapted to different works . 3.2 ADVCODEC ( SENT ) In this subsection , we describe AdvCodec ( Sent ) and explain how to utilize it to attack sentiment classification models and question answering systems . The main idea comes from the fact that tree structures sometimes have better performances than sequential recurrent models ( Li et al. , 2015 ; Iyyer et al. , 2014 ; 2018 ) and the fact that it is inherently flexible to add perturbations on hierarchical nodes of the tree structures . Motivated by this , we design a novel tree-based autoencoder to simultaneously preserve similar semantic meaning and original syntactic structure . Encoder . We adopt the Stanford Treestructured LSTM ( Tai et al. , 2015 ) as our tree encoder . In the encoding phase , features are extracted and summed from bottom ( leaf node , i.e . word ) to top ( root node ) along the dependency tree , extracted by Stanford CoreNLP Parser ( Manning et al. , 2014 ) . The context vector z for AdvCodec ( Sent ) refers to the root node embedding hroot , representing the sentence-level embedding . Decoder . Following the same dependency tree , we design the text decoder as illustrated in Figure 2 . In the decoding phase , we start from the root node and traverse along the dependency tree in level-order . The hidden state hj of the next node j comes from ( i ) the hidden state hi of the current tree node , ( ii ) current node predicted word embedding wi , and ( iii ) the dependency embedding dij between the current node i and the next node j based on the dependency tree . The next node ’ s corresponding word yj is generated based on the output of the LSTM Cell oj via a linear layer that maps from the hidden presentation oj to the logits that represent the probability distribution of the tree ’ s vocabulary . oj , hj = LSTM ( [ hi ; wi ; dij ] ) ( 2 ) yj = W · oj + b ( 3 )

Summary: This paper proposes a new attack framework AdvCodec for adversarial text generation. The main idea is to use a tree-based autoencoder to embed text data into the continuous vector space and then optimize to find the adversarial perturbation in the vector space. The authors consider two types of attacks: concat attack and scatter attack. Experimental results on sentiment analysis and question answering, together with human evaluation on the generated adversarial text, are provided.