A Geometry-Inspired Attack for Generating Natural Language Adversarial Examples (2020)

Abstract: Generating adversarial examples for natural language is hard, as natural language consists of discrete symbols, and examples are often of variable lengths. In this paper, we propose a geometry-inspired attack for generating natural language adversarial examples. Our attack generates adversarial examples by iteratively approximating the decision boundary of Deep Neural Networks (DNNs). Experiments on two datasets with two different models show that our attack fools natural language models with high success rates, while only replacing a few words. Human evaluation shows that adversarial examples generated by our attack are hard for humans to recognize. Further experiments show that adversarial training can improve model robustness against our attack.

## introduction although deep neural networks (dnns) have been successful in many machine learning applications @xcite @xcite , researchers have demonstrated that dnns are remarkably vulnerable to adversarial attacks, which generate adversarial examples by adding small perturbations to the original input @xcite @xcite . adversarial examples are essential as they showcase the limitations of dnn models. like humans, good dnn models should be robust to small perturbations to inputs. if a dnn model judges two almost identical inputs differently, one must profoundly question the quality of the dnn. as such adversarial examples are more than just a gimmick: they are a proof of the fundamental limitations of a dnn model. previous research on adversarial attacks has been largely focused on images, e.g., @xcite . in this paper, we study how to adversarially attack natural language models. generating adversarial examples for natural language is fundamentally different from generating adversarial examples for images. images live in a continuous universe, where one can simply change pixel values. natural language sentences and words on the other hand are typically discrete. this discrete nature makes it difficult to apply existing attacks from the image domain directly to natural language, as an arbitrary point in the input space is unlikely to correspond to a valid natural language sentence or word. moreover, inputs of natural language to dnns are of variable lengths, which further complicates generating adversarial examples for natural language. despite these obstacles, researchers have proposed various attacks to generate adversarial examples for natural language. @xcite manage to fool a dnn model for machine reading by adding sentences to the original texts. @xcite generate adversarial examples for natural language by using an autoencoder. @xcite propose a gradient-based attack to generate adversarial examples in the granularity of individual characters. @xcite al. (2019) however, all these methods do not address the "geometry" of dnns, which has been shown to be a useful approach in the image domain @xcite @xcite . in this paper, we propose a geometry-inspired attack for generating natural language adversarial examples. our attack generates adversarial examples by iteratively approximating the decision boundary of dnns. we conduct experiments with convolutional neural networks (cnns) and recurrent neural networks (rnns) on two text classification tasks: the imdb movie review dataset, and ag's news dataset. experimental results show that our attack fools the models with high success rates while keeping the word replacement rates low. we also conduct a human evaluation, showing that adversarial examples generated by our attack are hard for humans to recognize. further experiments show that model robustness against our attack can be achieved by adversarial training. ## related work despite the success of deep neural networks (dnns) in many machine learning applications @xcite , researchers have revealed that such models are vulnerable to adversarial attacks, which fool dnn models by adding small perturbations to the original input @xcite . the vulnerability of dnn models poses threats to many applications requiring high-level security. for example, in the image domain, a small error in a self-driving car could lead to life threatening disaster. for natural language, a machine might misunderstand a meaning, coming to a wrong conclusion. researchers have also shown that a universal trigger could lead a system to generate highly offensive language @xcite . previously, researchers have developed various adversarial attacks for fooling dnn models for images. @xcite propose fast gradient signed method (fgsm), which aims to maximize the loss of the model with respect to the correct label. projected gradient descent (pgd) @xcite can be viewed as a multi-step version of fgsm. in each step, pgd generates a perturbation using fgsm, and then projects the perturbed input to an l ∞ ball. while these gradient-based methods are effective, researchers also show that leveraging geometry information of dnns can be helpful. @xcite and @xcite generate adversarial examples by iteratively approximating the decision boundary of dnns. although many methods have been proposed for generating adversarial examples for images, little attention has been paid to generating adversarial examples for natural language. generating adversarial examples for natural language is fundamentally different from generating adversarial examples for images. on the one hand, while pixel values of images are continuous, natural language consists of sequences of discrete symbols. moreover, natural language sentences and words are often of variable lengths. hence, existing adversarial attacks designed for images cannot be directly applied to natural language. despite obstacles, researchers have proposed various methods for generating adversarial examples for natural language. based on the granularity of adversarial perturbations, adversarial attacks for natural language models can be divided into three categories: character level, word level and sentence level. ## methodology our attack is a white-box attack in that the attacker has access to the architecture and parameters of the victim model. the attack crafts natural language adversarial examples by replacing original words with their synonyms. specifically, our attack can be divided into two steps: word selection and synonym replacement. in each iteration, the attack first selects a word from the original text, and then replaces the selected word with one of its synonyms to craft an adversarial example. the remainder of this section gives the details of our attack. ## experimental results we elaborate our experiments in this section foot_1 . section 4.1 details the experimental settings. section 4.2 describes the results of adversarial attacks. we conduct a human evaluation in section 4.3 to understand the perceptibility of our adversarial perturbations. section 4.4 gives the results of adversarial training, which we found can improve the robustness of dnn models against our attack. ## conclusion and future work in this paper, we propose a geometry-inspired attack for generating natural language adversarial examples. our attack generates adversarial examples by iteratively approximating the decision boundary of deep neural networks. experiments on two text classification tasks with two models show that our attack reaches high success rates while keeping low word replacement rates. human evaluation shows that adversarial examples generated by our attack are hard to recognize for humans. experiments also show that adversarial training increases model robustness against our attack. our current attack works for models with context-independent word embeddings. in the future, we would like to extend our attack to models using contextualized word embeddings, including elmo @xcite , bert @xcite , etc.