Grey-box Adversarial Attack And Defence For Sentiment Classification (2021)

Abstract: We introduce a grey-box adversarial attack and defence framework for sentiment classification. We address the issues of differentiability, label preservation and input reconstruction for adversarial attack and defence in one unified framework. Our results show that once trained, the attacking model is capable of generating high-quality adversarial examples substantially faster (one order of magnitude less in time) than state-of-the-art attacking methods. These examples also preserve the original sentiment according to human evaluation. Additionally, our framework produces an improved classifier that is robust in defending against multiple adversarial attacking methods. Code is available at: https://github.com/ibm-aur-nlp/adv-def-text-dist .

## introduction recent advances in deep neural networks have created applications for a range of different domains. in spite of the promising performance achieved by neural models, there are concerns around their robustness, as evidence shows that even a slight perturbation to the input data can fool these models into producing wrong predictions @xcite . research in this area is broadly categorised as adversarial machine learning, and it has two sub-fields: adversarial attack, which seeks to generate adversarial examples that fool target models; and adversarial defence, whose goal is to build models that are less susceptible to adversarial attacks. a number of adversarial attacking methods have been proposed for image recognition @xcite , nlp @xcite and speech recognition @xcite . these methods are generally categorised into three types: whitebox, black-box and grey-box attacks. white-box attacks assume full access to the target models and often use the gradients from the target models to guide the craft of adversarial examples. black-box attacks, on the other hand, assume no knowledge on the architecture of the target model and perform attacks by repetitively querying the target model. different from the previous two, grey-box attacks train a generative model to generate adversarial examples and only assume access to the target model during the training phrase. the advantages of greybox attacking methods include higher time efficiency; no assumption of access to target model during attacking phase; and easier integration into adversarial defending algorithms. however, due to the discrete nature of texts, designing grey-box attacks on text data remains a challenge. in this paper, we propose a grey-box framework that generates high quality textual adversarial examples while simultaneously trains an improved sentiment classifier for adversarial defending. our contributions are summarised as follows: most white-box methods are gradient-based, where some form of the gradients (e.g. the sign) with respect to the target model is calculated and added to the input representation. in image processing, the fast gradient sign method (fgsm; goodfellow et al. (2014)) is one of the first studies in attacking image classifiers. some of its variations include kurakin et al. (2016); dong et al. (2018). these gradient-based methods could not be applied to texts directly because perturbed word embeddings do not necessarily map to valid words. methods such as deepfool (moosavi-dezfooli et al., 2016) that rely on perturbing the word embedding space face similar roadblocks. to address the issue of embedding-to-word mapping, @xcite propose to use nearestneighbour search to find the closest words to the perturbed embeddings. however, this method treats all tokens as equally vulnerable and replace all tokens with their nearest neighbours, which leads to non-sensical, word-salad outputs. a solution to this is to replace tokens one-by-one in order of their vulnerability while monitoring the change of the output of the target models. the replacement process stops once the target prediction has changed, minimising the number of changes. examples of white-box attacks that utilise this approach include tyc @xcite and hot-flip @xcite . different to white-box attacks, black-box attacks do not require full access to the architecture of the target model. @xcite propose to estimate the loss function of the target model by querying its label probability distributions, while @xcite propose to construct a substitute of the target model by querying its output labels. the latter approach is arguably more realistic because in most cases attackers only have access to output labels rather than their probability distributions. there is relatively fewer studies on black-box attacks for text. an example is textfooler, proposed by @xcite , that generates adversarial examples by querying the label probability distribution of the target model. another is proposed by @xcite where genetic algorithm is used to select the word for substitution. grey-box attacks require an additional training process during which full access to the target model is assumed. however, post-training, the model can be used to generate adversarial examples without querying the target model. @xcite introduce a generative adversarial network to generate the image perturbation from a noise map. it is, however, not trivial to adapt the method for text directly. it is because text generation involves discrete decoding steps and as such the joint generator and target model architecture is non-differentiable. in terms of adversarial defending, the most straightforward method is to train a robust model on data augmented by adversarial examples. recently, more methods are proposed for texts, such as those based on interval bound propagation @xcite , and dirichlet neighborhood ensemble @xcite . ## methodology the purpose of adversarial attack is to slightly perturb an input example x for a pre-trained target model (e.g. a sentiment classifier) f so that f (x) = y, where y is the ground truth of x. the perturbed example x should look similar to x, which can be measured differently depending on the domain of the input examples. ## conclusion in this paper, we proposed a grey-box adversarial attack and defence framework for sentiment classification. our framework combines a generator with two copies of the target classifier: a static and an updated model. once trained, the generator can be used for generating adversarial examples, while the augmented (updated) copy of the classifier is an improved model that is less susceptible to adversarial attacks. our results demonstrate that the generator is capable of producing high-quality adversarial examples that preserve the original ground truth and is approximately an order of magnitude faster in creating adversarial examples compared to stateof-the-art attacking methods. our framework of building an improved classifier together with an attacking generator is also shown to be more effective than the baseline approach of training a classifier using data augmented by adversarial examples. the combined adversarial attack and defence framework, though only evaluated on sentiment classification, should be adapted easily to other nlp problems (except for the counter-fitted embeddings, which is designed for sentiment analysis). this framework makes it possible to train adversarial attacking models and defending models simultaneously for nlp tasks in an adversarial manner.