Question: What are the criteria for a good adversarial example in the context of images?
Evidence: Adversarial examples, a term introduced in BIBREF0, are inputs transformed by small perturbations that machine learning models consistently misclassify. The experiments are conducted in the context of computer vision (CV), and the core idea is encapsulated by an illustrative example: after imperceptible noises are added to a panda image, an image classifier predicts, with high confidence, that it is a gibbon. Interestingly, these adversarial examples can also be used to improve the classifier — either as additional training data BIBREF0 or as a regularisation objective BIBREF1 — thus providing motivation for generating effective adversarial examples.
The germ of this paper comes from our investigation of adversarial attack methods for natural language processing (NLP) tasks, e.g. sentiment classification, which drives us to quantify what is an “effective” or “good” adversarial example. In the context of images, a good adversarial example is typically defined according two criteria:
it has successfully fooled the classifier;
it is visually similar to the original example.
In NLP, defining a good adversarial example is a little more involving, because while criterion (b) can be measured with a comparable text similarity metric (e.g. BLEU or edit distance), an adversarial example should also:
be fluent or natural;
preserve its original label.
These two additional criteria are generally irrelevant for images, as adding minor perturbations to an image is unlikely to: (1) create an uninterpretable image (where else changing one word in a sentence can render a sentence incoherent), or (2) change how we perceive the image, say from seeing a panda to a gibbon (but a sentence's sentiment can be reversed by simply adding a negative adverb such as not). Without considering criterion (d), generating adversarial examples in NLP would be trivial, as the model can learn to simply replace a positive adjective (amazing) with a negative one (awful) to attack a sentiment classifier.
To the best of our knowledge, most studies on adversarial example generation in NLP have largely ignored these additional criteria BIBREF2, BIBREF3, BIBREF4, BIBREF5. We believe the lack of a rigorous evaluation framework partially explains why adversarial training for NLP models has not seen the same extent of improvement compared to CV models. As our experiments reveal, examples generated from most attacking methods are successful in fooling the classifier, but their language is often unnatural and the original label is not properly preserved.
The core contribution of our paper is to introduce a systematic, rigorous evaluation framework to assess the quality of adversarial examples for NLP. We focus on sentiment classification as the target task, as it is a popular application that highlights the importance of criteria discussed above. We test a number of attacking methods and also propose an alternative approach (based on an auto-encoder) for generating adversarial examples. We learn that a number of factors can influence the performance of adversarial attacks, including architecture of the classifier, sentence length and input domain.
In the context of images, a good adversarial example is typically defined according to two criteria: (a) it has successfully fooled the target classifier; (b) it is visually similar to the original example.
Answer: (a) it has successfully fooled the target classifier; (b) it is visually similar to the original example.