N atural A dversaries: Can Naturalistic Adversaries Be as Effective as Artificial Adversaries? (2022)

Abstract: While a substantial body of prior work has explored adversarial example generation for natural language understanding tasks, these examples are often unrealistic and diverge from the real-world data distributions. In this work, we introduce a two-stage adversarial example generation framework (NaturalAdversaries), for designing adversaries that are effective at fooling a given classifier and demonstrate natural-looking failure cases that could plausibly occur during in-the-wild deployment of the models. At the first stage a token attribution method is used to summarize a given classifier’s behavior as a function of the key tokens in the input. In the second stage a generative model is conditioned on the key tokens from the first stage. NaturalAdversaries is adaptable to both black-box and white-box adversarial attacks based on the level of access to the model parameters. Our results indicate these adversaries generalize across domains, and offer insights for future research on improving robustness of neural text classification models.

## introduction transformer models have gained prominence in nlp research due to their powerful performances on leaderboards. however, numerous studies have shown these neural models are brittle, frequently taking shortcuts to reach decisions rather than reasoning about the underlying semantics correctly @xcite or failing when exposed to adversarial perturbations of inputs (e.g, @xcite @xcite @xcite . due to the opaque nature of neural modeling, methods for adversarial example generation may also steer algorithms towards generating unlikely examples that exhibit unrealistic properties @xcite . in this work, we pose the question, "what does it really mean for an adversarial attack to be effective and can naturalistic adversaries match artificial ones?" we argue that effectiveness should be dependent not only on attacking accuracy, but on usefulness of adversaries for improving robustness under realistic conditions (e.g identifying social biases learned by neural models, @xcite @xcite @xcite . we propose a framework naturaladversaries foot_0 for generating convincingly naturalistic adversaries. we first approximate the behavior of a given classifier decision function f c (x) and then train a generative model f g (x) to mimic this behavior. as shown in figure 2 , we condition generative models on influential tokens extracted using black box or white box explainability methods @xcite , and a desired label (e.g. "entailment" or "contradiction"), to produce new examples that match a distribution learned from f c (x) through sampling. our results on two different tasks (hate speech detection and natural language inference) show that our approach leads to adversaries that are perceived by annotators as considerably more natural. while naturalistic adversaries are often less adversarial than artificial adversaries (following prior literature, e.g. @xcite , we find this depends on the evaluation setting and they can be better defenses. ## defining naturalness we first define what it means for machinegenerated adversaries to have the quality of "naturalness." prior work on text generation has defined this property in terms of linguistic competence @xcite , as well as enumerating undesirable characteristics that lower perceived naturalness like self-contradiction @xcite . in our work, we ask human figure 1 : averaged naturalness scores from human evaluation. the adversarial examples were generated from the dynahate test set @xcite using two common baselines (baseline) @xcite as well as naturaladversaries (ours). we show the distribution of scores for examples that are effective or ineffective respectively at fooling a roberta toxicity classification model @xcite . this shows that not only does naturaladversaries generate more natural examples, but naturalistic examples can also be effective at demonstrating adversarial behavior. input: influential tokens, y' y': entailment they did not seem happy. ## description of naturaladversaries our overall framework consists of two stages -(1) a probing stage where we identify the influential (high attribution) tokens and (2) an adversarial generative stage where we generate unseen challenging examples by conditioning on the extracted tokens and a reversed label ( §3.2). we focus on two types of explainability methods as a means of summarizing model behavior through sampling -(1) lime local linear explanation models @xcite and (2) gradient attribution scores @xcite . using these methods as part of our proposed method is advantageous since it doesn't require curation of cherry-picked examples to probe model behavior, and is agnostic to the specific internal structure of a given classifier. given a sequence of text tokens @xmath0, s 1 , . . . , s i , s i+1 , . . . ] with classifier label ŷ, each of these methods define a scoring function f attr (s i ) which we use for assigning attribution scores to each ## related work adversarial attacks prior work on adversarial attacks focus primarily on time-consuming and costly manual annotation @xcite , or automatic example construction that relies upon a predefined type of attack (e.g. testing robustness to syntactic and lexical errors, @xcite @xcite . the effect of complex adversarial attacks in the hate speech domain is also relatively unexplored. while @xcite recently address this, they do not consider naturalness. ## conclusion we introduce a framework for generation of naturalistic adversaries that is effective for multiple neural classifiers and across domains. we encourage further work on how naturalistic adversaries may improve robustness in real-world settings.