Balanced Adversarial Training: Balancing Tradeoffs between Fickleness and Obstinacy in NLP Models (2022)

Abstract: Traditional (fickle) adversarial examples involve finding a small perturbation that does not change an input’s true label but confuses the classifier into outputting a different prediction. Conversely, obstinate adversarial examples occur when an adversary finds a small perturbation that preserves the classifier’s prediction but changes the true label of an input.Adversarial training and certified robust training have shown some effectiveness in improving the robustness of machine learnt models to fickle adversarial examples. We show that standard adversarial training methods focused on reducing vulnerability to fickle adversarial examples may make a model more vulnerable to obstinate adversarial examples, with experiments for both natural language inference and paraphrase identification tasks. To counter this phenomenon, we introduce Balanced Adversarial Training, which incorporates contrastive learning to increase robustness against both fickle and obstinate adversarial examples.

## introduction interpreted broadly, an adversarial example is an input crafted intentionally to confuse a model. most research on adversarial examples, however, focuses on a definition of an adversarial example as an input that is constructed by making minimal perturbations to a normal input that change the model's output, assuming that the small perturbations preserve the original true label @xcite . such adversarial examples occur when a model is overly influenced by small changes in the input. attackers can also target the opposite objective-to find inputs with minimal changes that change the ground truth label but for which the model retains its prior prediction @xcite . various names have been used in the research literature for these two types of adversarial examples including perturbation or sensitivity-based and invariance-based examples @xcite , and over-sensitive and over-stable examples @xcite . to avoid confusions associated with these names, we refer them as fickle adversarial examples (the model changes its output too easily) and obstinate adversarial examples (the model doesn't change its output even though the input has changed in a way that it should). in nlp, synonym-based word substitution is a common method for constructing fickle adversarial examples @xcite since synonym substitutions are assumed to not change the true label for an input. these methods target a model's weakness of being invariant to certain types of changes which makes its predictions insufficiently responsive to small input changes. attacks based on antonyms and negation have been proposed to create obstinate adversarial examples for dialogue models @xcite . adversarial training is considered as the most effective defense strategy yet found against adversarial examples @xcite . it aims to improve robustness by augmenting the original training set with generated adversarial examples in a way that results in decision boundaries that correctly classify inputs that otherwise would have been fickle adversarial examples. adversarial training has been shown to improve robustness for nlp models (yoo and @xcite . recent works have also studied certified robustness training which gives a stronger guarantee that the model is robust to all possible perturbations of a given input @xcite . while prior work on nlp robustness focuses on fickle adversarial examples, we consider both fickle and obstinate adversarial examples. we then further examine the impact of methods designed to improve robustness to fickle adversarial examples on a model's vulnerability to obstinate adversarial examples. recent work in the vision domain demonstrated that increasing adversarial robustness of im-figure 1 : distance-oracle misalignment @xcite . while the model is trained to be robust to ϵbounded perturbation, it becomes too invariant to small changes in the example (obstinate example x) that lie on the other side of the oracle decision boundary. contributions. we study fickle and obstinate adversarial robustness in nlp models with a focus on synonym and antonym-based adversarial examples (figure 2 shows a few examples). we evaluate both kinds of adversarial robustness on natural language inference and paraphrase identification tasks with bert @xcite and roberta @xcite models. we find that there appears to be a tradeoff between robustness to synonym-based and antonym-based attacks. we show that while certified robust training increases robustness against synonym-based adversarial examples, it increases vulnerability to antonym-based attacks (section 3). we propose a modification to robust training, balanced adversarial training (bat), which uses a contrastive learning objective to help mitigate the distance misalignment problem by learning from both fickle and obstinate examples (section 4). we implement two versions of bat with different contrastive learning objectives, and show the effectiveness in improving both fickleness and obstinacy robustness (section 4.2). ## constructing adversarial examples we consider a classification task where the goal of the model f is to learn to map the textual input x, a sequence of words, x 1 , x 2 , ..., x l , to its ground truth label @xmath0, ..., c}. we assume there is a labeling oracle o that corresponds to ground truth and outputs the true label of the given input. we focus on word-level perturbations where the attacker substitutes words in the original input x with words from a known perturbation set (which we show how we construct it in the following sections). the goal of the attacker is to find an adversarial example x for input x such that the output of the model is different from what human would interpret, i.e. f (x) ̸ = o(x). ## robustness tradeoffs normally, adversarial defense methods only target fickle adversarial examples, so there is a risk that such methods increase vulnerability to obstinate adversarial examples. according to the distance-oracle misalignment assumption @xcite as depicted in figure 1 , the distance measure for finding adversarial examples and labeling oracle o is misaligned if we have ## balanced adversarial training in previous section, we argued that the tradeoff between fickleness and obstinacy can be attributed to distance-oracle misalignment. this section proposes and evaluates a modification to adversarial training that balances both kinds of adversarial examples. ## related work compared to fickle adversarial examples, obstinacy has been less studied in nlp as well as other domains. feng et al. (2018) delete words iteratively from the input to create examples that appear rubbish to human but retain the model's prediction with high confidence. @xcite use partof-speech and name entity based perturbations against reading comprehension models. @xcite study both types of attack strategies for dialogue models. they create obstinate adversarial examples by substituting words with antonyms or adding negation words to the input. our work is the first to study tradeoffs between fickle and obstinate adversarial examples in nlp, but a few previous works have considered these tradeoffs in the vision domain. jacobsen et al. (2019b) show that adversary can not only target the model's excessive sensitivity but its excessive invariance to small changes in the input. they propose an alternative training objective based on information theory to make the model less invariant to semantically meaningful changes. tramer et al. (2020) study the tradeoff between the two types of adversarial examples for image classifiers. they show that data augmentation can help increase robustness against obstinacy attacks, but is not sufficient to impede both types of attacks. our work differs in that we propose a new adversarial training method that improves model robustness against both types of adversarial examples. in addition, unlike images where human inspection is usually required to check whether the perturbed pixels would change the true label of the image, we are able to automate the process of generating obstinate examples for text. recent work introduce contrastive learning for image classifiers in the adversarial learning set-ting where an fickle adversarial augmentation is used to generate positive examples and negative examples are sampled from other images. @xcite generate diverse positive examples by launching instance-wise attack on augmented images and show that it improves model's fickleness robustness. ho and nvasconcelos (2020) create challenging positive pairs by using the gradients of the contrastive loss to generate fickle adversarial examples and they show that it improves model performance. ## conclusion we demonstrate the tradeoff between vulnerability to synonym-based (fickle) and antonym-based (obstinate) adversarial examples for nlp models and show that increasing robustness against synonym based attacks also increases vulnerability to antonym-based attacks. to manage this tension, we introduce a new adversarial training method, bat, which targets the distance-oracle misalignment problem and can help balance the fickleness and obstinacy in adversarial training.