Training NLI Models Through Universal Adversarial Attack (2023)

Abstract: “Pre-trained language models are sensitive to adversarial attacks, and recent works have demon-strated universal adversarial attacks that can apply input-agnostic perturbations to mislead mod-els. Here, we demonstrate that universal adversarial attacks can also be used to harden NLPmodels. Based on NLI task, we propose a simple universal adversarial attack that can misleadmodels to produce the same output for all premises by replacing the original hypothesis with anirrelevant string of words. To defend against this attack, we propose Training with UNiversalAdversarial Samples (TUNAS), which iteratively generates universal adversarial samples andutilizes them for fine-tuning. The method is tested on two datasets, i.e., MNLI and SNLI. It isdemonstrated that, TUNAS can reduce the mean success rate of the universal adversarial attackfrom above 79% to below 5%, while maintaining similar performance on the original datasets. Furthermore, TUNAS models are also more robust to the attack targeting at individual samples:When search for hypotheses that are best entailed by a premise, the hypotheses found by TUNASmodels are more compatible with the premise than those found by baseline models. In sum, weuse universal adversarial attack to yield more robust models. Introduction”

## introduction pre-trained models have achieved impressive performance among natural language processing (nlp) tasks, including natural language inference (nli) and machine reading comprehension (mrc) @xcite . nevertheless, these models are vulnerable under adversarial attacks @xcite . for most adversarial attack methods, the adversarial samples are input-specific, i.e., the adversarial perturbation is targeted at a specific input. more recently, however, studies have also shown the existence of universal adversarial attacks, which are input-agnostic @xcite . multiple methods have been proposed to find universal adversarial samples. one method is to append an input-agnostic string of words to any input to convert the input into an adversarial sample. for example, @xcite use gradient-based search to find strings that, when concatenated to any input, could result in specific model output. for instance, for models trained on snli, prepending "nobody" to the hypothesis could @xmath0, even when all the tested hypotheses are in fact entailed by the premises. another method is to randomly sample a large number of sentences and screen for universal adversarial samples. for example, @xcite use such a method to find sentences that a model always judges as the correct answer to multiple-choice mrc questions. the mainstream method to increase the robustness of models against adversarial attacks is adversarial training @xcite @xcite . in this process, adversarial samples are generated and injected into the training batch. adversarial training generally focuses on input-specific attacks, which involve small perturbations and targeting at individual samples. therefore, models fine-tuned with these methods still fail in universal adversarial attacks @xcite . besides, unlike input-specific attacks, universal attacks use single perturbation to cause the model fail in lots of samples, making it more effective to generate adversarial samples. recently, in the domain figure 1 : examples of the nli task and universal adversarial attack method adopted in this work. the model originally output the correct answers. nonetheless, when ubs, i.e., "a exceeds lowly herein1974", is presented as the hypothesis, the model is fooled to give out entailment prediction, even though they are actually irrelevant. of vision, some studies have also proposed to use universal adversarial samples for adversarial training @xcite , which is proved to be helpful for improving the robustness of the models. nonetheless, in the domain of nlp, efficient training with universal adversarial samples appears to be more challenging. generally, universal adversarial attacks for nlp models are achieved by appending an input-agnostic adversarial sequence to the input. training with such adversarial samples can easily lead to a degenerated solution of ignoring the appended adversarial sequence @xcite . to avoid such degenerated solutions, we propose a new universal adversarial attack method, where the adversarial samples are created by directly replacing specific components of the input with adversarial sequence. this work is based on nli, a task requires models to judge whether a premise can entail a hypothesis. specifically, instead of appending an adversarial sequence to the hypothesis, we create adversarial samples by replacing the original hypothesis with a string of words, referred to as the universal biased strings (ubss), as shown in figure 1 . here, ubss are the strings wrongly judged as being entailed by a large number of premises by the model. for an effective ubs, the model judges that it is entailed by any premise. we automatically generate ubss, and present them as hypothesis sentence to fool the models. the advantage of using ubss for attack is that they are guaranteed to be irrelevant to individual premises, since no string can be entailed by all premises. notably, although this work is based on the nli task, it can be easily adapted to describe, e.g., sentence similarity judgement, question answering, and other tasks that requires the judgement of the relationship between two sentences. in the following, we first described the method to search for the ubss and then introduced training with universal adversarial samples (tunas), a simple but effective training method to augment models by iteratively finding and correcting universal adversarial samples. it was demonstrated that popular transformer-based models were vulnerable to universal adversarial attack, and the ubss achieved a mean success rate higher than 79%, i.e., the model judged @xmath1. when the models were fine-tuned using tunas, however, the mean success rate of ubss dropped @xmath2. furthermore, when searching for strings that could be best entailed by a particular premise, the strings found by a model fine-tuned with tunas were more reasonable compared with that found by a baseline model. algorithm 1 ubs generation (gradient-based search) input: input premises, p ; vocabulary, v ; target model, f ; embedding layer, e; loss function, loss; parameter: search times, t ; ubs length, l; iterations, n ; candidates number, k; return ubss number, m ; output: m ubss 1: result ← ∅ 2: for i ← 1 to t do repeat search procedure for t times 3: result ← result + searchingbiasedstringsstep(...) 4: end for 5: return result 6: function searchingbiasedstringsstep 7: u bs ← s 0:l , @xmath3: memory ← ∅ 9: for iteration ← 1 to n do select candidates for each token in ubs 10: v cand ← top-k @xmath4, u bs), entailment), k) 11: for i ← 0 to l do for each token position 12: for @xmath5: u bs ← u bs 0:i ⊕ t ⊕ u bs i+1:l generate potential ubss 14: memory[u bs ] ← -loss(f (p, u bs ), entailment) evaluate potential ubss 15: end for 16: u bs ← arg max @xmath6: end for 18: end for 19: return top-k @xmath7, m ) 20: end function the other being the hypothesis, and a label indicating the relation between the premise and hypothesis, i.e., entailment, contradiction, or neutral. we tested three mainstream pre-trained transformer models, i.e., bert (devlin et al., 2019), roberta (liu et al., 2019), and deberta-v3 (he et al., 2020), and considered both the base version and large version of the models. the pre-trained models were provided by huggingface (wolf et al., 2020) and were fine-tuned based on snli or mnli, respectively. during fine-tuning, the inputs were formatted as [cls, premise, sep , hypothesis, sep ] ## biased strings for individual premises tunas could effectively increase the robustness to the ubs attack. the ubs attack, however, were particularly strong attacks that utilized a single word string to attack all possible premises. next, we evaluated whether tunas could also increase the robustness to attacks targeting at individual premises. ## related work and discussion adversarial attack. generally, the adversarial attacks are input-specific, which generate specialized perturbations for each input. @xcite attack the reading comprehension models by adding a distractor sentence to the input paragraph. @xcite use natural attacks to cause semantic collisions, i.e., irrelevant sentence pairs are judged to be similar by the nlp models. in these methods, an extra evaluation should be used to verify the golden labels of the adversarial samples. in this paper, we avoid human evaluation by generating ubss, which are inherent to be neutral with most of the premises. universal adversarial attacks are input-agnostic. @xcite and @xcite oncurrently propose to perform gradient-based search strategies to generate input-agnostic sequences, referred to as triggers, that can cause a model to output a specific prediction when concatenated to any input. @xcite extend it to generate natural triggers. @xcite propose a data-free attack method. most of the previous works construct the attack based on appending strategy, and aim at generating and analyzing universal adversarial triggers. in this work, we propose to use ubss directly for attack, and aim at augmenting the models through universal adversarial samples. here, we do not use append strategy to avoid models from learning to ignore attack positions during augmentation. adversarial training. adversarial training is one of the most successful approaches for defending against adversarial attacks @xcite , where adversarial samples are used for training to improve the robustness of models. universal adversarial training has proven to be beneficial in the domain of computer vision @xcite , and malware classification @xcite . @xcite augment the training procedure for multi-choice models using magnet options: the options irrelevant to the questions are still prone to be selected as the answer by the models. our work is more extensive as we utilize a searching method for generating ubss automatically, which is more effective in digging out the biases of the models. in this work, we use ungrammatical ubss for adversarial training. although the ungrammatical ubss are unlikely to appear in real-world scenarios, they have potential to reveal the biases learned by the models. meanwhile, they can serve as a cheap method to augment the models. results suggest that the model augmented by ungrammatical ubss also perform better in defending grammatical ubss attack. moreover, this work is based on nli task, but the ubss generation and application can be extended to many nlp tasks. for example, in multiple-choice task, e.g., race @xcite , the model can be fooled to choose a certain biased option as the answer. in span extraction tasks, e.g., squad @xcite , the model can be fooled to always output a certain biased span. in these cases, it is still feasible to generate universal adversarial examples and use them for adversarial training. ## conclusion universal adversarial attacks are effective in revealing the shallow heuristics learned by the models @xcite . here, we propose tunas, which utilizes universal adversarial samples to harden the models. a simple yet effective universal adversarial attack method is designed by replacing the hypotheses with ubss, which can achieve above 79% success rate among 2 nli tasks. the ubss are generated automatically by gradient-based method. in tunas, the universal adversarial samples are generated and used to train the models. the models fine-tuned using tunas show robustness against ubs attack, c c l 2 0 2 3 while maintaining comparable task performance. moreover, when searching biased strings for individual premises, models fine-tuned using tunas could generate strings better entailed by the premise.