Flooding- X : Improving BERT ’s Resistance to Adversarial Attacks via Loss-Restricted Fine-Tuning (2022)

Abstract: Adversarial robustness has attracted much attention recently, and the mainstream solution is adversarial training. However, the tradition of generating adversarial perturbations for each input embedding (in the settings of NLP) scales up the training computational complexity by the number of gradient steps it takes to obtain the adversarial samples. To address this problem, we leverage Flooding method which primarily aims at better generalization and we find promising in defending adversarial attacks. We further propose an effective criterion to bring hyper-parameter-dependent flooding into effect with a narrowed-down search space by measuring how the gradient steps taken within one epoch affect the loss of each batch. Our approach requires zero adversarial sample for training, and its time consumption is equivalent to fine-tuning, which can be 2-15 times faster than standard adversarial training. We experimentally show that our method improves BERT’s resistance to textual adversarial attacks by a large margin, and achieves state-of-the-art robust accuracy on various text classification and GLUE tasks.

## introduction despite their impressive performances on various nlp tasks, deep neural networks such as bert @xcite suffer a sharp performance degradation against deliberately constructed adversarial attacks @xcite @xcite @xcite . a line of work attempts to alleviate this problem by creating adversarially robust models via defense methods, including adversarial data augmentation @xcite , regularization @xcite , and adversarial training @xcite @xcite . data augmentation and adversarial training rely on additional adversarial examples generated either by hand-crafting or conducting gradient ascent on the clean data for virtual adversarial samples. however, generating adversarial examples scales up the cost of training computationally, which makes vanilla adversarial training almost impractical on large-scale nlp tasks like qnli (questionanswering nli, @xcite ). an increasing amount of researchers express their concern about the time-consuming property of standard adversarial training and offer cheaper but competitive alternatives by (i) replacing the perturbation generation with an additional generator network @xcite , or by (ii) combining the gradient computation of clean data and perturbations into one backward pass @xcite . these approaches still rely on additional adversarial examples generated either by the model itself or by an extra module. in this work, we propose a novel method, flooding-x, to largely improve adversarial robustness without any adversarial examples, maintaining the same computational cost as conventional bert fine-tuning. the vanilla flooding @xcite ) method is a practical regularization technique to boost model generalization by preventing further reduction of the training loss when it reaches a reasonably small value. it results in a model performing normal gradient descent when training loss is above the decided value but gradient ascent when below. by continuing to "random walk" with the same non-zero value as a "virtual loss", the model drifts into an area with a flat loss landscape that is claimed to lead to better generalization @xcite . interestingly, we find that flooding method is also promising in increasing models' resistance to adversarial attacks. despite the significant rise in robust accuracy, the so-called reasonably small value, which is a hyperparameter, takes effort to be found and varies for each dataset, which requires an overly extensive search among the numerous candidates. in an attempt to narrow down the candidates of hyper-parameter, we propose gradient accordance as an informative criterion for optimal values that bring flooding into effect, which is used as a building-block in flooding-x. we measure how accordant the gradients of the batches are by analyzing how the gradient descent steps based on part of an epoch affect the loss of each batch. gradient accordance is computationally friendly and is tractable during training process. experiments on various tasks show a close relation between gradient accordance and overfitting. as a result, we propose gradient accordance as a reliable flooding criterion to make the training loss flood around the level when the model has nearly overfitted. that is to say, we leverage the training loss of the model right before overfitting as the value of flood level. flooding-x is especially useful and shows great advantage over adversarial training in terms of computational cost when the training dataset is relatively large. experimental results demonstrate that our method achieves stated-of-the-art robust accuracy with bert on various tasks and improves its robust accuracy by 100 to 400% without using any adversarial example, consuming any extra training time, or conducting overly extensive search for hyper-parameter. our main contributions are as follows. ## gradient accordance as a criterion for flooding since flooding is proposed as an attempt to avoid overfitting, we intuitively suppose that the optimal flood level would be found at the stage when the model is about to overfit. that is, we leverage the training loss before overfitting as the flood level. inspired by influence function @xcite , we propose gradient accordance as a criterion for flooding, which is empirically proved to be reliable and indicative. we consider the effect of the model updated w.r.t. one epoch on each of its batches as a signal of overfitting. as is indicated by its name, this criterion measures the relation among the gradients of each batch on epoch level, evaluating whether the model updated on an epoch has the same positive effect on the batches on average. now we provide the formal definition of gradient accordance. ## experiments in this section, we provide comprehensive analysis on flooding-x through extensive experiments on five text classification datasets of various tasks and scales: sst @xcite , mrpc @xcite , qnli @xcite , imdb @xcite and ag news @xcite . the statistics of these involved datasets are illustrated in ## analysis and discussion in this section, we construct supplementary experiments to further analyze the effectiveness of flooding-x and its building block, i.e., gradient accordance. ## conclusion in this work, we propose flooding-x as an efficient and computational-friendly algorithm for improving bert's resistance to adversarial attacks. we first theoretically prove that the vanilla flooding method is able to boost model's adversarial robustness by leading it into a smooth parameter landscape. we further propose a promising and computationally tractable criterion, gradient accordance, to detect when the model is about to overfit and accordingly narrow down the hyperparameter space for flooding with an optimal flood level guaranteed. experimental results prove that gradient accordance is closely related with the phenomenon of overfitting, equipped with which flooding-x beats the well-received adversarial training methods and achieves state-of-the-art performances on various nlp tasks against different textual attack methods. this implies that adversarial examples, either generated by gradient-based algorithms or human efforts, are not a must for the improvement of adversarial robustness. we call for further exploration and deeper understanding in the nature of adversarial robustness and attacks.