Exploring Robust Overfitting for Pre-trained Language Models (2023)

Abstract: We identify the robust overfitting issue for pre-trained language models by showing that the robust test loss increases as the epoch grows. Through comprehensive exploration of the robust loss on the training set, we attribute robust overfitting to the model’s memorization of the adversarial training data. We attempt to mitigate robust overfitting by combining regularization methods with adversarial training. Following the philosophy that prevents the model from memorizing the adversarial data, we find that flooding, a regularization method with loss scaling, can mitigate robust overfitting for pre-trained language models. Eventually, we investigate the effect of flooding levels and evaluate the models’ adversarial robustness under textual attacks. Extensive experiments demonstrate that our methods can mitigate robust overfitting upon three top adversarial training methods and further promote adversarial robustness.

## introduction deep neural networks (dnns) suffer from adversarial robustness issues @xcite @xcite . recent literature has revealed their vulnerability to crafted adversarial examples on a wide range of natural language processing (nlp) tasks @xcite @xcite . among the corresponding defensive methods, gradient-based adversarial training (at) is often considered as the most effective one. building upon standard training, at additionally solves a max-min optimization problem to learn an adversarially robust model @xcite @xcite . surprisingly, a widely observed fact is that at, which is challenging to optimize, can also converge quickly on pre-trained @xcite . "-10" indicates that the number of attack iterations used in at is 10. the robust test losses of the three top at methods only increase as the training epochs grow. in contrast, the robust training losses converge to zero within very few epochs. we further explore the robust loss and accuracy curves on the training set. more specifically, we re-perform a pgd-10 attack on the training set to check the robust learning curves. we surprisingly observe that on the training set, both the robust loss and robust error under pgd-10 converge to small values. we also evaluate the adversarial robustness under different settings, such as datasets, model architectures, etc., and similar results are observed. with extensive empirical results, we argue that the model overfits the threat model used in at and loses the adversarial generalization ability. we hypothesise that the model simply memorizes the adversarial data during training and fails to generalize to robust testing. thus a poor adversarial generalization performance is observed on the test set. we make several attempts to mitigate robust overfitting issues in at using a series of regularization methods. the underlying philosophy is to prevent the model from memorizing adversarial data. in this way, we prevent the adversarially trained model from robust overfitting. eventually, we evaluate our methods against textual adversarial attacks and obtain improvements upon the existing at methods. our contributions can be summarized as follows: ## related work in this section, we briefly review the relevant work on at and robust overfitting, especially for nlp tasks. ## robust overfitting for prlms in this section, we explore the robust learning curves for prlms. by comparing the data loss distributions between training and testing, we identify the robust overfitting issue and attribute it to the model's memorization of adversarial data. ## mitigating robust overfitting in this section, we make several attempts to prevent prlms from getting overfitting in at. in standard training, regularization methods can mitigate overfitting and promote test performance. thus, it is intuitive to use regularization methods in at to avoid robust overfitting. ## discussion in this section, we discuss the effect of flooding in at and investigate why flooding can help adversarial generalization. ## conclusion robust overfitting prevents further improvement of adversarial robustness on prlms. while we adopt strong regularizers in at, weight decay and dropout contribute little to mitigating robust overfitting. to prevent the model from simply memorizing the adversarial training data, we combine flooding with at. experimental results on extensive datasets and model architectures demonstrate that a reasonable flooding level helps mitigate robust overfitting. as a preliminary study, this work identifies the robust overfitting issue for prlms. we hope the community can take robust overfitting into account when performing at to achieve adversarially robust models.