DSRM : Boost Textual Adversarial Training with Distribution Shift Risk Minimization (2023)

Abstract: Adversarial training is one of the best-performing methods in improving the robustness of deep language models. However, robust models come at the cost of high time consumption, as they require multi-step gradient ascents or word substitutions to obtain adversarial samples. In addition, these generated samples are deficient in grammatical quality and semantic consistency, which impairs the effectiveness of adversarial training. To address these problems, we introduce a novel, effective procedure for instead adversarial training with only clean data. Our procedure, distribution shift risk minimization (DSRM), estimates the adversarial loss by perturbing the input data’s probability distribution rather than their embeddings. This formulation results in a robust model that minimizes the expected global loss under adversarial attacks. Our approach requires zero adversarial samples for training and reduces time consumption by up to 70% compared to current best-performing adversarial training methods. Experiments demonstrate that DSRM considerably improves BERT’s resistance to textual adversarial attacks and achieves state-of-the-art robust accuracy on various benchmarks.

## introduction despite their impressive performance on various nlp tasks, deep neural networks (dnns), like bert @xcite , are highly vulnerable to adversarial exemplars, which arise by adding imperceptible perturbations among natural samples under semantic and syntactic constraints @xcite . such vulnerability of dnns has attracted extensive attention in enhancing defence techniques against adversarial examples @xcite , where the adversarial training approach (at) @xcite is empirically one of the best-performing algorithms to train networks robust to adversarial perturbations * corresponding author. @xcite . formally, adversarial training attempts to solve the following min-max problem under loss function l: due to the non-convexity of neural networks, finding the analytic solution to the above inner maximization (marked in red) is very difficult @xcite . the most common approach is to estimate the adversarial loss from the results of several gradient ascents, such as pgd @xcite and freelb @xcite . @xcite and @xcite generate meaningful sentences by restricting such perturbations to the discrete token embedding space, achieving competitive robustness with better interpretability @xcite . however, the impressive performance in adversarial training comes at the cost of excessive computational consumption, which makes it infeasible for large-scale nlp tasks @xcite . for example, freelb++ @xcite , which increases the perturbation intensity of the freelb algorithm to serve as one of the state-of-the-art methods, achieves optimal performance with nearly 15 times the training time. moreover, the adversarial samples generated by the aforementioned methods exhibit poor grammatical quality, which is unreasonable in the real world when being manually reviewed @xcite . some works attempt to speed up the training procedure by obtaining cheaper adversarial samples @xcite or generating diverse adversarial samples at a negligible additional cost @xcite . however, they still require a complex process for adversarial samples and suffer performance degradation in robustness. in this work, from another perspective of the overall distribution rather than the individual adversarial samples, we ask the following question: can we directly estimate and optimize the expectation of the adversarial loss without computing specific perturbed samples, thus circumventing the abovementioned problems in adversarial training? dsrm formalize the distribution distance between clean and adversarial samples to answer the question. our methodology interprets the generation of adversarial samples as an additional sampling process on the representation space, whose probability density is not uniformly distributed like clean samples. adversarial samples with higher loss are maximum points in more neighbourhoods and possess a higher probability of being generated. we subsequently proved that the intensity of adversarial perturbations naturally bound the wasserstein distance between these two distributions. based on this observation, we propose an upper bound for the adversarial loss, which can be effectively estimated only using the clean training data. by optimizing this upper bound, we can obtain the benefits of adversarial training without computing adversarial samples. in particular, we make the following contributions: ## methodology in this section, we first introduce our distribution shift risk minimization (dsrm) objective, a novel upper bound estimation for robust optimization, and subsequently, how to optimize the model parameters under dsrm. throughout our paper, we denote vectors as a, sets as a, probability distributions as p, and definition as . specificly, we denote an all-1 vector of length b as 1 1×b . considering a model parameterized by θ ∈ θ, the per-data loss function is denoted as l(θ, x, y) : θ × x × @xmath0. observing only the training set s t , the goal of model training is to select model parameters θ that are robust to adversarial attacks. ## experiments in this section, we comprehensively analyse dsrm versus other adversarial training methods in three evaluation settings for three tasks. ## analysis and discussion in this section, we construct supplementary experiments to analyze our dsrm framework further. ## conclusion in this paper, we delve into the training objective of adversarial training and verify that the robust optimization loss can be estimated by shifting the distribution of training samples. based on this discovery, we propose dsrm as an effective and more computationally friendly algorithm to overcome the trade-off between efficiency and effectiveness in adversarial training. dsrm optimizes the up-per bound of adversarial loss by perturbing the distribution of training samples, thus circumventing the complex gradient ascent process. dsrm achieves state-of-the-art performances on various nlp tasks against different textual adversarial attacks. this implies that adversarial samples, either generated by gradient ascent or data augmentation, are not necessary for improvement in adversarial robustness. we call for further exploration and understanding of the association between sample distribution shift and adversarial robustness. this section discusses the potential limitations of our work. this paper's analysis of model effects mainly focuses on common benchmarks for adversarial defence, which may introduce confounding factors that affect the stability of our framework. therefore, our model's performance on more tasks, e.g., the mrpc dataset for semantic matching tasks, is worth further exploring. in addition, the present work proposes to conduct adversarial training from the perspective of estimating the overall adversarial loss. we expect a more profound exploration of improving the accuracy and efficiency of such estimation. we are also aware of the necessity to study whether the properties of traditional methods, such as the robust overfitting problem, will also arise in dsrm-based adversarial training. we leave these problems to further work.