Self-Supervised Contrastive Learning with Adversarial Perturbations for Defending Word Substitution-based Attacks (2022)

Abstract: In this paper, we present an approach to improve the robustness of BERT language models against word substitution-based adversarial attacks by leveraging adversarial perturbations for self-supervised contrastive learning. We create a word-level adversarial attack generating hard positives on-the-fly as adversarial examples during contrastive learning. In contrast to previous works, our method improves model robustness without using any labeled data. Experimental results show that our method improves robustness of BERT against four different word substitution-based adversarial attacks, and combining our method with adversarial training gives higher robustness than adversarial training alone. As our method improves the robustness of BERT purely with unlabeled data, it opens up the possibility of using large text datasets to train robust language models against word substitution-based adversarial attacks.

## introduction pretrained language models such as bert @xcite have had a tremendous impact on many nlp tasks. however, several researchers have demonstrated that these models are vulnerable to adversarial attacks, which fool the model by adding small perturbations to the model input @xcite . a prevailing method to improve model robustness against adversarial attacks is adversarial training @xcite . in nlp, adversarial training in the input space has been challenging, as existing natural language adversarial attacks are too slow to generate adversarial examples on the fly during training @xcite @xcite . while some recent works @xcite have started exploring efficient input space adversarial training (e.g., for text classification), scaling adversarial training to * the first two authors contributed equally to this work. in this work, we in particular focus on improving the robustness of bert against word substitutionbased adversarial attacks. we propose an approach to adversarially finetune bert-like models without using any labeled data. in order to achieve this, we rely on self-supervised contrastive learning @xcite . self-supervised contrastive learning has recently gained attention in the community and contrastive learning has been used to learn better representations for text classification @xcite @xcite . however, how to use these methods to improve model robustness remains an open question. we combine self-supervised contrastive learning with adversarial perturbations by using adversarial attacks to generate hard positive examples for contrastive learning. to efficiently create adversarial examples, we leverage an adversarial attack, that is capable of generating multiple adversarial examples in parallel. the attack adversarially creates hard positive examples for contrastive learning by iteratively replacing words to follow the direction of the contrastive loss (see fig. 2 ). experiments show that our method can improve the robustness of pretrained language models without looking at the labels (in other words, before finetuning). additionally, by combining our method with adversarial training, we are able to obtain better robustness than conducting adversarial training alone (see section 4.4). our study of the vector representations of clean examples and their corresponding adversarial examples indeed explains that our method improves model robustness by pulling clean examples and adversarial examples closer. our contributions 1 in this paper are two-fold. on the one hand, we improve the robustness of the pretrained language model bert against word substitution-based adversarial attacks by using selfsupervised contrastive learning with adversarial perturbations (see section 3.2). on the other hand, to facilitate adversarial self-supervised contrastive learning, we create for bert a word-level adversarial attack to create hard positive examples. the attack makes contrastive learning and adversarial training with on-the-fly generated adversarial examples possible. additionally, we also show that our method is capable of using out-of-domain data to improve model robustness (see ## methodology in this section, we describe our method for selfsupervised contrastive learning with adversarial perturbations. specifically, section 3.1 gives the background and motivation of our problem, and section 3.2 describes the adversarial contrastive learning framework. finally, in section 3.3, we bert 𝑋 attack for contrastive loss 𝑋 ! 𝑋 "#$ 𝒛 ! 𝒛 𝒛 𝒂𝒅𝒗 pu ll p u s h pu sh contrastive learning adversarial attack (a) for the original example x, we obtain the hard positive example x ′ by geometry attack for contrastive loss (see section 3.3). (b) before contrastive learning, in the vector space, the clean example z, the hard positive example z ′ , and the adversarial example z adv are far from each other. contrastive learning pulls the clean example z, and the hard positive example z ′ together. (c) after contrastive learning, the clean example z, the hard positive example z ′ , and the adversarial example z adv are close. we omit mlp in this figure for simplicity. we use a different color to show another example from the dataset. see section 3 for details. note that the adversarial example x adv and its corresponding vector z adv are not used in contrastive learning. we nevertheless show x adv and z adv for illustration purposes. ## conclusion and future work in this paper, we improve the robustness of pretrained language models against word substitutionbased adversarial attacks by using self-supervised contrastive learning with adversarial perturbations. our method is different from previous works as we can improve model robustness without accessing annotated labels. furthermore, we also conduct word-level adversarial training on bert with onthe-fly generated adversarial examples. our adversarial training is different from previous works in that (1) it is on the word level; (2) we generate adversarial examples on the fly, instead of generating a fixed adversarial set beforehand. experiments show that our method improves model robustness. we find that combining our method with adversarial training results in better robustness than conducting adversarial training alone. in the future, we plan to scale our method to even larger out-ofdomain datasets.