Virtual Data Augmentation: A Robust and General Framework for Fine-tuning Pre-trained Models (2021)

Abstract: Recent works have shown that powerful pre-trained language models (PLM) can be fooled by small perturbations or intentional attacks. To solve this issue, various data augmentation techniques are proposed to improve the robustness of PLMs. However, it is still challenging to augment semantically relevant examples with sufficient diversity. In this work, we present Virtual Data Augmentation (VDA), a general framework for robustly fine-tuning PLMs. Based on the original token embeddings, we construct a multinomial mixture for augmenting virtual data embeddings, where a masked language model guarantees the semantic relevance and the Gaussian noise provides the augmentation diversity. Furthermore, a regularized training strategy is proposed to balance the two aspects. Extensive experiments on six datasets show that our approach is able to improve the robustness of PLMs and alleviate the performance degradation under adversarial attacks. Our codes and data are publicly available at blue https://github.com/RUCAIBox/VDA .

## introduction recently, pre-trained language models (plms) such as bert @xcite and roberta @xcite have achieved remarkable success in various natural language processing (nlp) tasks @xcite @xcite . as a general and effective approach, fine-tuning plms on specific datasets has become the mainstream paradigm for developing nlp applications. despite the success, researchers have found that plms can be easily fooled by adversarial attacks @xcite . although encapsulated into a black box, these attack strategies can detect the vulnerabilities of a plm via intentional queries @xcite † † corresponding author 2021; @xcite , and then add small perturbations (e.g., synonyms substitution) into the input texts for misleading plms to incorrect predictions. as found in previous works @xcite @xcite , a possible reason of the vulnerability is that these plms do not generalize well on semantic neighborhood around each example in the representation space. to solve this issue, adversarial data augmentation (ada) methods @xcite @xcite have been proposed by revising original data to augment attackrelated data for training. however, due to the discrete nature of language, it is challenging to generate semantically relevant and sufficiently diverse augmentations. although attempts by leveraging expert knowledge @xcite and victim models @xcite have achieved better performance, their generalizability and flexibility is highly limited. recently, virtual adversarial training @xcite is applied to various nlp models for improving the performance and robustness @xcite , which usually generates gradient-based perturbation on the embedding space as virtual adversarial samples. however, it is hard to explicitly constrain the gradient-based perturbation within the same semantic space as the original sample. in addition, unlike attacks in computer vision @xcite , textual adversarial attacks are discrete (e.g., word replacement) and are hard to be captured by gradient-based perturbations. to solve these challenges, we propose virtual data augmentation (vda), a robust and general framework for fine-tuning pre-trained models. our idea is to generate data augmentations at the embedding layer of plms. to guarantee semantic relevance, we consider a multinomial mixture of the original token embeddings as the augmented embedding for each position of the input. in the she is a good student ## related work we review the related work in the following three aspects. ## preliminary this work seeks to improve the fine-tuning performance of pre-trained language models (plm), in that the fine-tuned model will become more robust to data permutations or attacks. specially, we take the text classification task as an example task to illustrate our approach, where a set of n labeled texts { x i , y i } are available. each labeled text consists of a text x i and a label y i from the label set y. we refer to the adversarial example generated from a text x i as adversarial text, denoted by xi . the purpose of adversarial examples is to enhance the model robustness in resisting intentional data perturbations or attacks. let f denote a plm parameterized by θ. following @xcite , we incorporate adversarial examples to improve the fine-tuning of plms. to conduct the adversarial learning, we formulate the learning objective as follows: for the plm f , we assume that it is already pretrained on general-purpose large-scale text data, we would like to fine-tune its parameter θ based on some downstream task. the plms are usually developed based on multi-layered transformer architecture such as bert @xcite and roberta @xcite , where a sequence of tokens will be encoded into a sequence of contextual representations. here, we take the representation of the first token (i.e., @xcite ) as the input of the classifier, and optimize the classification performance with the cross-entropy loss. ## our approach in this section, we describe our proposed framework virtual data augmentation (vda) for robustly fine-tuning plms. our framework consists of two important ingredients, namely embedding augmentation and regularized training. ## experiment -main results we demonstrate the effectiveness of vda for finetuning plms in the text classification task. ## experiment -analysis and extension in this section, we continue to study and analyze the effectiveness of our proposed vda. ## conclusion in this work, we proposed the framework virtual data augmentation (vda), for robustly fine-tuning pre-trained language models. it is a general framework agnostic to downstream tasks, model architectures and learning strategies. in vda, we augmented new embeddings by making weighted aggregation on token embedding matrix according to a multinomial mixture distribution. to construct the mixture distribution, we utilized a masked language model to generate the substitution probability for guaranteeing semantic consistency, and a gaussian noise to provide diversity. and we also adopted a regularized training strategy to further enhance the robustness. extensive experiments on six datasets have demonstrated that the proposed approach can effectively improve the robustness of various plms.