Learning to Perturb Word Embeddings for Out-of-distribution QA (2021)

Abstract: QA models based on pretrained language models have achieved remarkable performance on various benchmark datasets. However, QA models do not generalize well to unseen data that falls outside the training distribution, due to distributional shifts. Data augmentation (DA) techniques which drop/replace words have shown to be effective in regularizing the model from overfitting to the training data. Yet, they may adversely affect the QA tasks since they incur semantic changes that may lead to wrong answers for the QA task. To tackle this problem, we propose a simple yet effective DA method based on a stochastic noise generator, which learns to perturb the word embedding of the input questions and context without changing their semantics. We validate the performance of the QA models trained with our word embedding perturbation on a single source dataset, on five different target domains. The results show that our method significantly outperforms the baseline DA methods. Notably, the model trained with ours outperforms the model trained with more than 240K artificially generated QA pairs.

## introduction deep learning models have achieved impressive performances on a variety of real-world natural language understanding tasks such as text classification, machine translation, question answering, and text generation to name a few @xcite . recently, language models that are pretrained with a large amount of unlabeled data have achieved breakthrough in the performance on these downstream tasks @xcite , even surpassing human performance on some of them. the success of such data-driven language model pretraining heavily depends on the amount and diversity of training data available, since when * * equal contribution trained with a small amount of highly-biased data, the pretrained models can overfit and may not generalize well to out-of-distribution data. data augmentation (da) techniques @xcite @xcite can prevent this to a certain extent, but most of them are developed for image domains and are not directly applicable to augmenting words and texts. perhaps the most important desiderata for an augmentation method in supervised learning, is that it should not change the label of an example. for image domains, there exist several well-defined data augmentation techniques that can produce diverse augmented images without changing the semantics. in contrast, for natural language processing (nlp), it is not straightforward to augment the input texts without changing their semantics. a simple augmentation technique that preserves semantics is replacing words with synonyms or using back translation @xcite . however, they do not effectively improve the generalization performance because the diversity of viable transformations with such techniques is highly limited @xcite . some recent works @xcite propose data augmentation methods tailored for nlp tasks based on dropping or replacing words and show that such augmentation techniques improve the performance on the out-ofdomain as well as the in-domain tasks. as shown in fig. 1 , however, we have observed that most existing data augmentation methods for nlp change the semantics of original inputs. while such change in the semantics may not be a serious problem for certain tasks, it could be critical for question answering (qa) task since its sensitivity to the semantic of inputs. for instance, replacing a single word with a synonym (@xmath0. 1 ) might cause the drastic semantic drift of the answer @xcite . thus, word-based to address such limitations of the existing data augmentation techniques for qa, we propose a novel da method based on learnable word-level perturbation, which effectively regularizes the model to improve its generalization to unseen questions and contexts with distributional shifts. specifically, we train a stochastic perturbation function to learn how to perturb each word embedding of the input without changing its semantic, and augment the training data with the perturbed samples. we refer to this data augmentation method as stochastic word embedding perturbation (swep). the objective of the noise generator is to maximize the log-likelihood of the answer of the input with perturbation, while minimizing the kullback-leibler (kl) divergence between prior noise distribution and conditional noise distribution of given input. since the perturbation function maximizes the likelihood of the answer of the perturbed input, it learns how to add noise without changing the semantics of the original input. furthermore, minimizing the kl divergence prevents generating identical noise as the variance of the prior distribution is non-zero, i.e. we can sample diverse noise for the same input. we empirically validate our data augmentation method on both extractive and generative qa tasks. we train the qa model on the squad dataset @xcite with our learned perturbations, and evaluate the trained model on the five different domains -bioasq @xcite , new york times, reddit post, amazon review, and wikipedia @xcite as well as squad to measure the generalization performance on out-of-domain and in-domain data. the experimental results show that our method improves the in-domain performance as well as out-of-domain robustness of the model with this simple yet effective approach, while existing baseline methods often degrade the performance of the qa model, due to semantics changes in the words. notably, our model trained only with the squad dataset shows even better performance than the model trained with 240,422 synthetic qa pairs generated from a question generation model. our contribution in this work is threefold. ## conclusion we proposed a simple yet effective data augmentation method based on a stochastic word embedding perturbation for out-of-distribution qa tasks. specifically, our stochastic noise generator learns to generate the adaptive noise depending on the contextualized embedding of each word. it maximizes the likelihood of input with perturbation, such that it learns to modulate the intensity of perturbation for each word embedding without changing the semantic of the given question and paragraph. we augmented the training data with the perturbed samples using our method, and trained the model with only a single source dataset and evaluate it on datasets from five different domains as well as the in-domain dataset. based on the experimental results, we verified that our method improves both the performance of in-domain generalization and robustness to distributional shifts, outperforming the baseline data augmentation methods. further quantitative and qualitative analysis suggest that our method learns to generate adaptive perturbation without a semantic drift.