Adversarial Training for Machine Reading Comprehension with Virtual Embeddings (2021)

Abstract: Adversarial training (AT) as a regularization method has proved its effectiveness on various tasks. Though there are successful applications of AT on some NLP tasks, the distinguishing characteristics of NLP tasks have not been exploited. In this paper, we aim to apply AT on machine reading comprehension (MRC) tasks. Furthermore, we adapt AT for MRC tasks by proposing a novel adversarial training method called PQAT that perturbs the embedding matrix instead of word vectors. To differentiate the roles of passages and questions, PQAT uses additional virtual P/Q-embedding matrices to gather the global perturbations of words from passages and questions separately. We test the method on a wide range of MRC tasks, including span-based extractive RC and multiple-choice RC. The results show that adversarial training is effective universally, and PQAT further improves the performance.

## introduction neural networks have achieved superior performance on many tasks, but they are vulnerable to adversarial examples @xcite ) -examples that have been mixed with certain perturbations. adversarial training (at) @xcite uses both clean and adversarial examples to improve the robustness of the model for image classification. in the field of nlp, @xcite have applied adversarial training on text classification tasks and improved the model performance. from then on, many at methods has been proposed @xcite @xcite @xcite @xcite . they mostly adopt a general at strategy, but focus less on the adaptation of at to nlp tasks. to explore this adaptation, in this work, we aim to apply adversarial training passage: ... the rock cycle is an important concept in geology which illustrates the relationships between these three types of rock, and magma. when a rock crystallizes from melt (magma and/or lava), it is an igneous rock. ... question: an igneous rock is a rock that crystallizes from what? the objective of mrc is to let a machine read the given passages and ask it to answer the related questions. there are several types of mrc tasks. in this work we focus on span-based extractive rc @xcite @xcite and multiple-choice rc @xcite . to apply adversarial training on mrc tasks, we notice that there are several salient characteristics of mrc compared to other tasks such as image classification: (1) the inputs are discrete. unlike pixels, which can take continuous values, words are discrete tokens. (2) the tokens in the input sequences are not independent. a word may occur in an input sequence several times. after the embedding layer, these occurrences are represented by the word vectors with the same value and hold the same semantic meaning (although the word may be polysemous). (3) the roles of passages and questions are different. given a question as the query, the model needs to look up the correct answer in the passage. people have utilized the first characteristic to apply adversarial training by perturbing input word vectors instead of tokens. however, the second and third characteristics have been largely ignored. for example, in to take the second and the third characteristics into consideration, we propose a novel adversarial training method called pqat. the core of pqat is the virtual p/q-embeddings, which are two independent embedding spaces for passages and questions. each time we calculate perturbations, p/qembeddings gather the perturbations from passages and questions for each word, then generate a global and role-aware perturbation for each word from passages and questions separately. for example, in we have applied adversarial training on several mrc tasks, including span-based extractive rc and multiple-choice rc. results show that adversarial training improves the mrc model performance universally and consistently, even over the strong pre-trained model baseline. furthermore, the pqat method outperforms the standard at on both normal datasets and adversarial datasets. lastly, our results verify the usefulness of incorporating information of task form into the design of the adversarial training method. ## standard adversarial training adversarial training first constructs adversarial examples by generating worst-case perturbations that maximize the current loss, then minimize the loss on those adversarial examples. in nlp tasks, a popular approach to generate perturbations is to perturb word vectors from the embedding layer @xcite . we denote the input token sequence as x and the operation of looking up in an embedding layer e as emb(e, •). the objective rock igneous rock igneous p: when a rock crystallizes from melt, it is an igneous rock. q: an igneous rock is a rock that crystallizes from what? p-embeddings q-embeddings of at is ## adversarial training for mrc in the above algorithm, when generating the perturbations on x vec through backward propagation, each word vector x i vec is perturbed independently, like the pixels in an image. it ignores the semantic relationship among the word vectors of a word's different occurrences. to make the perturbation on each occurrence aware of other occurrences of the same word, we adapt at by gathering not only the perturbations on each word vector, but also the perturbations on the embedding matrix. the latter can be seen as the global perturbations, which provide context-insensitive semantic information. the global perturbations are rather coarsegrained, since all the occurrences of the same word receive the same global perturbation. note that in mrc tasks, words in passages and questions play different roles. thus, to keep this information, we distinguish the words in passages and questions by creating two virtual embedding matrices p and q: p-embedding matrix p collects the perturbations of all the words from the passages; q-embedding matrix q for the questions. we give an illustration in figure 1 . p/q-embedding matrices are virtual since they only provide perturbations, no the real word vectors. during training, perturbations from virtual embeddings and word vectors are summed up to form the adversarial input z vec . the final objective is we list the overall algorithm of adversarial training for mrc in algorithm 1. we initialize p and q with the gaussian distribution. for each batch, we perform k-step gradient descent (line 9-22): we look up the original word vectors and p/q-embedding vectors from the embedding layer e and the p/q-embedding matrices. the adversarial inputs are constructed by summing them with local perturbations δ. then we compute the gradients of model parameters g t , local perturbations g δ and p/q-embedding matrices g p and g q . these gradients can be calculated in a single backward pass. lastly, we update the virtual embeddings and local perturbations (line 18-21). note that p/q-embedding matrices serve as the containers for perturbations. when the training is finished, p/q-embedding matrices are no longer needed and can be discarded. ## conclusion we have applied adversarial training on a wide range of mrc tasks, including span-based extractive rc and multiple-choice rc. especially, we have proposed a novel adversarial training method pqat, which uses virtual p/q-embedding matrices to generate global and role-aware perturbations that consider the characteristics of mrc tasks. our experiments demonstrate that adversarial training improves the mrc model performance universally and consistently, even over the strong pre-trained model baseline. the pqat method further improves the model performance over the standard at on both normal datasets and adversarial datasets.