Robust Machine Reading Comprehension by Learning Soft labels (2020)

Abstract: Neural models have achieved great success on the task of machine reading comprehension (MRC), which are typically trained on hard labels. We argue that hard labels limit the model capability on generalization due to the label sparseness problem. In this paper, we propose a robust training method for MRC models to address this problem. Our method consists of three strategies, 1) label smoothing, 2) word overlapping, 3) distribution prediction. All of them help to train models on soft labels. We validate our approach on the representative architecture - ALBERT. Experimental results show that our method can greatly boost the baseline with 1% improvement in average, and achieve state-of-the-art performance on NewsQA and QUOREF.

## introduction extractive reading comprehension is a challenging task in the field of natural language processing. its objective is, as shown in fig. 1 , to detect a fragment from a passage to answer a given question. our model needs to understand the context, locate the correct answer with exact word boundaries. since multiple questions can be derived for a passage with different fragments of the words as the corresponding answers, the task is deemed as a benchmark for the deep understanding of human language, becoming a hot research topic in recent years. usually, there exist multiple answers (or answer instances) for a given question in the extractive reading comprehension scenario. these answers generally express the same meaning with different combination of words, or same expression but appear repeatedly in different locations, e.g. the multiple correct answers shown in fig. 1 . however, the application of the standard cross-entropy loss in training allows only one correct answer to be considered, with other candidate answers ignored. paragraph: one of the first norman mercenaries to serve as a byzantine general was hervé in the 1050s. by then however, there were already norman mercenaries serving as far away as trebizond and georgia. . . . question: when did hervé serve as a byzantine general? answer1: 1050s answer2: in the 1050s licence details: http://creativecommons.org/licenses/by/4.0 a variety of methods have been proposed to address this issue @xcite @xcite . most of them are focused on the training algorithm of mrc model, using reinforcement learning or modifying loss function to generalize the word overlapping between a candidate answer and the correct answer. using this strategy, the training of model can be very complex, with word overlapping in only one training sample into account, missing other q&a patterns in the training set. in order to better resolve this problem, we choose a focus on the training data perspective and propose soft label based data augmentation without modifying the model structure and training algorithm. as a model-independent data augmentation method, our methods permits a flexibility of using different methods to construct soft label, and to design the framework of the model. altogether we test 3 different methods to generate soft labels, including word overlapping, on the state-of-the-art albert model @xcite . all the methods can boost albert with notable improvement. in the best case, albert can be improved with about 2% by soft label based augmentation, proving our approach simple yet effective. ## soft label based data augmentation in this section, we will introduce our soft label based data augmentation methods. we investigate three implementations of soft label: label smoothing, word overlapping, and distribution prediction. a brief illustration and comparison of the three methods is shown in fig. 2 . label smoothing label smoothing was first proposed in the field of computer vision @xcite . for a training sample (x, y), the probability of the correct category q(y|x) is defined as 1 and other categories q(¬y|x) is as 0, and thereby there is a golden distribution q. the loss function of the training sample is usually defined as the cross-entropy loss shown in the eq.( 1 ). word overlapping although label smoothing can weaken the golden answer, it fails to strengthen other possible correct answers. therefore, most of the existing researches soften the labels based on word overlapping. the word overlapping is measured by token-level f1 score, and normalized with softmax function upon all possible answer spans as shown in eq.( 3 ). then the start/end label distribution is calculated by marginalizing it with respect to all possible end/start positions: distribution prediction word overlapping is more enlightening, but this information is still obtained from one training sample. moreover, it may generate incorrect label distribution. in the previous example, [the 1050s] and [1050s .] have higher f1 score compared with [in the 1050s], but obviously [in the 1050s] is more likely to be correct. therefore, we build another kind of soft label, the target distribution q is predicted by another model following the idea of knowledge distillation @xcite , while some details are implemented differently. instead of controlling the temperature, we propose cross-decoding similar to cross-validation. first, randomly divide the data set t into many smaller sets {t 1 , t 2 , . . . , t n }, select {t 2 , t 3 , . . . , t n } to train a model f 1 , then predict the probability of start/end position p s , p e on the reserved sample @xmath0. second, using the predicted p as new target distribution q , we construct a soften example (x, y ) on t 1 . at the next iterations, we select different parts to decode, until all samples in t have predicted label, i.e. @xmath1, y )|@xmath2. since f is trained on a large-scale data set {t 2 , t 3 , . . . , t n }, the predicted distribution q can be considered as composed all the q&a patterns in {t 2 , t 3 , . . . , t n }. thus, the potentially correct candidate answers (the q&a patterns that appeared in the training set, to be more precise) will have higher probability, which will become a more informative guidance for the mrc model. ## conclusion in this paper, we propose a simple yet effective data augmentation strategy based on soft label to capture the multiple correct answers in extractive reading comprehension task. we investigate 3 methods to generate soft label, i.e. label smoothing, word overlapping and distribution prediction, and validate them by albert on squad 2.0, newsqa and quoref datasets. the experimental results indicate that all strategies are positive, with the predicted distribution top-performed with state-of-the-art results on newsqa and quoref and outperforms human on squad 2.0 and newsqa. finally, we would suggest that phenomenon of multiple answers in the mrc dataset is indeed widespread, and more effective approaches are desired to address this issue.