Learn with Noisy Data via Unsupervised Loss Correction for Weakly Supervised Reading Comprehension (2020)

Abstract: Weakly supervised machine reading comprehension (MRC) task is practical and promising for its easily available and massive training data, but inevitablely introduces noise. Existing related methods usually incorporate extra submodels to help filter noise before the noisy data is input to main models. However, these multistage methods often make training difficult, and the qualities of submodels are hard to be controlled. In this paper, we first explore and analyze the essential characteristics of noise from the perspective of loss distribution, and find that in the early stage of training, noisy samples usually lead to significantly larger loss values than clean ones. Based on the observation, we propose a hierarchical loss correction strategy to avoid fitting noise and enhance clean supervision signals, including using an unsupervisedly fitted Gaussian mixture model to calculate the weight factors for all losses to correct the loss distribution, and employ a hard bootstrapping loss to modify loss function. Experimental results on different weakly supervised MRC datasets show that the proposed methods can help improve models significantly.

## introduction machine reading comprehension (mrc) @xcite is a well-known nlp task, and has made significant progress in recent years @xcite @xcite . to learn a well-performed mrc system, large amount of human annotated data is required. however, human annotation is high-cost in real-world application, and it is hard to control the quality for some of hard instances. recent approach @xcite utilized a distantly supervised method to collect the excerpts for answers. it greatly scales up the dataset and reduces the cost, but introduces more harmful noisy samples inevitably. there are many of approaches proposed to filter noise for question answering (qa) recently. @xcite and @xcite adopted a paragraph selector to calculate confidences of paragraphs to help filter noisy ones before they are input into the main model. @xcite designed a submodel to generate labels to supervise the training of the selector. back to mrc, @xcite further proposed to generate labels for unlabeled samples, then train an extra refinery model to refine the overall labels for multilingual mrc task with limited training data. admittedly, these multistage methods have achieved certain improvements, but rely heavily on the selector, retriever or refinery. the qualities of these complementary models are hard to be controlled, and make training difficult. in fact, we can explore another novel idea that exploits the essential characteristics of noise itself to help alleviate its effect for mrc task. inspired by the idea of learning with noisy labels in image classification @xcite , we explore and find that the loss distribution of weakly supervised mrc training data has inspiring characteristics. as shown in figure 1 (a), at the beginning of training, losses of noisy samples are generally greater than losses of clean samples significantly. and in figure 1 (b), during training, the losses of all samples roughly converge into two clusters according to values. in addition, we have noticed that without correction, noise tends to attract more attention due to the produced larger loss, which causes the model optimized into wrong direction easily. we argue that it is one of the essential reasons why the performance will be hurt by much noise. in this paper, our main idea for improving original models is to correct the loss distribution based on the above findings, which reduces losses of noisy samples thus avoiding fitting noise and pushes models to pay more attention to the supervision signals from clean data, as shown in figure 2 . specifically, instead of modifying the original structures of previous well-performing models, we first choose to fit a 2-component gaussian mixture model (gmm) to loss distribution unsupervisedly, then infer the probabilities of samples being clean or noisy through the posterior probability provided by gmm. note that we have verified in section 5.2 that the noise recognition accuracy can even exceed 90% by gmm. based on the inferred results of gmm, we then automatically produce weight factors for all losses. specifically, we assign larger factors to losses that have higher probabilities of being clean samples by gmm, while assign lowers ones to losses that are more likely to be noisy. note that in the traditional case without correction, the weight factors of all losses can be regarded as an uniform distribution. in addition, we also propose to use the hard bootstrapping loss to replace standard cross-entropy loss to further correct loss values of the individual samples to further avoid fitting noise. our contributions are summarized as: (1) we explore the essential characteristics of noise in weakly supervised mrc from the perspective of loss distribution, and offer new ideas for this task and other related nlp tasks in weakly supervised manner; (2) we propose the hierarchical loss correction method to avoid fitting noise and strengthen the supervision from clean samples, which uses unsupervisedly fitted gmm to calculate weight factors for correcting loss distribution, and uses hard bootstrapping loss to modify loss function; (3) we conduct ample experiments on two types of multiple weakly supervised datasets, and experimental results show that the proposed method can improve models significantly. ## methodology we propose hierarchical loss correction strategy to avoid fitting noise and enhance supervision signals from clean samples. the overall framework of the proposed methods is shown in figure 2 . we first model loss by fitting a gmm, then perform loss correction operation before back propagation. ## conclusion in this paper, we explore natural characteristics of noise from perspective of loss, and find in early stages of training, noisy samples usually result in significantly larger losses than clean samples. based on the observation, we propose a hierarchical loss correction strategy to avoid fitting noise and strengthen supervision signals from clean samples by incorporating an unsupervisedly fitted gmm and modifying original loss funciton to hard bootstrapping loss. we conducted ample experiments on multiple weakly supervised mrc datasets. experimental results show that the proposed methods can effectively help models to achieve significant improvements.