A Self-Training Method for Machine Reading Comprehension with Soft Evidence Extraction (2020)

Abstract: Neural models have achieved great success on machine reading comprehension (MRC), many of which typically consist of two components: an evidence extractor and an answer predictor. The former seeks the most relevant information from a reference text, while the latter is to locate or generate answers from the extracted evidence. Despite the importance of evidence labels for training the evidence extractor, they are not cheaply accessible, particularly in many non-extractive MRC tasks such as YES/NO question answering and multi-choice MRC. To address this problem, we present a Self-Training method (STM), which supervises the evidence extractor with auto-generated evidence labels in an iterative process. At each iteration, a base MRC model is trained with golden answers and noisy evidence labels. The trained model will predict pseudo evidence labels as extra supervision in the next iteration. We evaluate STM on seven datasets over three MRC tasks. Experimental results demonstrate the improvement on existing MRC models, and we also analyze how and why such a self-training method works in MRC.

## introduction machine reading comprehension (mrc) has received increasing attention recently, which can be roughly divided into two categories: extractive and non-extractive mrc. extractive mrc requires a model to extract an answer span to a question from reference documents, such as the tasks in squad @xcite and coqa @xcite . in contrast, non-extractive mrc infers answers based on some evidence in reference documents, including yes/no question answering @xcite , multiple-choice mrc @xcite @xcite , and open domain question answering @xcite . as shown in manually annotating the golden evidence is expensive. therefore, some recent efforts have been dedicated to improving mrc by leveraging noisy evidence labels when training the evidence extractor. some works @xcite generate distant labels using hand-crafted rules and external resources. some studies @xcite adopt reinforcement learning (rl) to decide the labels of evidence. however, such rl methods suffer from unstable training. more distant supervision techniques are also used to refine noisy labels, such as deep probability logic @xcite , but they are hard to transfer to other tasks. nevertheless, improving the evidence extractor remains challenging when golden evidence labels are not available. in this paper, we present a general and effective method based on self-training @xcite to improve mrc with soft evidence extraction when golden evidence labels are not available. following the self-training paradigm, a base mrc model is iteratively trained. at each iteration, the base model is trained with golden answers, as well as noisy evidence labels obtained at the preceding it-q: did a little boy write the note? d: ...this note is from a little girl. she wants to be your friend. if you want to be her friend, ... a: no q: is she carrying something? d: ...on the step, i find the elderly chinese lady, small and slight, holding the hand of a little boy. in her other hand, she holds a paper carrier bag. ... a: yes ## related work early mrc studies focus on modeling semantic matching between a question and a reference document @xcite @xcite . in order to mimic the reading mode of human, hierarchical coarse-to-fine methods are proposed @xcite . such models first read the full text to select relevant text spans, and then infer answers from these relevant spans. extracting such spans in mrc is drawing more and more attention, though still quite challenging @xcite . evidence extraction aims at finding evidential and relevant information for downstream processes in a task, which arguably improves the overall performance of the task. not surprisingly, evidence extraction is useful and becomes an important component in fact verification @xcite @xcite , multiple-choice reading comprehension @xcite @xcite , open-domain question answering @xcite , multi-hop reading comprehension @xcite , natural language inference @xcite , and a wide range of other tasks @xcite . in general, evidence extraction in mrc can be classified into four types according to the training method. first, unsupervised methods provide no guidance for evidence extraction @xcite . second, supervised methods train evidence extraction with golden evidence labels, which sometimes can be generated automatically in extractive mrc settings @xcite @xcite . third, weakly supervised methods rely on noisy evidence labels, where the labels can be obtained by heuristic rules @xcite . moreover, some data programming techniques, such as deep probability logic, were proposed to refine noisy labels @xcite . last, if a weak extractor is obtained via unsupervised or weakly supervised pre-training, reinforcement learning can be utilized to learn a better policy of evidence extraction @xcite . for non-extractive mrc tasks, such as ynqa and mmrc, it is cumbersome and inefficient to annotate evidence labels @xcite . although various methods for evidence extraction have been proposed, training an effective extractor is still a challenging problem when golden evidence labels are unavailable. weakly supervised methods either suffer from the low performance or rely on too many external resources, which makes them diffi-cult to transfer to other tasks. rl methods can indeed train a better extractor without evidence labels. however, they are much more complicated and unstable to train, and highly dependent on model pre-training. our method is based on self-training, a widely used semi-supervised method. most related studies follow the framework of traditional self-training @xcite and co-training @xcite , and focus on designing better policies for selecting confident samples. co-trade @xcite evaluates the confidence of whether a sample has been correctly labeled via a statistic-based data editing technique @xcite . self-paced co-training @xcite adjusts labeled data dynamically according to the consistency between the two models trained on different views. a reinforcement learning method @xcite designs an additional q-agent as a sample selector. ## conclusion and future work we present an iterative self-training method (stm) to improve mrc models with soft evidence extraction, when golden evidence labels are unavailable. in this iterative method, we train the base model with golden answers and pseudo evidence labels. the updated model then generates new pseudo evidence labels, which can be used as additional supervision in the next iteration. experiment results show that our proposed method consistently improves the base models in seven datasets for three mrc tasks, and that better evidence extraction indeed enhances the final performance of mrc. as future work, we plan to extend our method to other nlp tasks which rely on evidence finding, such as natural language inference.