Advances in extractive machine reading comprehension (MRC) rely heavily on the collection of large scale human-annotated training data ( in the form of "question-paragraph-answer span") . A single question-answer example provides limited supervision, while an explanation in natural language describing human's deduction process may generalize to many other questions that share similar solution patterns. In this paper, we focus on "teaching" machines on reading comprehensionwith ( a small number of ) natural language explanations . We propose a data augmentation framework that exploits the compositional nature of explanations to rapidly create pseudo-labeled data for training downstream MRC models. Structured variables and rules are extracted from each explanation and formulated into neural module teacher, which employs softened neural modules and combinatorial search to handle linguistic variations and overcome sparse coverage . The proposed work is particularly effective when limited annotation effort is available, and achieved a practicable F1 score of 59.80\% with supervision from 52 explanationson the SQuAD dataset .
Advances in machine reading comprehension (MRC) rely heavily on the collection of large scale human-annotated examples in the form of (question, paragraph, answer) triples. In contrast, humans are typically able to generalize with only a few examples, relying on deeper underlying world knowledge, linguistic sophistication, and/or simply superior deductive powers . In this paper, we focus on "teaching" machines reading comprehension, using a small number of semi-structured explanations that explicitly inform machines why answer spans are correct. We extract structured variables and rules from explanations and compose neural module teachers that annotate instances for training downstream MRC models. We use learnable neural modules and soft logic to handle linguistic variation and overcome sparse coverage ; the modules are jointly optimized with the MRC model to improve final performance. On the SQuAD dataset, our proposed method achieves 70.14\% F1 score with supervision from 26 explanations, comparable to plain supervised learning using 1,100 labeled instances, yielding a 12x speed up .
D
extractive
R
training data (
examples
R
"question-paragraph-answer span")
(question, paragraph, answer) triples. In contrast, humans are typically able to generalize with only a few examples, relying on deeper underlying world knowledge, linguistic sophistication, and/or simply superior deductive powers
D
A single question-answer example provides limited supervision, while an explanation in natural language describing human's deduction process may generalize to many other questions that share similar solution patterns.
R
on reading comprehensionwith (
reading comprehension, using
R
) natural language explanations . We propose a data augmentation framework that exploits the compositional nature of explanations to rapidly create pseudo-labeled data
semi-structured explanations that explicitly inform machines why answer spans are correct. We extract structured variables and rules from explanations and compose neural module teachers that annotate instances
R
Structured variables and rules are extracted from each explanation and formulated into neural module teacher, which employs softened neural modules and combinatorial search
We use learnable neural modules and soft logic
R
variations
variation
R
. The proposed work is particularly effective when limited annotation effort is available, and achieved a practicable
; the modules are jointly optimized with the MRC model to improve final performance. On the SQuAD dataset, our proposed method achieves 70.14\%
D
of 59.80\%
R
52 explanationson the SQuAD dataset
26 explanations, comparable to plain supervised learning using 1,100 labeled instances, yielding a 12x speed up