Training Question Answering Models From Synthetic Data (2020)

Abstract: Question and answer generation is a data augmentation method that aims to improve question answering (QA) models given the limited amount of human labeled data. However, a considerable gap remains between synthetic and human-generated question-answer pairs. This work aims to narrow this gap by taking advantage of large language models and explores several factors such as model size, quality of pretrained models, scale of data synthesized, and algorithmic choices. On the SQuAD1.1 question answering task, we achieve higher accuracy using solely synthetic questions and answers than when using the SQuAD1.1 training set questions alone. Removing access to real Wikipedia data, we synthesize questions and answers from a synthetic text corpus generated by an 8.3 billion parameter GPT-2 model and achieve 88.4 Exact Match (EM) and 93.9 F1 score on the SQuAD1.1 dev set. We further apply our methodology to SQuAD2.0 and show a 2.8 absolute gain on EM score compared to prior work using synthetic data.

## introduction one of the limitations of developing models for question answering, or any deep learning application for that matter, is the availability and cost of labeled training data. a common approach to alleviate this need is semi-supervised learning, wherein one trains a model on existing data and uses it to label more data for training @xcite @xcite . this technique has demonstrated benefits in recent literature for image classification @xcite and question answering (qa) tasks @xcite . however, the complexities of generating questions and answers text albert einstein is known for his theories of special relativity and general relativity. he also made important contributions to statistical mechanics, especially his mathematical treatment of brownian motion, his resolution of the paradox of specific heats, and his connection of fluctuations and dissipation. despite his reservations about its interpretation, einstein also made contributions to quantum mechanics and, indirectly, quantum field theory, primarily through his theoretical studies of the photon. 117m which two concepts made einstein's post on quantum mechanics relevant? 768m albert einstein also made significant contributions to which field of theory? 8.3b because of his work with the photon, what theory did he indirectly contribute to? human what theory did einstein have reservations about? in natural language proves challenging for existing methods, with a large gap in quality remaining between synthetic and human-generated data. in this work, we close this gap using only synthetic questions generated from large models. we also show that naive scaling of @xcite alone is insufficient. we demonstrate that answer candidate generation is foundational to synthetic question quality, and there are issues aligning the answer distribution which do not improve with scale. throughout this paper we focus primarily on question generation as a data augmentation method for existing question answering tasks. focus on this type of semi-supervised setting is a necessary first step to enable future work applying question generation to low-resource tasks. similar to prior work @xcite , we use a 3-step modeling pipeline consisting of unconditional answer extraction from text, question generation, and question filtration. our approach for training question generators on labeled data uses pretrained gpt-2 decoder models and a next-token-prediction language modeling objective, trained using a concatenation of context, answer, and question tokens. we demonstrate that pretraining large generative transformer models up to 8.3b parameters improves the quality of generated questions. additionally, we propose an overgenerate and filter approach to further improve question filtration. the quality of questions produced by this pipeline can be assessed quantitatively by finetuning qa models and evaluating results on the squad dataset. we demonstrate generated questions to be comparable to supervised training with human-labeled data. we measure this by using the ratio of accuracies between a qa model trained on synthetically generated data and a model trained on human labelled data. for answerable squad1.1 questions we recover 100.8% of fully supervised em and 100.1% of fully supervised f1 scores, when training on purely synthetic questions and answers generated from unlabeled data. specifically, we achieve scores of 88.4 and 94.1 compared to supervised training which achieves 87.7 em and 94.0 f1. finetuning the resulting model on real squad1.1 data reaches 89.4 em and 95.1 f1 score, which is higher than any prior bert-based approach. in going further, we show that qa models can be successfully trained from fully synthetic data, by running question and answer generation on a corpus generated from an unconditional gpt-2 model, and achieve an em of 88.4 and f1 of 93.9. this approach performs comparably to generating questions from real data recovering 100.3% of fully supervised em and f1 scores. in summary, our contributions are as follows: ## method in this work we seek to generate high quality training data for squad style extractive question answering over a given set of documents d. this requires us to sample (c, q, a) triples for given paragraph contexts @xmath0, a|c), where q is a question resulting in answer a, which exists as a contiguous span of text in c. leveraging the roundtrip consistency method (alberti et al., 2019a), we achieve this by using a three step approach consisting of answer generation â ∼ p(a|c), question generation q ∼ p(q|â, c), and roundtrip filtration â ? = a * ∼ p(a|c, q). as illustrated by algorithm 1 in the appendix a.1, the synthesized dataset of triples is then used to finetune and train a bert-based qa model similar to @xcite . ## experiment setup for all the implementations and training of transformer models we use the megatron-lm codebase @xcite . for off-the-shelf weights and implementations of bert-large we rely on the huggingface's transformers codebase @xcite . for model configurations, hidden size, number of layers, and attention heads, we used the configurations detailed in megatron-lm. to finetune our gpt-2 models we reused the pretraining hyperparameters detailed in appendix a.3, except for a batch size of 32, and a learning rate of 2e-5 decaying to zero over six epochs of finetuning data. finetuning our bert models for filtration, answer generation, and question answering was all done with a learning rate of 1e-5 and a cosine decay scheduled over 2 epochs of training data. bert pretraining details are described in appendix a.4. we refer to our models as bert-345m (345 million parameters) and bert-1.2b (1.2 billion parameters) and the original bert model as bert-large. to train and evaluate the question generation pipeline for our ablation studies in sections 5 and 6 we used a data partitioning scheme as shown in figure 4 in appendix a.5. a similar data pipeline has been employed in concurrent work of @xcite . we split the squad training data into equal portions, partitioning the data randomly into two sets of documents. one half of the documents is used to train the answer generator, question generator, and filtration models while the second half of the documents is used to generate synthetic data to finetune a qa model. the finetuned qa model is then evaluated on squad dev set, where the evaluation results are used as a surrogate measure of synthetic data quality. the partitioning of the dataset is done to avoid leakage and overfitting between the data seen at training time and generation time thereby testing the generalization capabilities of our models. since shuffling is done randomly we repeat this process 5 times with different seeds for every ablation study and report the mean of our results. due to the large hyperparmeter space we do not perform any learning rate search and use the static learning schedules as described above. we note that the data partitioning is done only for the ablation studies and hyperparameter selection. for the final models in figure 1 and tables 2 & 3 , we use the entire squad dataset. text source finetune # questions em f1 source data size data wikipedia 638 mb synthetic 19,925,130 88.4 94.1 +squad 20,012,729 89.4 95.2 8.3b gpt-2 480 mb synthetic 17,400,016 88.4 93.9 +squad 17,487,615 89.1 94.9 squad1.1 14mb squad 87,599 87.7 94.0 ## results in this section we present our results using the best combination of models, algorithms, and parameters. in the following sections, we will perform detailed ablation study and show contributions from each of these choices. we train a 1.2 billion parameter answer generator, question generator, and question filtering model. in these experiments we use the entire squad1.1 dataset instead of only training on half of the labeled data since we are not doing any model or hyperparameter search. we then use these models to label synthetic data from two sources outside of squad1.1 . we first label data from real wikipedia documents with the overlapping documents from the squad1.1 training and dev set removed. in parallel we label data from synthetic wikipedia documents generated by an 8.3b gpt-2 model. this model was first trained with the megatron-lm codebase for 400k iterations before being finetuned on only wikipedia documents for 2k iterations. this allows us to generate high quality text from a distribution similar to wikipedia by using top-p (@xmath1.96) nucleus sampling. in figure 1 we examine the relationship between squad1.1 score and the amount of text labeled. we find that the performance of training with purely synthetic data observes a log-linear relationship that begins to saturate at approximately 100 mb of text labeled. however, finetuning these models on labeled squad1.1 data demonstrates continued improvement even beyond saturation. the performance of these post finetuned models continues to improve even past 500 mb of data labeled. in question generation we aim to effectively expand the knowledge and capabilities of our qa models by generating synthetic qa pairs not in the original dataset. the unsupervised lm takes this further to generate these qa pairs from synthetic text. generating synthetic questions from synthetic text is important because it draws upon knowledge not explicitly stated in the original corpus. while preliminary, it's noteworthy that the fully synthetic experiment works so well, and our results point towards a promising direction for future work. we provide qa examples generated from real and synthetic wikipedia in appendix a.7 & a.8. comparison with prior work. to quantify the improvements in question generation quality de-rived from improvements to language models and our generation techniques we compare our results to the original roundtrip consistency work from @xcite . we generate 3 million questions from real wikipedia text and finetune the public bert-large model on this data. we then finetune the model on the human-generated squad2.0 dataset and evaluate on the dev set. unlike prior work we do not generate any unanswerable questions, yet we find in data labeling cost: to label 8 million datapoints we used approximately 5200 gpu hours and 7200 cpu hours. with azure costs of ∼3$ per gpu hour this comes out to about 16 thousand dollars for the whole dataset or .2¢ per datum. with software optimization we expect further reductions. ## model scale we show in this section that as we improve pretraining tasks, pretraining scale, and model scale, synthetic data also improves. to show improvements in question generation we track the resulting squad1.1 evaluation score when a bert-style model is finetuned on the synthetic data. ## modeling choices while developing our synthetic data generation pipeline we explored several modeling and algorithmic choices before scaling up the model size and data quantity used. we pursued three axis of investigation, ablating choices for each model component of our pipeline at a time. ## related work early work using rule based question generation @xcite proposed the idea of over-generating and re-ranking questions with regression models learned over handcrafted linguistic features. @xcite used learned lstm models on extractive question answering datasets such as squad . these early works focused primarily on generating questions without explicit extracted answers in the text. @xcite proposed a two-stage neural model which added a model to estimate candidate answers and using the answers to generate questions. the current state of the art leverages transformer based language modeling including @xcite @xcite . @xcite uses seq2seq models to generate questions, and then enforce answer consistency on synthetic questions to filter out poorly generated questions in a technique called roundtrip consistency. @xcite uses a unified transformer rather than a seq2seq model to generate qa data in conjunction with roundtrip consistency. @xcite learn a model that generates unanswerable questions from an answerable example. the process of generating answers for answeraware question generation in recent literature has primarily leveraged cloze fill-in-the-blank passages to highlight an answer in a given context. some work uses ner or linguistic parsers to select passages for cloze translation as in @xcite . these methods are only able to generate answers for a subset of questions as squad1.1 is only made up of 52% named entity answers. more recent work such as @xcite use model based approaches to match the answer distribution of qa datasets and extract more complex answers. to improve the quality of synthetic data generation and downstream qa models, improving language model quality is crucial. in addition to pretraining task innovation, bert @xcite , roberta @xcite and al-bert @xcite have showed that increasing the size of available pretraining data directly improves downstream discriminative task performance. t5 @xcite , gpt-2 @xcite , ctrl @xcite , megatron-lm @xcite , and @xcite have shown that increasing language model scale improves the quality, coherency, and correctness of text generation. the models used in @xcite @xcite @xcite also demonstrate that larger models allow for better control in conditional language generation. squad style extractive question answering is not the only form of question answering. there are many other datasets covering a wide range of qa such as multihop @xcite @xcite , yes-no question @xcite , trivia questions @xcite , analytical questions @xcite , conversational and generative qas @xcite , unanswerable questions @xcite , and large multitask question answering datasets @xcite . while these are outside the scope of the current work, the insights developed improving quality for extractive squad questions will aid question generation in other domains. ## conclusions and future work we build upon existing work in large scale language modeling and question generation to push the quality of synthetic question generation. with our best models, we generate large question answering datasets from unlabeled wikipedia documents and finetune a 345 million parameter bert-style model achieving 88.4 em score. finetuning the resulting model on real squad1.1 data further boosts the em score to 89.4. this amounts to a 1.7 point improvement over our fully supervised baseline. finally, we generate synthetic text from a wikipedia-finetuned gpt-2 model, generate answer candidates and synthetic questions based on those answers, and then train a bert-large model and achieve similar question answering accuracy. doing so required us to scale model size for our answer generators, question generators, and filtration models. we hope that better synthetic questions will enable new breakthroughs in question answering systems and related natural language tasks. of particular interest for future work is handling low-resource question answering domains. for such a regime, one needs to analyze the effect of domain transfer and bootstrapping from a very small human labelled dataset. extension of this work to unanswerable and boolean questions is also a future work direction. more generally application of this work to multi dataset question generation with datasets such as multiqa @xcite is a promising avenue for future work. question: what indicates there must be data deletion early on in the visual pathway? context: evidence suggests that our visual processing system engages in bottom-up selection. for example, inattentional blindness suggests that there must be data deletion early on in the visual pathway. this bottom-up approach allows us to respond to unexpected and salient events more quickly and is often directed by attentional selection. this also gives our visual system the property of being goal-directed. many have suggested that the visual system is able to work efficiently by breaking images down into distinct components. additionally, it has been argued that the visual system takes advantage of redundancies in inputs in order to transmit as much information as possible while using the fewest resources. question: what type of antibiotic is cefalotin? question: what did "wanted dead or alive" rank on the billboard hot 100? context: "wanted dead or alive" is a song by american rock band bon jovi. it is from their 1986 album "slippery when wet". the song was written by jon bon jovi and richie sambora and was released in 1987 as the album's third single. during a february 20, 2008 encore performance in detroit, jon bon jovi told the crowd about running into bob seger at a pistons game. as he introduced his song "wanted dead or alive", he said it was inspired by seger's "turn the page" hit and called the song the band's anthem. the song peaked at #7 on the "billboard" hot 100 chart and #13 on the mainstream rock tracks chart, making it the third single from the album to reach the top 10 of the hot 100. as a result, "slippery when wet" was the first hard rock/glam metal album to have 3 top 10 hits on the "billboard" hot 100. question: who played the role of othello in the scene? context: the book begins when kostya and his fellow students are waiting for their first lesson with the director. they are excited and nervous at the prospect of meeting, and are surprised when he tells them that their first exercise is to put on a few scenes from a play. kostya and two of his friends perform scenes from "othello", with kostya taking the leading role. afterwards the director tells them their mistakes. question: who broke the phantom's mind? context: in the final episode of the game, it is revealed that fulbright is in fact deceased, and that the fulbright seen throughout the game is an international spy known as the phantom posing as him, as well as the one behind most of the game's major events. seven years prior to the game's events, the phantom was the catalyst of the ur-1 incident, having murdered metis cykes, athena's mother, sabotaged the hat-1 shuttle, and leaving simon blackquill to take the fall for the crime after seemingly incriminating evidence was found to point to simon as the only suspect. simon willingly allowed himself to be imprisoned in order to protect athena and to draw the phantom out, but athena suffered severe trauma from the ordeal, having believed for 7 years that she had actually murdered her mother, when in fact she stabbed the phantom in the hand in self-defense. in the present day, the phantom attempted to finish their case, murdering clay terran and bombing both the hat-2 shuttle and a courtroom in a desperate attempt to destroy incriminating evidence from the ur-1 incident. the phantom possesses a unique psychological makeup, showing very little, if any, emotion of any sort, nor any fear. the phantom also has no sense of self, claiming they do not know what their original gender, face, nationality, or identity even was in the beginning; having taken on so many disguises and identities, the phantom is an endless void. however, phoenix, apollo, and athena eventually managed to break the emotionless phantom severely in court, causing them to suffer a severe identity crisis, moments before an unseen sniper rifle takes the phantom's life. question: who was miss united kingdom in 1997? context: vicki-lee walberg (born 11 october 1975) is a model who was miss united kingdom in 1997, and made the top 10 at the miss world 1997 pageant. she was the last title holder to advance to the semifinal of the contest. walberg later went on to work in television and was a 'dolly dealer' in bruce forsyth's play your cards right on itv during its 2002 revival. question: what was the final score for the tottenham home match against newcastle united? context: he scored his first premier league hattrick in a 4-0 away win on boxing day against aston villa. on 5 january 2013, bale scored in the fa cup third round fixture against coventry city as well as assisting clint dempsey on both of his goals in a 3-0 win. on 30 january, bale scored a magnificent solo effort in the 1-1 draw with norwich city. bale then scored against west bromwich albion in a 1-0 away win on 3 february. bale then took his goal tally of the season to 15 goals with a brace against newcastle united in a match which spurs won 2-1. this took spurs into third place, and strengthened their champions league ambitions. question: who was arrested along with ernst sekunna? context: the arrests started in march 1917, with chandra kanta chakraverty "a thin-faced, falsetto-voiced hindu, a native of bengal, and a speaker of many languages", and the german, ernst sekunna, being arrested on charges of conspiracy. most of the others were arrested on april 8, including franz bopp, the german consul general for san francisco, e. h. von schack, deus dekker and wilhelm von brincken. the indian nationalists were accused of taking "advantage of american neutrality to plot on american soil against the allies" at "the expense of the laws and hospitality of the united states". the two men had also taken out trade names to do business as "the oriental society", "the oriental kitchen",and the "oriental review", and purchased of land in an isolated part of new york state. question: what protected the hulls of the chiyoda? context: "chiyoda" was a belted cruiser based on a much scaled-down version of the royal navy's. the hull was made of 84 watertight compartments, protected with harvey armor. originally designed to carry 12.6 inch canet guns, the plan was abandoned due to excessive top weight. instead, the design was changed so that her main battery consisted of ten qf 4.7 inch /40 naval guns in single mounts, mounted one each in the bow and stern, and four on each side in sponsons. the use of the elwick quick-firing technology resulted in an increase in the rate of fire by sixfold over previous cruiser designs. her secondary battery consisted of 14 qf 3 pounder hotchkiss and three 11-mm, 10-barrel nordenfelt guns. she was also equipped with three whitehead torpedo tubes mounted on the main deck. as was standard practice at the time, the prow was reinforced for ramming.