Title: ASGEN: ANSWER-CONTAINING SENTENCE GENERATION TO PRE-TRAIN QUESTION GENERATOR FOR DATA AUGMENTATION IN QUESTION ANSWERING

Abstract: Numerous machine reading comprehension (MRC) datasets often involve manual annotation, requiring enormous human effort, and hence the size of MRC data remains significantly smaller than that of those available for unsupervised learning, limiting the generalization capability. To overcome this issue, a new approach, which can generate synthetic question-and-answer data from large corpora such as Wikipedia, has been recently proposed. Such synthetic data can be utilized as additional data to pre-train the main MRC model before fine-tuning it using real, existing MRC data. However, the quality of generated questions and answers is still far from being satisfactory, so previous work introduced a pretraining technique for the question generator by pre-training on the generation of the next sentence that follows a paragraph. However, the next sentence that follows a paragraph may have little relevance to the questions or answers from within the paragraph, and thus it is not the ideal candidate for pre-training question generation. In response, we propose a novel method called Answer-containing Sentence Generation (ASGen). Our approach is composed of multiple stages, involving two advanced techniques, (1) dynamically determining K answers from a given document and (2) pre-training the question generator using the task of generating the answer-containing sentence. We evaluate the question generation capability of our method by comparing the BLEU score with existing methods and test our method by fine-tuning the MRC model using the downstream MRC data after training on synthetic data. Experimental results show that our approach achieves outperforms existing methods achieving new state-of-the-art results on SQuAD question generation and increases the performance of the state-of-the-art MRC models across a range of datasets such as SQuAD-v1.1, SQuAD-v2.0, KorQuAD, and QUASAR-T with no architectural modifications to the original MRC model.

1. INTRODUCTION:

Machine reading comprehension (MRC), which finds an answer to a given question from given paragraphs called context, is an essential task in natural language processing. With the use of high-quality human-annotated datasets for this task, such as SQuAD-v1.1 (Rajpurkar et al., 2016) , SQuAD-v2.0 (Rajpurkar et al., 2018) , and KorQuAD (Lim et al., 2019) , researchers have proposed MRC models, often surpassing human performance on these datasets. These datasets commonly involve finding a short snippet within a paragraph as an answer to a given question.

However, these datasets require a significant amount of human annotation to create pairs of a question and its relevant answer from a given context. Often the size of the annotated data is relatively small compared to that of data used in other unsupervised tasks such as language modeling. Hence, researchers often rely on the two-phase training method of transfer learning, i.e., pre-training the model using large corpora from another domain in the first phase, followed by fine-tuning it using the main MRC dataset in the second phase.

Most state-of-the-art models for MRC tasks involve such pre-training methods. Peters et al. (2018) present a bidirectional contextual word representation method called ELMo, which is pre-trained on a large corpus, and its learned contextual embedding layer has been widely adapted to many other MRC models. Devlin et al. (2019a) show that pre-training with a masked language model on a large corpus and then fine-tuning on a downstream dataset results in significant performance improvements.

However, pre-training on another domain task and then fine-tuning on a downstream task may suffer from performance degradation, depending on which pre-training task is used in the first phase. For example, Yang et al. (2019) show that the pre-training task of next sentence classification decreases performance on the downstream MRC tasks. To handle this problem, generating synthetic data similar to the those of a downstream task is crucial to obtain a properly pre-trained model. Recently, researchers have studied a model for generating synthetic MRC data from large corpora such as Wikipedia. This is essentially a form of transfer learning, by training a generation model and using this model to create synthetic data for training the MRC model, before fine-tuning on the downstream MRC dataset. Golub et al. (2017) suggest a two-stage synthesis network that decomposes the process of generating question-answer pairs into two steps, generating a fixed number (K) of answers conditioned on the paragraph, and question generation conditioned on the paragraph and the generated answer. Devlin et al. (2019b) introduced a pre-training technique for the question generator of this method by pretraining on the generation of next-sentence that follows the paragraph.

However, choosing a fixed number (K) of candidate answers from each paragraph will lead to missing candidates if K is too small, and will lead to having lower-quality candidates if K is too big. Moreover, the next sentence generation task is not conditioned on the answer, despite the answer being a strong conditional restriction for question generation task. Also, the next sentence that follows a paragraph may have little relevance to the questions or answers from within the paragraph, and hence is not the ideal candidate for pre-training question generation.

To address these issues, we propose Answer-containing Sentence Generation (ASGen), a novel method for a synthetic data generator with two novel processes, (1) dynamically predicting K answers to generate diverse questions and (2) pre-training the question generator on answer-containing sentence generation task. We evaluate the question generation capability of our method by comparing the BLEU score with existing methods and test our method by fine-tuning the MRC model on downstream MRC datasets after training on the generated data. Experimental results show that our approach outperforms existing generation methods, increasing the performance of the state-ofthe-art MRC models across a wide range of MRC datasets such as SQuAD-v1.1, SQuAD-v2.0, KorQuAD, and QUASAR-T (Dhingra et al., 2017) without any architectural modifications to the MRC model.

2. PROPOSED METHOD:

This section discusses the details of our proposed ASGen method. ASGen consists of a BERT-based generative model (BertGen) and answer-containing sentence generation pre-training (AS). First, we will describe how BertGen model generates synthetic data from Wikipedia. Next, we will explain the novel components of our methods and how we pre-trained the question generator in BertGen based on them. BertGen encodes paragraphs in Wikipedia with two separate generation networks, the answer generator and the question generator.

Answer Generator. As shown in Fig. 2 -(1), we generate the number of answer candidates K for a given context without the question by applying a fully connected feed-forward layer on the contextual embedding of classification token "[CLS]". To make the contextual embeddings and to predict answer spans, we utilize a BERT (Devlin et al., 2019a) encoder (Fig. 2-BERT Encoder-A ). Depending on the predicted number K, we select the K top candidate answer spans from the context. As shown in Fig. 2-( 2), we use the K selected candidate answer spans as input to the question generator.

Q t-1 Q t 0 Q t-2 Q t-1 K (1)

Question Generator. Next, as shown in Fig. 2-( 2), we generate a question conditioned on each answer predicted from the answer generator. Specifically, we pass as input to a BERT encoder the context and an indicator for the answer span location in the context (Fig. 2 -BERT Encoder-Q). Next, a Transformer decoder (Vaswani et al., 2017) generates the question word-by-word based on the encoded representation of the context and the answer span. For pre-training such a question generator on an answer-containing sentence generation task, we exclude the answer-containing sentence from the original context and train the model to generate the excluded sentence given the modified context and the answer span as input.

Finally, we generate questions and answers from a large corpus, e.g., all the paragraphs in Wikipedia in this paper. After generating such data, we train the MRC model on the generated data in the first phase and then fine-tune on the downstream MRC dataset (such as SQuAD) in the second phase. In this paper, we use BERT as the default MRC model, since it exhibits state-of-the-art performance in many MRC datasets.

2.1. DYNAMIC ANSWER PREDICTION:

The most natural method for humans to create a question-answer pair from a given context is to select the answer first and then create a corresponding question. In this situation, we conjecture that a human is more likely to choose as an answer a phrase that is "answer-like", such as keyphrases, nouns, dates, names, etc. There may be several answers in the context that are likely to be selected by humans as answers, especially if the context is lengthy or if it contains multiple nouns, dates, names, etc. 1 2 3 4 5 6 7 8 9 For example, the context "Barack Hussein Obama II is an American attorney and politician who served as the 44th president of the United States from 2009 to 2017" can have as possible answers "Barack Hussein Obama", "44th", "United States", "2009 to 2017", etc. As shown in Fig. 4 , to see these characteristics, we examine the distribution of the number of answers in the SQuAD dataset and hypothesize that there exists an underlying pattern in the number of answers that occur in a context. The conventional method to generate multiple answers from a context is to draw a fixed number (K) of answers. However, this approach can generate low-quality answers if K is too big, and it can impact the number and diversity of the generated answers if K is too small. Therefore, we predict the number of answers K in a given context W = {w t } T 0 using regression as,

{w enc t } T t=0 = BERT Encoder-A(W) t , K = f k (w enc 0 )

, where T is the number of word tokens in the context with position 0 reserved for classification token '[CLS]', and f k represents a fully connected unit with two hidden layers that have hidden dimensions equal to H and 1, respectively, where H is the hidden dimension of BERT Encoder-A.

To calculate the score s i for start index i of a predicted answer span, we compute the dot product of the encoder output with a trainable start vector S. For each start index i, we calculate the span end index score e i,j for end index j in a similar manner with a trainable end vector E, but conditioned on i, i.e.,

{s i } T i=0 = S • w enc i {e i,j } T,T i,j=0 = E • f s (w enc j ⊕ w enc i ),

where f s represents a fully connected layer with hidden dimension H and ⊕ indicates the concatenation operation. For training, we use the mean squared error loss between K and ground-truth number of answers. We also use cross-entropy loss on the s i ,e i,j and ground truth start/end of the answer span for each token. Predicting the number of answers and predicting the span are jointly trained to minimize the sum of their respective losses.

During inference, we choose the K top answer spans with the highest score summation of start index score and end index score, i.e.,

A span = {(i, j) | 1 ≤ i < T and i ≤ j < T }, a k = max({a | #{(i, j) | (i, j) ∈ A span and s i + e i,j ≥ a} = K}), A span k = {(i, j) | (i, j) ∈ A span and s i + e i,j ≥ a k }.

The K selected answer spans A span k are then given to the question generator as input in the form of an indication of the answer span location.

2.2. PRE-TRAINING QUESTION GENERATOR:

In order to generate questions conditioned on different answers that may arise in a context, we generate a question for each of the K answers. Devlin et al. (2019b) previously proposed to pre-train this generation model with an unsupervised task that generates the next sentence following a given paragraph to improve generation performance. We identify several issues with this approach. The final question generation task has the form of sentence generation given an answer and a context, while the next-sentence generation has no answer component. The next-sentence generation task is not conditioned on the answer, despite the answer being a strong conditional constraint for the question generation task. Also, the next sentence that follows a paragraph may have little relevance to the questions or answers from within the paragraph, and hence is not the ideal candidate for pre-training question generation.

To address these issues, we modify the context to exclude the sentence containing our previously generated answer and pre-train our generator on the task of generating this excluded answercontaining sentence, conditioned on the answer and the modified context. Specifically, we exclude answer-containing sentence S ans while leaving the answer and modify the original context D to D ans as

S start = {p | sentence start index = p}, S ans = {(p, q, i, j) | max({p|p≤i}), min({q|q≥j}), (i, j) ∈ A span k , p ∈ S start , q ∈ S start }, D ans = [D [:p] ; D [i:j] ; D [q:] ], (p, q, i, j) ∈ S ans .

Note that we change S ans to not exclude the answer-containing sentence in the case of fine-tuning on the question generation, i.e., S ans = {(p, q, i, j)|p = i, q = j}.

Afterwards, we pass the previously generated answer to the sequence-to-sequence generation model as a segmentation encoding M ans that identifies the answer part within the context, i.e.,

M ans = [m 0 * p; m 1 * (j -i); m 0 * (T -q)], (p, q, i, j) ∈ S ans ,

where m 0 and m 1 indicate trainable vectors corresponding to segmentation id 0 and 1, respectively. Here we tag the segmentation id for each word in the context as 0 and each word in the answer as 1. A * B indicates the operation of concatenating vector A for B many times.

Next, we generate answer-containing sentence embedding W g = {w g t } T 0 using a Transformer sequence-to-sequence model (the encoder part is initialized with BERT) as

w g t = Transformer Decoder({w g i } t-1 i=0 , BERT Encoder-Q(D ans , M ans )).

Finally, we calculate the loss of the generation model with cross-entropy over generated sentence words, i.e.,

{w o t } T t=0 = {Softmax(w g t E)} T t=0 , L = - T t=1 D i=1 y t,i log(w o t,i ) + (1 -y t,i )log((1 -w o t,i )) /T,

where y indicates a ground-truth one-hot vector of the answer-containing sentence word (the question word in the case of fine-tuning), D is the vocabulary size, and E ∈ R d×D represents a word embedding matrix shared between the BERT Encoder-Q and the Transformer decoder.

In this manner, we pre-train the question generation model using a task similar to the final task of conditionally generating the question from a given answer and a context.

3. EXPERIMENTAL SETUP:

Pre-training Dataset. To build the dataset for answer-containing sentence generation tasks (AS) and the synthetic MRC data for pre-training the downstream MRC model, we collect all paragraphs from the entire English Wikipedia dump (Korean Wikipedia dump for KorQuAD) and synthetically generate questions and answers on these paragraphs. We apply extensive filtering and cleanup to only retain high quality collected paragraphs from Wikipedia. Detailed pre-processing steps for obtaining the final Wikipedia dataset can be found in the supplemental material.

Using the answer generator in ASGen (BertGen+AS), we generate 43M answer-paragraph pairs (Full-Wiki) from the final Wikipedia dataset for pre-training on answer-containing sentence generation. For ablation studies on pre-training approaches, we also sample 2.5M answer-paragraph pairs (Small-Wiki) from Full-Wiki and 25K answer-paragraph pairs (Test-Wiki) to evaluate the pretraining method. Finally, using the question generator in ASGen (BertGen+AS), we generate one question for each answer-paragraph pair in Full-Wiki and create the final synthetic MRC data containing 43M triples of a paragraph, its question and its answer.

Benchmark Datasets. In most MRC datasets, a question and a context are represented as a sequence of words, and the answer span (indices of start and end words) is annotated from the context words based on the question. Among these datasets, we choose SQuAD as the primary benchmark dataset for question generation, since it is the most popular human-annotated MRC dataset. SQuAD-v1.1 (Rajpurkar et al., 2016) consists of crowd-sourced questions and answers based on contexts from Wikipedia articles. We compare our question generation capability with existing question generation methods such as UniLM (Dong et al., 2019) . For fair comparison, we split the training set of SQuAD-v1.1 data into our own training and test sets, and keep the original development set as our dev set, as previously done in Du et al. (2017) , Kim et al. (2019) , and Dong et al. (2019) . We call this dataset as Test Split11 . We also evaluate on the reversed dev-test split, called Test Split2.

To evaluate the effect of generated synthetic MRC data, we evaluate the fine-tuned MRC model on the downstream MRC dataset after training on the generated synthetic data. We perform this on SQuAD-v1.1 and SQuAD-v2.0 (Rajpurkar et al., 2018) . We also evaluate on KorQuAD (Lim et al., 2019) which is another dataset created with the same procedure as SQuAD-v1.1 for Korean language. To show that our generated data is useful for other MRC datasets, we fine-tune and test the MRC model on QUASAR-T (Dhingra et al., 2017) which is large-scale MRC dataset, after training on the synthetic data that generated from SQuAD-v1.1.

Implementation Details. For the answer generator, we use BERT (Devlin et al., 2019a) and two fully connected layers to predict the number of answers K. For the BertGen model, we use pretrained uncased BERT (Base) as encoder and 12 layers of Transformer as decoder. For the generation of unanswerable questions as in SQuAD-v2.0, we separate unanswerable cases and answerable cases and train separate generation models. For the final MRC model, we use BERT (Large) which is the state-of-the-art model on multiple datasets with all official hyper-parameters. We use the Mecab (Kudo, 2006) tokenizer for Korean to separate postposition words which do not exist in English.

Comparison of the Pre-training Method. We compare our question generation pre-training method, which is pre-training on answer-containing sentence generation task (AS), with a method from Devlin et al. (2019b) , which is pre-training on next-sentence generation task (NS), and with a method from Golub et al. (2017) , which only trains question generation on final MRC dataset. We reproduced these methods on BertGen as they were described in their original work for comparison. Note that 'BertGen+AS' is equivalent to 'ASGen'. We generate synthetic data from Wikipedia using these approaches which are trained on the target downstream MRC datasets except for QUASAR-T.

In the case of QUASAR-T, we use synthetic data which is generated by ASGen trained on SQuAD-v1.1. To check the effectiveness of our method on downstream MRC tasks, we evaluate our generated data on SQuAD-v1.1, SQuAD-v2.0, KorQuAD and QUASAR-T by training state-of-the-art models (BERT and BERT+CLKT2 ) on generated data followed by fine-tuning on the train set for each dataset. The structure of 'BERT + CLKT' model is the same as that of original BERT except that the model is pre-trained for the Korean language. Due to the absence of common pre-trained BERT for Korean, we used this model as a baseline to demonstrate the effectiveness of our method.

4. QUANTITATIVE RESULTS:

Dynamic Answer Prediction. We conducted an experiment to demonstrate the performance of our method in generating the number of answers in a given context. As shown in Table 1 , in the case of fixed K, the mean absolute error from the ground-truth K gt is the smallest at K pred = 5 and the values are 1.92 and 0.99 for Test Split1 and Test Split2, respectively. Thresholding on the sum of the start and end logits with a fixed threshold value which minimizes the mean absolute error results in (Dong et al., 2019) 22.1 23.8 ASGen (Full-Wiki) 21.5 24.7 ASGen (Full-Wiki) (Large)

25.4 28.0 an error of 2.31 and 1.12, respectively in the two splits. In contrast, our answer generator generates a more appropriate number of answers than the fixed K approach, by reducing the mean absolute error between the ground-truth K gt and the prediction K pred of 1.24 and 0.76, respectively for the two splits.

Question Generation. To evaluate our question generator, we fine-tune the model on both Test Split1 and Test Split2, after pre-training answer-containing sentence generation on Full-Wiki. As shown in Table 2 , ASGen outperforms existing methods by 0.9 BLEU-4 score on Split2, 24.7 for ASGen vs. 23.8 for UniLM. Moreover, our final question generation model, ASGen (Large), outperforms existing methods by a large margin in BLEU-4 score on both splits, 25.4 for ASGen (Large) vs. 22.1 for UniLM for Split1 and 28.0 for ASGen (Large) vs. 23.8 for UniLM for Split2.

To show the effectiveness of our answer-containing sentence pre-training task (AS), we compare between various pre-training tasks. As shown in Table 3 , AS is shown to perform better than NS, e.g. 21.5 vs. 18.2 and 24.7 vs. 19.7 in the two splits, respectively. Note that conditioning on a given answer has only a small effect on AS, e.g. 19.4 vs 19.5. This implies the performance gain is largely due to pre-training on the answer-containing sentence generation task rather than conditioning on a given answer.

We also compare the BLEU-4 scores between before and after applying AS on other existing question generation models. We reproduce Zhao et al. ( 2018) and use the official code of Dong et al. (2019) . As shown in Table 4 , AS consistently improves the performance of other question generation models with no architecture changes or parameter tuning.

Downstream Task Performance. We conduct experiments by training MRC models on the synthetic data generated by ASGen from Wikipedia before fine-tuning the model on the downstream dataset to show the effectiveness of our synthetic data generation. For each dataset, the MRC model is pre-trained on the corresponding generated synthetic data and fine-tuned on the downstream data. As shown in Table 5 , the MRC model pre-trained on the synthetic data generated by ASGen shows an improvement of 1.9 F1 score on SQuAD-v1.1, 4.0 F1 score on SQuAD-v2.0, and 0.5 F1 score on KorQuAD from the state-of-the-art baseline models. Moreover, using the synthetic data generated from ASGen shows better performance than using the synthetic data generated from 'BertGen+NS' on both SQuAD-v1.1 and SQuAD-v2.0 downstream data.

Effects of MRC and Synthetic Data Size. Fig. 5 shows the effects of synthetic data with respect to the size of the synthetic and real MRC data. In Fig. 5-(a) , where we fix the size of synthetic data as 43M, the F1 score of MRC model pre-trained on the synthetic data generated by ASGen consistently outperforms that of BertGen+NS. In particular, performance difference becomes apparent for a small size of real MRC data, while the performance gap diminishes for a large size. Such a gap may become insignificant for a sufficient size of real MRC data, but for the current size of SQuAD data (87K in total) AS still improves the performance.

As shown in Fig. 5 -(b), we also conducted experiments by training the MRC model using a different amounts of generated synthetic data for the same number of iterations, while using the full size of real SQuAD data. The total number of training steps for all data sizes is kept the same as that of 10M synthetic data. A larger size of generated data consistently gives better performance.

Transfer Learning to Other Datasets. In this experiment, we first fine-tune ASGen using SQuAD-v1.1, and using synthetic data generated by this ASGen, we train BERT MRC model. Afterwards, we fine-tune BERT for the downstream MRC task using QUASAR-T, in order to verify that the data generated in this manner is useful for other MRC datasets. QUASAR-T has two separate datasets, one with short snippets as context, and the other with long paragraphs as context. As shown in Table 6 , training with our synthetic data is shown to improve the F1 score by 2.2 and 1.7 for the two cases, respectively.

5. QUALITATIVE RESULTS:

Comparison of Question Generation. We qualitatively compare the generated questions after pretraining with NS and AS to demonstrate the effectiveness of our method. For the correct answer "49.6%" as shown in the first sample in Table 7 , NS omitted "Fresno", which is a critical word to make the question specific, while AS's question does not suffer from this issue. Note that the word "Fresno" occurs in the answer-containing sentence. This issue also occurs in the second sample, where NS uses the word "available" rather than the more relevant words from the answer-containing sentence, but AS uses many of these words such as "most" and "popular" to generate contextually rich questions. Also, the question from NS asks about "two" libraries, while the answer has "three" libraries, showing the lack of sufficient conditioning on the answer. The third sample also shows that AS draws more context-related questions than NS by including the exact subject "TARDIS" to use for the corresponding answer in a similar vein.

6. RELATED WORK:

Machine Reading Comprehension. For MRC tasks, a large number of datasets have been proposed, most often focused on finding an answer span for a question from a given paragraph. Popular and fully human-annotated datasets include SQuAD-v1.1 (Rajpurkar et al., 2016) , SQuAD-v2.0 (Rajpurkar et al., 2018) , KorQuAD (Lim et al., 2019), and HotpotQA (Yang et al., 2018) . However, these datasets are relatively small with around 100K samples each, which is far smaller than those datasets used for unsupervised tasks such as language modeling.

Question Generation. Question generation methods have been actively studied for various purposes including data augmentation in question answering. Du et al. (2017) proposed an attention-based model for question generation by encoding sentence-level as well as paragraph-level information. Song et al. (2018) introduced a query-based generative model to jointly solve question generation and answering tasks. Kim et al. (2019) separately encoded the answer and the rest of the paragraph for proper question generation. Zhao et al. ( 2018) utilized a gated self-attention encoder with a max-out unit to handle long paragraphs. Our proposed method (AS) can further improve the question generation quality of these methods by pre-training them with an answer-containing sentence generation task.

Transfer Learning. Pre-training methods have been increasingly popular in natural language processing to obtain contextualized word representations. Open-GPT (Radford et al., 2018) , BERT (Devlin et al., 2019a ), XLNet (Yang et al., 2019) , and UniLM (Dong et al., 2019 ) use a Transformer module (Vaswani et al., 2017) to learn different styles of language models on a large dataset fol- lowed by fine-tuning on the downstream task. While our approach is similar to these approaches, our pre-training task for question generator generates answer-containing sentences to learn better representations for the question generation task. Synthetic Data Generation. Subramanian et al. (2018) show that neural models generate better answers than using off-the-shelf tools for selecting named entities and noun phrases. Golub et al. (2017) proposed to separate the answer generation and the question generation. This model generates questions conditioned on generated answers, and then they evaluate the quality of the synthetic data by training an MRC model with them before fine-tuning on SQuAD. Inspired by the observations from previous studies, we improved the performance of answer generation and question generation by using a newly designed models as well as a novel pre-training technique.

7. CONCLUSIONS:

We propose two advanced training methods for generating high-quality and diverse synthetic data for MRC. First, we dynamically choose the K top answer spans from an answer generator and then generate the sentence containing the corresponding answer span as a pre-training task for the question generator. Using the proposed methods, we generate 43M synthetic training samples and train the MRC model before fine-tuning on the downstream MRC dataset. Our proposed method outperforms existing questions generation methods achieving new state-of-the-art results on SQuAD question generation and consistently shows the performance improvement for the state-of-the-art models on SQuAD-v1.1, SQuAD-v2.0, KorQuAD, and QUASAR-T datasets without any architectural modification to the MRC model. 2) --QUASAR-T(Short) 76.6(±0.1) 80.3(±0.2) 76.6(±0.2) 80.0(±0.3) QUASAR-T(Long) 74.9(±0.6) 78.1(±0.7) 74.0(±0.8) 76.9(±0.9)

:

We use the identical splitting of SQuAD data provided by UniLM from its publicly available website (https://github.com/microsoft/unilm)

'BERT+CLKT with ASGen' model can be found as 'BERT-CLKT-MIDDLE' on the leaderboard (https: //korquad.github.io/KorQuad%201.0)

Figures and Tables:

Figure 1: Overview of our Answer-containing Sentence Generation (ASGen) method.

Figure 3: Detailed procedure of generating the synthetic MRC data from Wikipedia using ASGen.

Figure 4: Distribution of the number of answers in each paragraph in SQuAD-v1.1.

MRC model on SQuAD devset (b) Used amount of synthetic data.

Figure 5: Comparison of F1 of the BERT model on SQuAD-v1.1 dev set with different data sizes.

(omit) Question by NS Which two libraries are available for undergraduates to use? Question by AS What are the three most popular libraries for undergraduates? Context (omit) ... He fled from Gallifrey in a stolen Mark I Type TARDIS "Time and Relative Dimension in Space" time machine which allows him to travel across time and space. The TARDIS has a "chameleon circuit"...(omit) Question by NS What does the doctor refer to? Question by AS What does the TARDIS stand for?

Mean absolute error of prediction K pred with respect to ground-truth K gt . The results are obtained on SQuAD Test Split1 and Test Split2.

Table content: <table><tr><td>Model</td><td colspan="2">Mean Absolute Error Test Split1 Test Split2</td></tr><tr><td>Thresholding on Answer Logits</td><td>2.31</td><td>1.12</td></tr><tr><td>Fixed-K (K pred = 5)</td><td>1.92</td><td>0.99</td></tr><tr><td>Dynamic-K Answer Prediction</td><td>1.24</td><td>0.76</td></tr></table>

Comparison of BLEU-4 scores with existing models on SQuAD Test Split1 and Test Split2. ASGen (Large) has 24 layers of encoder and decoder. Those models with * are reproduced.

Table content: <table><tr><td>Question Generation Model</td><td colspan="2">BLEU-4 score Test Split1 Test Split2</td></tr><tr><td>Du et al. (2017)</td><td>12.3</td><td>-</td></tr><tr><td>Zhao et al. (2018)*</td><td>12.8</td><td>14.9</td></tr><tr><td>ASs2s (Kim et al., 2019)</td><td>16.2</td><td>-</td></tr><tr><td>Zhao et al. (2018)</td><td>-</td><td>16.4</td></tr><tr><td>UniLM</td><td/><td/></tr></table>

Comparison among pre-training methods of the question generator in ASGen, i.e, without pre-training, pre-training on NS, pre-training on AS, pre-training on AS without conditioning on a given answer. Note that we use Small-Wiki for comparison of pre-training except those entries including "(Full-Wiki)".

Table content: <table><tr><td/><td/><td>BLEU-4 score</td><td/></tr><tr><td>Model + pre-training method</td><td colspan="3">Wikipedia SQuAD v1.1 SQuAD v1.1</td></tr><tr><td/><td>Test-Wiki</td><td>Test Split1</td><td>Test Split2</td></tr><tr><td>BertGen</td><td>-</td><td>14.4</td><td>16.9</td></tr><tr><td>BertGen + NS</td><td>1.4</td><td>17.5</td><td>19.2</td></tr><tr><td>BertGen + AS wo/ condition to a given answer</td><td>6.8</td><td>19.4</td><td>21.2</td></tr><tr><td>BertGen + AS [ASGen]</td><td>7.1</td><td>19.5</td><td>22.2</td></tr><tr><td>BertGen + NS (Full-Wiki)</td><td>3.4</td><td>18.2</td><td>19.7</td></tr><tr><td>BertGen + AS (Full-Wiki) [ASGen]</td><td>11.3</td><td>21.5</td><td>24.7</td></tr></table>

Effects of pre-training on answer-containing sentence generation (AS) on other existing methods. We use Small-Wiki data to pre-train existing models. Those models with * are reproduced.

Table content: <table><tr><td/><td/><td>BLEU-4 score</td><td/></tr><tr><td>Model + pre-training method</td><td colspan="3">Wikipedia SQuAD v1.1 SQuAD v1.1</td></tr><tr><td/><td>Test-Wiki</td><td>Test Split1</td><td>Test Split2</td></tr><tr><td>Zhao et al. (2018)*</td><td>-</td><td>12.8</td><td>14.9</td></tr><tr><td>Zhao et al. (2018)* + AS</td><td>6.8</td><td>14.0</td><td>16.2</td></tr><tr><td>UniLM (Dong et al., 2019)</td><td>-</td><td>22.1</td><td>23.8</td></tr><tr><td>UniLM (Dong et al., 2019) + AS</td><td>8.9</td><td>22.9</td><td>24.5</td></tr></table>

Comparison of EM/F1 scores of fine-tuned MRC model on SQuAD v1.1, SQuAD v2.0, and KorQuAD dev sets using their corresponding synthetic data for pre-training.

Table content: <table><tr><td/><td>Model</td><td colspan="2">Synthetic data</td><td/><td colspan="3">SQuAD v1.1 SQuAD v2.0 KorQuAD EM F1 EM F1 EM F1</td></tr><tr><td/><td/><td>-</td><td/><td/><td colspan="3">83.9 90.9 78.8 81.8</td><td>-</td><td>-</td></tr><tr><td/><td>BERT</td><td colspan="2">BertGen BertGen + NS</td><td/><td colspan="3">85.1 91.4 80.9 83.9 85.6 92.3 81.5 85.1</td><td>--</td><td>--</td></tr><tr><td/><td/><td colspan="6">BertGen + AS [ASGen] 86.3 92.8 82.5 85.8</td><td>-</td><td>-</td></tr><tr><td/><td>BERT + CLKT</td><td colspan="3">-BertGen + AS [ASGen]</td><td>--</td><td>--</td><td>--</td><td>--</td><td>87.1 94.5 87.8 95.0</td></tr><tr><td>F1 score of MRC model on SQuAD devset</td><td>82 84 86 88 90 92 94 80</td><td/><td/><td>ASGen BertGen+NS wo/ Synthetic</td><td/><td/></tr><tr><td/><td>1.25K</td><td>5K</td><td>20K</td><td>87K(Max)</td><td/><td/></tr></table>

Comparison of EM/F1 scores of the BERT fine-tuned on QUASAR-T dataset. The synthetic data are generated from ASGen trained on SQuAD-v1.1.

Table content: <table><tr><td colspan="2">Model Synthetic data</td><td>Short(Dev) Short(Test) Long(Dev) Long(Test) EM F1 EM F1 EM F1 EM F1</td></tr><tr><td>BERT</td><td colspan="2">-ASGen (SQuAD-v1.1) 76.5 80.1 76.5 80.0 74.2 77.4 73.8 76.5 74.3 78.6 74.1 77.8 72.1 75.6 72.1 74.8</td></tr></table>

Examples from SQuAD-v1.1 dev set demonstrating generated questions. We compare our method (AS) with NS. Colored Text indicates a given answer.ContextThe 2010 United States Census reported that Fresno had a population of 494,665. The population density was 4,404.5 people per square mile.(1,700.6km). The racial makeup of Fresno was 245,306 ( 49.6% ) White, 40,960 (8.3%) African American, 8525 (1.7%) Native American ... (omit) Question by NS What percent of the population is White? Question by AS What percentage of the Fresno population is White? Context (omit) ... According to the American Library Association, this makes it the largest academic library in the United States, and one of the largest in the world. Cabot Science Library, Lamont Library, and Widener Library are three of the most popular libraries for undergraduates to use, with ...

Table content: <table/>

Additional experiments for effectiveness of AS on SQuAD Test-Split3. We use Small-Wiki data to pre-train existing models. Those models with * are reproduced.

Table content: <table><tr><td/><td colspan="2">BLEU-4 score</td></tr><tr><td>Model + pre-training method</td><td colspan="2">Wikipedia SQuAD v1.1</td></tr><tr><td/><td>Test-Wiki</td><td>Test Split3</td></tr><tr><td>Zhao et al. (2018)</td><td>-</td><td>16.8</td></tr><tr><td>Zhao et al. (2018)*</td><td>-</td><td>16.2</td></tr><tr><td>Zhao et al. (2018)* + AS</td><td>6.8</td><td>17.5</td></tr></table>

Standard errors of EM/F1 scores in downstream MRC tasks with ASGen.

Table content: <table><tr><td>Dataset</td><td>EM</td><td>Dev set</td><td>F1</td><td>EM</td><td>Test set</td><td>F1</td></tr><tr><td>SQuAD-v1.1</td><td colspan="3">86.1(±0.2) 92.6(±0.1)</td><td>-</td><td/><td>-</td></tr><tr><td>SQuAD-v2.0</td><td colspan="3">82.3(±0.3) 85.6(±0.</td><td/><td/><td/></tr></table>

Peer review (iclr2020):
This paper proposes a pretraining technique for question generation, where an answer candidate is chosen beforehand, and the objective is to predict the answer containing sentence given a paragraph excluding this sentence and the target answer candidate. The intuition of this method is that question generation requires to generate a sentence which contains the information about the answer while being conditioned on the given paragraph. In particular, the paper compares its approach to Devlin’s presentation (https://nlp.stanford.edu/seminar/details/jdevlin.pdf according to the references; is it not a published work?) which uses next sentence generation for pretraining, that is less related to the downstream question generation task. The proposed pretrained model is then used to finetune on a standard question generation task, which is then used to generate synthetic QA pairs for data augmentation in QA. The proposed method is evaluated on four datasets (SQuAD v1, SQuAD v2, KorQuAD, QUASAR-T) and have shown substantial performance gain. The strength of this paper is clear to me: the idea in the paper is new and interesting, and they provide a good intuition of why the proposed learning objective is helpful compared to the previous methods. However, I have several concerns as follows. First, the performance improvements are marginal (less than 1.0 on three datasets, except QUASAR-T, which is actually interesting given it is under transfer learning setting). Second, there is a very related work, “Dong et al. Unified Language Model Pre-training for Natural Language Understanding and Generation NeurIPS 2019” which also proposes a new pretraining approach for question generation and data augmentation for question answering. Not only this submission did not discuss this paper (it was posted in May 2019, which is substantially ahead of ICLR deadline), the result is substantially worse in both BLEU score of question generation and EM/F1 of end QA performance after data augmentation on SQuAD v2. Third, the paper does not discuss any work in question generation overall, although the task of question generation was studied for decades, including question generation as data augmentation for question answering. I believe this paper should discuss previous work in question generation and compare the performance with them. Some of the work on question generation. - Heilman & Smith. Good Question! Statistical Ranking for Question Generation. NAACL 2010. - Wang et al. Learning to ask questions in open-domain conversational systems with typed decoders. ACL 2018. - Zhao et al. Paragraph-level Neural Question Generation with Maxout Pointer and Gated Self-attention Network. EMNLP 2018. - Kim et al. Improving Neural Question Generation Using Answer Separation. AAAI 2019. Some of the work on question generation for question answering. - Du et al. Learning to ask: Neural question generation for reading comprehension. ACL 2017. - Song et al. A Unified Query-based Generative Model for Question Generation and Question Answering. AAAI 2018. - Tang et al. Learning to Collaborate for Question Answering and Asking. NAACL 2018. - Sachan & Xing, Self-training for jointly learning to ask and answer questions. NAACL 2018. - Tang et al. Question Answering and Question Generation as Dual Tasks. AAAI 2018. - Lewis et al. Unsupervised Question Answering by Cloze Translation. ACL 2019. Fourth, it looks like the paper uses the development set for both development and the final evaluation. The paper should either use a portion of the train set for development and treat the original development set as the test set, or just show the final model’s performance on the hidden test set by submitting to the leaderboard. (Especially, for Table 3, did authors reimplement BERT+NS and evaluate on the dev set? Because Devlin’s presentation only shows the result on the test set from the official leaderboard.) (This is not a weakness of the paper, but more a subjective opinion about the paper*s claim) The paper claims answer-containing sentence generation is close to question generation. However, unlike question generation where core information about the answer is given as input text, answer containing sentence generation should generate the information about the answer itself with no given information. Consider the first example in Table 5. Question generation requires to read “The racial makeup of Fresno was 245,306 (49.6%)” and generate question “What percentage of the Fresno population is White?” However, in the proposed pretraining technique, the generation model is supposed to generate “The racial makeup of Fresno was 245,306 (49.6%)” with no information. In fact, I think the fact that answer candidate is given makes it closer to question generation task, rather than “generating answer-containing sentence” is the key. (By the way, I am giving 3 as a rating although my actual rating is closer to 4 or 5, because 4 or 5 is blocked in the review system. I am happy to increase the score after rebuttals.) -------------------------------------------------------------------------------------------------------------------------------------------- Now, here are clarification questions. Regarding Section 2.1 1) Did you attempt to predict ‘number of answer candidates’ by regression or classification? Which objective function did you use? Based on the current description, it looks like it’s neither regression nor classification which makes me confused. 2) I believe the second last formula in this section should say “#{...}=k” instead of “#{...}”? 3) Have you compared with thresholding instead of predicting the number of answer candidates? 4) What is the objective of training the span selection model? There are multiple candidate spans given a single sentence, and hopefully you do not want to discourage candidates except only one. 5) How was this model trained (both predicting the number and predicting the span)? Was it trained on QA datasets such as SQuAD? Then, did you train this model & go through preprocessing separately for each dataset? (Except QUASAR-I which uses synthetic data from SQuAD v1.) 6) I believe that the most popular approach in question generation literature is selecting named entities & noun phrases using off-the-shelf tools, and wonder if authors have compared their method with this baseline. Regarding Section 2.2 1) Although this section is the most important section in the paper, the description is a bit confusing to me. My understanding is that, if the initial paragraph has three sentences, <Sentence 1>, “The racial makeup of Fresno was 245,306 (49.6%).”, and <Sentence 3>, and the answer prediction model (described in Section 2.1) chooses “49.6%”, the question generation model is given “<Sentence 1> 49.6% <Sentence 3>”, and the model is supposed to generate “The racial makeup of Fresno was 245,306 (49.6%).”. Is it correct? 2) Did you insert any special token to indicate whether it is the sentence or the target answer candidate, or which sentence is before or after the target sentence? 3) Have you tried the formulation of the masked language model, instead of the proposed model? Regarding Section 3 Is preprocessing the same for all Golub et al 2017, NS and AS? Regarding Section 4 1) I am curious why the improvement in BLEU-4 score (in Table 2) is significant but the improvement in the end task (in Table 3) is marginal. Do you have any intuition on it? 2) For Figure 5(a), do you have results with BERT without any data augmentation as well? I wonder if using too small number of SQuAD annotations makes question generation inaccurate, which makes synthetic data be bad quality and hurts the performance compared to not using any synthetic data. 3) How many synthetic QA pairs are used for Table 3? Curious since the model gets around 92.5 with the largest amount of data shown in Figure 5(b) but the number reported in Table 3 is 92.8. **** Update on Nov 10 **** Increasing the score to 6.