SlowGuess/ACL_2020_OCR huggingface dataset:
Zhongli Li†*, Wenhui Wang‡, Li Dong‡, Furu Wei‡, Ke Xu†
†Beihang University
$^{\ddagger}$ Microsoft Research
{lizhongli@,kexu@nlsde.}buaa.edu.cn
{wenwan, lidong1, fuwei}@microsoft.com
Question Answering (QA) has shown great success thanks to the availability of large-scale datasets and the effectiveness of neural models. Recent research works have attempted to extend these successes to the settings with few or no labeled data available. In this work, we introduce two approaches to improve unsupervised QA. First, we harvest lexically and syntactically divergent questions from Wikipedia to automatically construct a corpus of question-answer pairs (named as REFQA). Second, we take advantage of the QA model to extract more appropriate answers, which iteratively refines data over REFQA. We conduct experiments<sup>1</sup> on SQuAD 1.1, and NewsQA by fine-tuning BERT without access to manually annotated data. Our approach outperforms previous unsupervised approaches by a large margin and is competitive with early supervised models. We also show the effectiveness of our approach in the few-shot learning setting.
Extractive question answering aims to extract a span from the given document to answer the question. Rapid progress has been made because of the release of large-scale annotated datasets (Rajpurkar et al., 2016, 2018; Joshi et al., 2017), and well-designed neural models (Wang and Jiang, 2016; Seo et al., 2016; Yu et al., 2018). Recently, unsupervised pre-training of language models on large corpora, such as BERT (Devlin et al., 2019), has brought further performance gains.
However, the above approaches heavily rely on the availability of large-scale datasets. The collection of high-quality training data is time-consuming and requires significant resources, es
especially for new domains or languages. In order to tackle the setting in which no training data available, Lewis et al. (2019) leverage unsupervised machine translation to generate synthetic context-question-answer triples. The paragraphs are sampled from Wikipedia. NER and noun chunkers are employed to identify answer candidates. Cloze questions are first extracted from the sentences of the paragraph, and then translated into natural questions. However, there are a lot of lexical overlaps between the generated questions and the paragraph. Similar lexical and syntactic structures render the QA model tend to predict the answer just by word matching. Moreover, the answer category is limited to the named entity or noun phrase, which restricts the coverage of the learnt model.
In this work, we present two approaches to improve the quality of synthetic context-question-answer triples. First, we introduce the REFQA dataset, which harvests lexically and syntactically divergent questions from Wikipedia by using the cited documents. As shown in Figure 1, the sentence (statement) in Wikipedia and its cited documents are semantically consistent, but written with different expressions. More informative context-question-answer triples can be created by using the cited document as the context paragraph and extracting questions from the statement in Wikipedia. Second, we propose to iteratively refine data over REFQA. Given a QA model and some REFQA examples, we first filter its predicted answers with a probability threshold. Then we refine questions based on the predicted answers, and obtain the refined question-answer pairs to continue the model training. Thanks to the pretrained linguistic knowledge in the BERT-based QA model, there are more appropriate and diverse answer candidates in the filtered predictions, some of which do not appear in the candidates extracted by NER tools. We also show
*Contribution during internship at Microsoft Research.  
<sup>1</sup>The code and data are available at https://github.com/Neutralzz/RefQA.
6719
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6719-6728 July 5 - 10, 2020. ©2020 Association for Computational Linguistics
that iteratively refining the data further improves model performance.
We conduct experiments on SQuAD 1.1 (Rajpurkar et al., 2016), and NewsQA (Trischler et al., 2017). Our method yields state-of-the-art results against strong baselines in the unsupervised setting. Specifically, the proposed model achieves 71.4 F1 on the SQuAD 1.1 test set and 45.1 F1 on the NewsQA test set without using annotated data. We also evaluate our method in a few-shot learning setting. Our approach achieves 79.4 F1 on the SQuAD 1.1 dev set with only 100 labeled examples, compared to 63.0 F1 using the method of Lewis et al. (2019).
To summarize, the contributions of this paper include: i) REFQA constructing in an unsupervised manner, which contains more informative context-question-answer triples. ii) Using the QA model to iteratively refine and augment the question-answer pairs in REFQA.
Extractive Question Answering Given a document and question, the task is to predict a continuous sub-span of the document to answer the question. Extractive question answering has garnered a lot of attention over the past few years. Benchmark datasets, such as SQuAD (Rajpurkar et al., 2016, 2018), NewsQA (Trischler et al., 2017) and TriviaQA (Joshi et al., 2017), play an important role in the progress. In order to improve the performance on these benchmarks, several models have been proposed, including BiDAF (Seo et al., 2016), R-NET (Wang et al., 2017), and QANet (Yu et al., 2018). Recently, unsupervised pre-training of language models such as BERT (Devlin et al., 2019), achieves significant improvement. However, these powerful models rely on the availability of human-labeled data. Large annotated corpora for a specific domain or language are limited and expensive to construct.
Semi-Supervised QA Several semi-supervised approaches have been proposed to utilize unlabeled data. Neural question generation (QG) models are used to generate questions from unlabeled passages for training QA models (Yang et al., 2017; Zhu et al., 2019b; Alberti et al., 2019; Dong et al., 2019). However, the methods require labeled data to train the sequence-to-sequence QG model. Dhingra et al. (2018) propose to collect synthetic context-question-answer triples by gen
erating cloze-style questions from the Wikipedia summary paragraphs in an unsupervised manner.
Unsupervised QA Lewis et al. (2019) have explored the unsupervised method for QA. They create synthetic QA data in four steps. i) Sample paragraphs from the English Wikipedia. ii) Use NER or noun chunkers to extract answer candidates from the context. iii) Extract "fill-in-the-blank" cloze-style questions given the candidate answer and context. iv) Translate cloze-style questions into natural questions by an unsupervised translator. Compared with Dhingra et al. (2018), Lewis et al. (2019) attempt to generate natural questions by training an unsupervised neural machine translation (NMT) model. They train the NMT model on non-aligned corpora of natural questions and cloze questions. The unsupervised QA model of Lewis et al. (2019) achieves promising results, even outperforms early supervised models. However, their questions are generated from the sentences or sub-clauses of the same paragraphs, which may lead to a biased learning of word matching since its similar lexicons and syntactic structures. Besides, the category of answer candidates is limited to named entity or noun phrase, which restricts the coverage of the learnt QA model.
3 Harvesting REFQA from Wikipedia
In this section, we introduce REFQA, a question answering dataset constructed in an unsupervised manner. One drawback of Lewis et al. (2019) is that questions are produced from the paragraph sentence that contains the answer candidate. So there are considerable expression overlaps between generated questions and context paragraphs. In contrast, we harvest informative questions by taking advantage of Wikipedia's reference links, where lexical and syntactic differences exist between the article and its cited documents.
As shown in Figure 1, given statements in Wikipedia paragraphs and its cited documents, we use the cited documents as the context paragraphs and generate questions from the sub-clauses of statements. In order to generate question-answer pairs, we first find answer candidates that appear in both sub-clauses and context paragraphs. Next, we convert sub-clauses into the cloze questions based on the candidate answers. We then conduct cloze-to-natural-question translation by a depen
6720
WIKIPEDIA
In August 2013, Guillermo crashed a Matt Damon interview, about his upcoming movie Elysium, by promoting his own movie called "Estupido", about a stupid man, which poster had an arrow pointing towards Matt Damon.[17] At the end of the interview, Matt removed the poster, revealing on the other 'name of another Guillery'
Matt. Matt accamera, startsDuring Kimmyhis appearancewas removedinterview AfflAffleck stars iastronaut Mar
Cited Document
... In the clip, Kimmel sidekick/parking lot security guard Guillermo Rodriguez interrupted an interview Damon is giving while sitting in front of a poster for "Elysium," and propped up his own movie poster for a film called "Estupid." The sign was bright yellow, with the title in big bold letters and an arrow pointed down at Damon. ...
find answer position
Guillermo crashed a Matt Damon interview, about his upcoming movie Elysium
answer  $\downarrow$  Answer a
Cloze Question
Guillermo crashed a Matt Damon interview, about his upcoming movie [THING]
Natural Question  $q$
What his upcoming movie about Guillermo crashed a Matt Damon interview
Context  $c$
In the clip, Kimmel... interrupted an interview Damon is giving while sitting in front of a poster for "Elysium."
dency tree reconstruction algorithm. We describe the details as follows.
3.1 Context and Answer Generation
Statements in Wikipedia and its cited documents often have similar content, but are written with different expressions. Informative questions can be obtained by taking the cited document as the context paragraph, and generate questions from the statement. We crawl statements with reference links from the English Wikipedia. The cited documents are obtained by parsing the contents of reference webpages.
Given a statement and its cited document, we restrict the statement to its sub-clauses, and extract answer candidates (i.e., named entities) that appear in both of them by using a NER toolkit. We then find the answer span positions in the context paragraph. If the candidate answer appears multiple times in the context, we select the position whose surrounding context has the most overlap with the statement.
We first generate cloze questions (Lewis et al., 2019) from the sub-clauses of Wikipedia statements. Then we introduce a rule-based method to rewrite them to more natural questions, which utilizes the dependency structures.
3.2.1 Cloze Generation
Cloze questions are the statements with the answer replaced to a mask token. Following Lewis et al. (2019), we replace answers in statements
with a special mask token, which depends on its answer category<sup>2</sup>. Using the statement and the answer (with a type label PRODUCT) from Figure 1, this leaves us with the cloze question "Guillermo crashed a Matt Damon interview, about his upcoming movie [THING]".
3.2.2 Translate Clozes to Natural Questions
We perform a dependency reconstruction to generate natural questions. We move answer-related words in the dependency tree to the front of the question, since answer-related words are important. The intuition is that natural questions usually start with question words and question focus (Yao and Van Durme, 2014).
As shown in Figure 2, we apply the dependency parsing to the cloze questions, and translate them to natural questions by three steps: i) We keep the right child nodes of the answer and prune its lefts. ii) For each node in the parsing tree, if the subtree of its child node contains the answer node, we move the child node to the first child node. iii) Finally, we obtain the natural question by inorder traversal on the reconstructed tree. We apply the same rule-based mapping as Lewis et al. (2019), which replaces each answer category with the most appropriate wh* word. For example, the thing category is mapped to "What".
2We obtain the answer type labels by a NER toolkit, and group these labels to high-level answer categories, which are used as our mask tokens, e.g., PRODUCT corresponding to THING, LOC corresponding to PLACE.
6721
Figure 2: Example of translating cloze questions to natural questions. The node with light yellow color indicates that its subtree contains the answer node.
4 Iterative Data Refinement
In this section, we propose to iteratively refine data over REFQA based on the QA model. As shown in Figure 3, we use the QA model to filter REFQA data, find appropriate and diverse answer candidates, and use these answers to refine and augment REFQA examples. Filtering data can get rid of some noisy examples in REFQA, and pretrained linguistic knowledge in the BERT-based QA model finds more appropriate and diverse answers. We produce questions for the refined answers, then continue to train the QA model on the refined and filtered triples.
4.1 Initial QA Model Training
The first step of iterative data refinement is to train an initial QA model. We use the REFQA examples  $S_{I} = \{(c_{i},q_{i},a_{i})\}_{i = 1}^{N}$  to train a BERT-based QA model  $P(a|c,q)$  by maximizing:
$$
\sum_ {S _ {I}} \log P \left(a _ {i} \mid c _ {i}, q _ {i}\right) \tag {1}
$$
where the triple consists of context  $c_{i}$ , question  $q_{i}$  and answer  $a_{i}$ .
4.2 Refine Question-Answer Pairs
As shown in Figure 3, the QA model  $P(a|c,q)$  is used to refine the REFQA examples. We first conduct inference on the unseen data (denoted as  $S_U$ ), and obtain the predicted answers and their probabilities. Then we filter the predicted answers with
Figure 3: Overview of our iterative data refinement process. "QG" is the process of question generation as described in Section 3.2. We produce new training data and iteratively train the QA model.
a confidence threshold  $\tau$ :
$$
Z _ {A} = \{a _ {i} ^ {\prime} | P (a _ {i} ^ {\prime} | c _ {i}, q _ {i}) \geq \tau \} _ {(c _ {i}, q _ {i}, a _ {i}) \in S _ {U}}
$$
where  $a_{i}^{\prime}$  represents the predicted answer.
For each predicted answer  $a_{i}^{\prime}$ , if it agrees with the gold answer  $a_{i}$ , we keep the original question. For the case that  $a_{i}^{\prime} \neq a_{i}$ , we treat  $a_{i}^{\prime}$  as our new answer candidate. Besides, we use the question generator (Section 3.2) to refine the original question  $q_{i}$  to  $q_{i}^{\prime}$ .
In this step, using the QA model for filtering helps us get rid of some noisy examples. The refined question-answer pairs  $(q_i', a_i')$  can also augment the REFQA examples. The pretrained linguistic knowledge in the BERT-based QA model is supposed to find more novel answers, i.e., some candidate answers are not extracted by the NER toolkit. With the refined answer spans, we then use the question generator to produce their corresponding questions.
4.3 Iterative QA Model Training
After refining the dataset, we concatenate them with the filtered examples whose candidate answers agree with the predictions. The new training set is then used to continue to train the QA model. The training objective is defined as:
$$
\begin{array}{l} \max  \sum_ {a _ {i} ^ {\prime} \in Z _ {A}} [   \mathbb {I} (a _ {i} ^ {\prime} = a _ {i}) \log P (a _ {i} | c _ {i}, q _ {i}) \tag {2} \\ + \mathbb {I} (a _ {i} ^ {\prime} \neq a _ {i}) \log P (a _ {i} ^ {\prime} | c _ {i}, q _ {i} ^ {\prime}) ], \\ \end{array}
$$
6722
where  $\mathbb{I}(\cdot)$  is an indicator function (i.e., 1 if the condition is true).
Using the resulting QA model, we further refine question-answer pairs and repeat the training procedure. The process is repeated until the performance plateaus, or no new data available. Besides, in order to obtain more diverse answers during iterative training, we apply a decay factor  $\gamma$  for the threshold  $\tau$ . The pseudo code of iterative data refinement is presented in Algorithm 1.
We evaluate our proposed method on two widely used extractive QA datasets (Rajpurkar et al., 2016; Trischler et al., 2017). We also demonstrate the effectiveness of our approach in the few-shot learning setting.
5.1 Configuration
REFQA Construction We collect the statements with references from English Wikipedia following the procedure in (Zhu et al., 2019a). We only consider the references that are HTML pages, which results in 1.4M statement-document pairs.
In order to make sure the statement is relevant to the cited document, we tokenize the text, remove stop words and discard the examples if more than
half of the statement tokens are not in the cited document. The article length is limited to 1,000 words for cited documents. Besides, we compute ROUGE-2 (Lin, 2004) as correlation scores between statements and context. We use the score's median (0.2013) as a threshold, i.e., half of the data with lower scores are discarded. We obtain 303K remaining data to construct our REFQA.
We extract named entities as our answer candidates, using the NER toolkit of Spacy. We split the statements into sub-clauses with Berkeley Neural Parser (Kitaev and Klein, 2018). The questions are generated as in Section 3.2. We also discard sub-clauses that are less than 6 tokens, to prevent losing too much information of original sentences. Finally, we obtain 0.9M REFQA examples.
Question Answering Model We adopt BERT as the backbone of our QA model. Following (Devlin et al., 2019), we represent the question and passage as a single packed sequence. We apply a linear layer to compute the probability of each token being the start or end of an answer span. We use Adam (Kingma and Ba, 2015) as our optimizer with a learning rate of 3e-5 and a batch size of 24. The max sequence length is set to 384. We split the long document into multiple windows with a stride of 128. We use the uncased version of BERTLarge (Whole Word Masking). We evaluate on the dev set every 1000 training steps, and conduct early stopping when the performance plateaus.
Iterative Data Refinement We uniformly sample 300k data from REFQA to train the initial QA model. We split the remaining 600k data into 6 parts for iterative data refinement. For each part, we use the current QA model to refine question-answer pairs. We combine the refined data with filtered data in a 1:1 ratio to continue training the QA model. Specially, we keep the original answer if its prediction is a part of the original answer during inference. The threshold  $\tau$  is set to 0.15 for filtering the model predictions. The decay factor  $\gamma$  is set to 0.9.
We conduct evaluation on the SQuAD 1.1 (Rajpurkar et al., 2016), and the NewsQA (Trischler et al., 2017) datasets. We compare our proposed approach with previous unsupervised approaches and several supervised models. Performance is measured via the standard Exact Match (EM) and F1 metrics.
6723
Models	SQuAD 1.1	NewsQA
Dev Set	Test Set	Dev Set	Test Set
Supervised Methods
DCR (Yu et al., 2016)	62.5 / 71.2	62.5 / 71.0	- / -	- / -
mLSTM (Wang and Jiang, 2016)	64.1 / 73.9	64.7 / 73.7	34.4 / 49.6*	34.9 / 50.0*
FastQAExt (Weissenborn et al., 2017)	70.3 / 78.5	70.8 / 78.9	43.7 / 56.1	42.8 / 56.1
R-NET (Wang et al., 2017)	71.1 / 79.5	71.3 / 79.7	- / -	- / -
BERT-Large (Devlin et al., 2019)	84.2 / 91.1	85.1 / 91.8	- / -	- / -
SpanBERT (Joshi et al., 2019)	- / -	88.8 / 94.6	- / -	- / 73.6
Unsupervised Methods
Dhingra et al. (2018)†	28.4 / 35.8	- / -	- / -	- / -
Lewis et al. (2019)	- / -	44.2 / 54.7	- / -	- / -
Lewis et al. (2019)‡	45.4 / 55.6	- / -	19.6 / 28.5	17.9 / 27.0
Our REFQA	57.1 / 66.8	55.8 / 65.5	29.0 / 42.2	27.6 / 41.0
+ Iterative Data Refinement	62.5 / 72.6	61.1 / 71.4	33.6 / 46.3	32.1 / 45.1
Dhingra et al. (2018) propose to train the QA model on the cloze-style questions. Here we take the unsupervised results that re-implemented by Lewis et al. (2019) with BERT-Large. The other unsupervised QA system (Lewis et al., 2019) borrows the idea of unsupervised machine translation (Lample et al., 2017) to convert cloze questions into natural questions. For a fair comparison, we use their published data<sup>3</sup> to re-implement their approach based on BERT-Large (Whole Word Masking) model.
Table 1 shows the main results on SQuAD 1.1 and NewsQA. Training QA model on our REFQA outperforms the previous methods by a large margin. Combining with iterative data refinement, our approach achieves new state-of-the-art results in the unsupervised setting. Our QA model attains 71.4 F1 on the SQuAD 1.1 test set and 45.1 F1 on the NewsQA test set without using their annotated data, outperforming all of the previous unsupervised methods. In particular, the results are competitive with early supervised models.
We conduct ablation studies on the SQuAD 1.1 dev set, in order to better understand the contributions of different components in our method.
Table 1: Results (EM / F1) of our method, various baselines and supervised models on SQuAD 1.1, and NewsQA. “*” means results taken from Trischler et al. (2017), “†” means results taken from Lewis et al. (2019), and “‡” means our reimplementation on BERT-Large (Whole Word Masking).
Identity	Noise	UNMT	DRC
WIKI	20.8 / 30.5	36.6 / 45.6	40.5 / 49.1	26.3 / 35.7
REFQA	42.5 / 51.6	45.1 / 53.5	43.4 / 52.0	49.2 / 58.8
Table 2: Results (EM / F1) of REFQA and WIKI datasets with different cloze translation methods on the SDuAD 1.1 dev set. "DRC" is short for dependency reconstruction.
5.3.1 Effects of REFQA
We conduct experiments on REFQA and another synthetic dataset (named as WIKI). The WIKI dataset is constructed using the same method as in Lewis et al. (2019), which uses Wikipedia pages as context paragraphs for QA examples. In addition to the dependency reconstruction method (Section 3.2.2), we compare three cloze translation methods proposed in Lewis et al. (2019).
Identity Mapping generates questions by replacing the mask token in cloze questions with a relevant wh* question word.
Noise Cloze first applies a noise model, such as permutation, and word drop, as in Lample et al. (2017), and then applies the "Identity Mapping" translation.
UNMT converts cloze questions into natural questions following unsupervised neural machine translation. Here we directly use the published model of Lewis et al. (2019) for evaluation.
<sup>3</sup>https://github.com/facebookresearch/UnsupervisedQA
6724
it finished first in the Arbitron ratings in April 1990	he was sold to Colin Murphy's Lincoln City for a fee of 15,000
UNMT: Who finished it first in the ratings in April 1990?	UNMT: How much do we need Colin Murphy's Lincoln City for a fee?
DRC: Who ratings in it finished first in April 1990	DRC: How much of a fee for he was sold to Colin Murphy's Lincoln City
Table 3: Examples of generated questions using UNMT and our method. "DRC" is short for our dependency reconstruction. The blue words indicate extracted answers.
Iter.	Size	EM / F1
Initial QA Model	300k	57.1 / 66.8
Training on
Filtered Data	X	464k	57.4 / 67.1
Refined Data	X	100k	61.0 / 70.7
Refined + Filtered Data	X	200k	61.8 / 71.0
Refined Data	✓	6×15k	60.1 / 70.0
Refined + Filtered Data	✓	6×30k	62.5 / 72.6
For a fair comparison, we sample 300k training data for each dataset, and fine-tune BERT-Base for 2 epochs. As shown in Table 2, training on our REFQA achieves a consistent gain over all cloze translation methods. Moreover, our dependency reconstruction method is also favorable compared with the "Identity Mapping" method. The improvement of DRC on WIKI is smaller than on REFQA. We argue that it is because WIKI contains too many lexical overlaps, while DRC mainly focuses on providing structural diversity.
We present the generated questions of our method (DRC) and UNMT in Table 3. Most natural questions follow a similar structure: question word (what/who/how), question focus (name/money/time), question verb (is/play/take) and topic (Yao and Van Durme, 2014). Compared with UNMT, our method adjusts answer-related words in the dependency tree according to the linguistic characteristics of natural questions.
5.3.2 Effects of Data Combination