SlowGuess/ACL_2020_OCR huggingface dataset:
Clark and Gardner (2018) for TriviaQA-Wiki $^6$ , we only keep the top 8 ranked paragraphs up to 400 tokens for each document-question pair for both training and evaluation. Following Min et al. (2019), for NarrativeQA we define the possible answer string sets  $\mathcal{A}$  using Rouge-L (Lin, 2004) similarity with crowdsourced abstractive answer strings. We use identical data preprocessing and the evaluation script provided by the authors.
In this work, we use the BERT-base model for text encoding and train our model with the default configuration as described in (Devlin et al., 2019), fine-tuning all parameters. We fine-tune for 3 epochs on TriviaQA and 2 epochs on NarrativeQA.
5.2 Optimization and Inference for Latent Variable Models
Here we look at the cross product of optimization (HardEM vs MML) and inference (Max vs Sum) for all distant supervision assumptions that result in models with latent variables. We therefore exclude H1 and look at the other two hypotheses, H2 and H3, each coupled with a span-based (Span) or position-based (Pos) formulation and a paragraph-level (P) or a document level (D) probability space. The method used in Min et al. (2019) corresponds to span-based H2-P with HardEM training and Max inference. The results are shown in Fig. 3.
First, we observe that inference with Sum leads to significantly better results on TriviaQA under H2-P and H2-D, and slight improvement under H3-D. On NarrativeQA, inference with Max is better. We attribute this to the fact that correct answers often have multiple relevant mentions for TriviaQA (also see §5.6), whereas for NarrativeQA this is rarely the case. Thus, inference with Sum in NarrativeQA could potentially boost the probability of irrelevant frequent strings.
Consistent with (Min et al., 2019), we observe that span-based HardEM works better than span-based MML under H2-P, with a larger advantage on NarrativeQA than on TriviaQA. However, under H2-D and H3-D, span-based MML performs consistently better than span-based HardEM. For position-based objectives, MML is consistently better than HardEM (potentially because HardEM may decide to place its probability mass on begin-end position combinations that do not contain mentions of strings in  $\mathcal{A}$ ). Finally, it can be ob
For inference with marginal  $(\sum)$  scoring, we use an approximate scheme where we only aggregate probabilities of candidates strings generated from a 20-best list of begin/end answer positions for each paragraph.
$^{6}$ https://github.com/allenai/document-ga
5662
(a)TriviaQA F1
(b) NarrativeQA Rouge-L
Figure 3: Comparison of different optimization and inference choices grouped by distant supervision hypothesis based on dev set results for TriviaQA and NarrativeQA.
served that under each distant supervision hypothesis/probability space combination, the position-based MML is always the best among the four objectives. Position-based objectives may perform better due to the independence assumptions for begin/end positions of the model we use and future work may arrive at different conclusions if position dependencies are integrated. Based on this thorough exploration, we focus on experimenting with position-based objectives with MML for the rest of this paper.
5.3 Probability Space and Distant Supervision Assumptions
In this subsection, we compare probability space and distant supervision assumptions. Table 3 shows the dev set results, where the upper section compares paragraph-level models (H1-P, H2-P), and the lower section compares document-level models (H1-D, H2-D, H3-D). The performance of models with both Max and Sum inference is shown. We report F1 and Exact Match (EM) scores for TriviaQA, and Rouge-L scores for NarrativeQA.
For TriviaQA, H3-D achieves significantly bet-
Objective	Infer	TriviaQA	NarrativeQA
F1	EM	Rouge-L
Paragraph-level Models
H1-P	Max	67.9	63.3	55.3
Sum	70.4	66.0	53.6
H2-P	Max	71.9	67.7	59.2
Sum	73.0	69.0	57.8
Document-level Models
H1-D	Max	55.8	51.0	59.4
Sum	65.2	61.2	59.1
H2-D	Max	70.3	66.2	60.1
Sum	72.4	68.4	59.9
H3-D	Max	75.1	70.6	59.1
Sum	75.3	70.8	59.2
Table 3: Comparison of distant supervision hypotheses using MML-Pos objectives on TriviaQA and NarrativeQA dev sets.
ter results than other formulations. Only H3-D is capable of "cleaning" noise from positive paragraphs that don't have a correct answer (e.g. paragraph two in Fig. 1), by deciding which  $\mathcal{A}$ -consistent mention to trust. The paragraph-level models H1-P and H2-P outperform their corresponding document-level counterparts H1-D and H2-D. This may be due to the fact that without H3, and without predicting NULL, D models do not learn to detect irrelevant paragraphs.
Unlike for TriviaQA, H2-D models achieve the best performance for NarrativeQA. We hypothesize this is due to the fact that positive paragraphs that don't have a correct answer are very rare in NarrativeQA (as summaries are relatively short and answer strings are human-annotated for the specific documents). Therefore, H3 is not needed to clean noisy supervision, and it is not useful since it also leads to a reduction in the number of positive examples (coverage) for the model. Here, document-level models always improve over their paragraph counterparts, by learning to calibrate paragraphs directly against each other.
5.4 Multi-Objective Formulations and Clean Supervision
Here we study two methods to further improve weakly supervised QA models. First, we combine two distant supervision objectives in a multitask manner, i.e. H2-P and H3-D for TriviaQA, and H2-P and H2-D for NarrativeQA, chosen based on the results in §5.3. H2 objectives have higher coverage than H3 while being more susceptible
5663
Objective	Clean	Infer	TriviaQA	NarrativeQA
F1	EM	Rouge-L
Single-objective
Par	X	Max	71.9	67.7	59.2
Sum	73.0	69.0	57.8
✓	Max	74.2	70.1	61.7
Sum	74.9	70.9	61.7
Doc	X	Max	75.1	70.6	60.1
Sum	75.3	70.8	59.9
✓	Max	75.5	70.8	62.8
Sum	75.5	70.9	62.9
Multi-objective
Par + Doc	X	Max	75.6	71.2	60.5
Sum	75.9	71.6	60.5
✓	Max	75.8	71.2	63.0
Sum	76.2	71.7	63.1
to noise. Paragraph-level models have the advantage of learning to score irrelevant paragraphs (via NULL outcomes). Note that we use the same parameters for the two objectives and the multi-objective formulation does not have more parameters and is no less efficient than the individual models. Second, we use external clean supervision from SQUAD 2.0 (Rajpurkar et al., 2018) to train the BERT-based QA model for 2 epochs. This model matches the  $\mathbb{P}$  probability space and is able to detect both NULL and extractive answer spans. The resulting network is used to initialize the models for TriviaQA and NarrativeQA. The results are shown in Table 4.
It is not surprising that using external clean supervision improves model performance (e.g. (Min et al., 2017)). We note that, interestingly, this external supervision narrows the performance gap between paragraph-level and document-level models, and reduces the difference between the two inference methods.
Compared with their single-objective components, multi-objective formulations improve performance on both TriviaQA and NarrativeQA.
5.5 Test Set Evaluation
Table 5 reports test set results on TriviaQA and NarrativeQA for our best models, in comparison to recent state-of-art (SOTA) models. For TriviaQA, we report F1 and EM scores on the full test set and the verified subset. For NarrativeQA, Rouge
Table 4: Dev set results comparing multi-objectives and clean supervision.  $\checkmark$  indicates the QA model is pre-trained on SQUAD.
TriviaQA Wiki
Full	Verified
F1	EM	F1	EM
Ours (H2-P+H3-D)	76.3	72.1	85.5	82.2
w/o SQUAD	75.7	71.6	83.6	79.6
(Wang et al., 2018b)	71.4	66.6	78.7	74.8
(Clark and Gardner, 2018)	68.9	64.0	72.9	68.0
(Min et al., 2019)	67.1	-	-	-
NarrativeQA Summary
Rouge-L
Ours (H2-P+H2-D)	62.9
w/o SQUAD	60.5
(Nishida et al., 2019)	59.9
w/o external data	54.7
(Min et al., 2019)	58.8
Table 5: Test set results on TriviaQA Wiki and NarrativeQA Summaries. "w/o SQUAD" refers to our best model without pretraining on SQUAD 2.0. "w/o external data" refers to the model from (Nishida et al., 2019) without using MS MARCO data (Bajaj et al., 2018).
L scores are reported.
Compared to recent TriviaQA SOTA (Wang et al., 2018b), our best models achieve 4.9 F1 and 5.5 EM improvement on the full test set, and 6.8 F1 and 7.4 EM improvement on the verified subset. On the NarrativeQA test set, we improve Rouge-L by 3.0 over (Nishida et al., 2019). The large improvement, even without additional fully labeled data, demonstrates the importance of selecting an appropriate probability space and interpreting the distant-supervision in a way cognizant of the properties of the data, as well as selecting a strong optimization and inference method. With external fully labeled data to initialize the model, performance is further significantly improved.
In this subsection, we carry out analyses to study the relative performance of paragraph-level and document-level models, depending on the size of answer string set  $|\mathcal{A}|$  and the number of  $\mathcal{A}$ -consistent spans, which are hypothesized to correlate with label noise. We use the TriviaQA dev set and the best performing models, i.e. H2-P and H3-D with Sum inference.
We categorize examples based on the size of their answer string set,  $|\mathcal{A}|$ , and the size of their corresponding set of  $\mathcal{A}$ -consistent spans,  $|\mathcal{I}|$ . Specifically, we divide the data into 4 subsets and
5664
Subset	|A|	|I|	Size	H2-P	H3-D	Δ
Qss	= 1	≤ 5	2585	66.8	67.4	0.6
Qls	> 1	≤ 5	853	68.7	70.1	1.4
Qsl	= 1	> 5	1149	82.0	84.9	2.9
Qll	> 1	> 5	3034	86.3	88.4	2.1
Table 6: F1 scores on 4 subsets of TriviaQA dev, grouped by the size of their answer string sets  $\mathcal{A}$  and corresponding set of possible mentions  $\mathcal{I}$ .  $\Delta$  indicates the improvement from H2-P to H3-D.
report performance separately on each subset, as shown in Table 6. In general, we expect  $\mathcal{Q}^{sl}$  and  $\mathcal{Q}^{ll}$  to be noisier due to the larger  $\mathcal{I}$ , where  $\mathcal{Q}^{sl}$  potentially includes many irrelevant mentions while  $\mathcal{Q}^{ll}$  likely contains more incorrect answer strings (false aliases). We can observe that the improvement is more significant for these noisier subsets, suggesting document-level modeling is crucial for handling both types of label noise.
Distant supervision has been successfully used for decades for information extraction tasks such as entity tagging and relation extraction (Craven and Kumlien, 1999; Mintz et al., 2009). Several ways have been proposed to learn with DS, e.g., multi-label multi-instance learning (Surdeanu et al., 2012), assuming at least one supporting evidence (Hoffmann et al., 2011), integration of label-specific priors (Ritter et al., 2013), and adaptation to shifted label distributions (Ye et al., 2019).
Recent work has started to explore distant supervision to scale up QA systems, particularly for open-domain QA where the evidence has to be retrieved rather than given as input. Reading comprehension (RC) with evidence retrieved from information retrieval systems establishes a weakly-supervised QA setting due to the noise in the heuristics-based span labels (Chen et al., 2017; Joshi et al., 2017; Dunn et al., 2017; Dhingra et al., 2017). One line of work jointly learns RC and evidence ranking using either a pipeline system (Wang et al., 2018a; Lee et al., 2018; Kratzwald and Feuerriegel, 2018) or an end-to-end model (Lee et al., 2019).
Another line of work focuses on improving distantly-supervised RC models by developing learning methods and model architectures that can better use noisy labels. Clark and Gardner (2018) propose a paragraph-pair ranking objective, which has components of both our H2-P and H3-D
position-based formulations. They don't explore multiple inference methods or combinations of objectives and use less powerful representations. In (Lin et al., 2018), a coarse-to-fine model is proposed to handle label noise by aggregating information from relevant paragraphs and then extracting answers from selected ones. Min et al. (2019) propose a hard EM learning scheme which we included in our experimental evaluation.
Our work focuses on examining probabilistic assumptions for document-level extractive QA. We provide a unified view of multiple methods in terms of their probability space and distant supervision assumptions and evaluate the impact of their components in combination with optimization and inference methods. To the best of our knowledge, the three DS hypotheses along with position and span-based interpretations have not been formalized and experimentally compared on multiple datasets. In addition, the multi-objective formulation is new.
In this paper, we demonstrated that the choice of probability space and interpretation of the distant supervision signal for document-level QA have a large impact, and that they interact. Depending on the properties of the data, different configurations are best, and a combined multi-objective formulation can reap the benefits of its constituents.
A future direction is to extend this work to question answering tasks that require reasoning over multiple documents, e.g., open-domain QA. In addition, the findings may generalize to other tasks, e.g., corpus-level distantly-supervised relation extraction.
Some of the ideas in this work originated from Hao Cheng's internship with Google Research. We would like to thank Ankur Parikh, Michael Collins, and William Cohen for discussion and detailed feedback on this work, as well as other members from the Google Research Language team and the anonymous reviewers for valuable suggestions. We would also like to thank Sewon Min for generously sharing the processed data and evaluation script for NarrativeQA.
Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang. 2018. MS MARCO: A human generated machine reading comprehension dataset. CoRR, abs/1611.09268.
Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to answer open-domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1870-1879. Association for Computational Linguistics.
Christopher Clark and Matt Gardner. 2018. Simple and effective multi-paragraph reading comprehension. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 845–855. Association for Computational Linguistics.
Mark Craven and Johan Kumlien. 1999. Constructing biological knowledge bases by extracting information from text sources. In Proceedings of the Seventh International Conference on Intelligent Systems for Molecular Biology.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186. Association for Computational Linguistics.
Bhuwan Dhingra, Kathryn Mazaitis, and William W. Cohen. 2017. Quasar: Datasets for question answering by search and reading. CoRR, abs/1707.03904.
Matthew Dunn, Levent Sagun, Mike Higgins, V. Ugur Güney, Volkan Cirik, and Kyunghyun Cho. 2017. Searchqa: A new q&a dataset augmented with context from a search engine. CoRR, abs/1704.05179.
Raphael Hoffmann, Congle Zhang, Xiao Ling, Luke Zettlemoyer, and Daniel S. Weld. 2011. Knowledge-based weak supervision for information extraction of overlapping relations. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 541-550. Association for Computational Linguistics.
Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601-1611. Association for Computational Linguistics.
Rudolf Kadlec, Martin Schmid, Ondrej Bajgar, and Jan Kleindienst. 2016. Text understanding with the attention sum reader network. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 908-918. Association for Computational Linguistics.
Tomáš Kocisky, Jonathan Schwarz, Phil Blunsom, Chris Dyer, Karl Moritz Hermann, Gábor Melis, and Edward Grefenstette. 2018. The NarrativeQA reading comprehension challenge. Transactions of the Association for Computational Linguistics, 6:317-328.
Bernhard Kratzwald and Stefan Feuerriegel. 2018. Adaptive document retrieval for deep question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 576-581. Association for Computational Linguistics.
Jinhyuk Lee, Seongjun Yun, Hyunjae Kim, Miyoung Ko, and Jaewoo Kang. 2018. Ranking paragraphs for improving answer recall in open-domain question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 565-569. Association for Computational Linguistics.
Kenton Lee, Ming-Wei Chang, and Kristina Toutanova. 2019. Latent retrieval for weakly supervised open domain question answering. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6086–6096. Association for Computational Linguistics.
Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74-81. Association for Computational Linguistics.
Yankai Lin, Haozhe Ji, Zhiyuan Liu, and Maosong Sun. 2018. Denoising distantly supervised open-domain question answering. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1736-1745.
Sewon Min, Danqi Chen, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2019. A discrete hard EM approach for weakly supervised question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2844-2857. Association for Computational Linguistics.
Sewon Min, Minjoon Seo, and Hannaneh Hajishirzi. 2017. Question answering through transfer learning from large fine-grained supervision data. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 510-517. Association for Computational Linguistics.
Mike Mintz, Steven Bills, Rion Snow, and Daniel Jurafsky. 2009. Distant supervision for relation extraction without labeled data. In Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP, pages 1003-1011. Association for Computational Linguistics.
Kyosuke Nishida, Itsumi Saito, Kosuke Nishida, Kazutoshi Shinoda, Atsushi Otsuka, Hisako Asano, and Junji Tomita. 2019. Multi-style generative reading comprehension. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 2273-2284. Association for Computational Linguistics.
Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don't know: Unanswerable questions for SQuAD. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 784-789. Association for Computational Linguistics.
Alan Ritter, Luke Zettlemoyer, Oren Etzioni, et al. 2013. Modeling missing data in distant supervision for information extraction. Transactions of the Association for Computational Linguistics, 1:367-378.
Mihai Surdeanu, Julie Tibshirani, Ramesh Nallapati, and Christopher D. Manning. 2012. Multi-instance multi-label learning for relation extraction. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL '12, pages 455-465, Stroudsburg, PA, USA. Association for Computational Linguistics.
Shuohang Wang, Mo Yu, Xiaoxiao Guo, Zhiguo Wang, Tim Klinger, Wei Zhang, Shiyu Chang, Gerald Tesauro, Bowen Zhou, and Jing Jiang. 2018a. R3: Reinforced reader-ranker for open-domain question answering. In AAAI Conference on Artificial Intelligence.
Wei Wang, Ming Yan, and Chen Wu. 2018b. Multigranularity hierarchical attention fusion networks for reading comprehension and question answering. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1705-1714. Association for Computational Linguistics.
Qinyuan Ye, Liyuan Liu, Maosen Zhang, and Xiang Ren. 2019. Looking beyond label noise: Shifted label distribution matters in distantly supervised relation extraction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3839-3848. Association for Computational Linguistics.