SlowGuess/NAACL_2021_OCR huggingface dataset:
shown in Table 1. Qualitatively, we found no evidence of retrieval usage in a publicly hosted ELI5 model demo by Jernite (2020).<sup>10</sup> A possible explanation for this issue is high source-reference divergence, a common problem in table-to-text generation (Wiseman et al., 2017; Tian et al., 2019). In Table 2 and Table 4, we measure the  $n$ -gram overlap of top-ranked gold validation answers (Gold Ans) with predicted retrievals. This overlap is low and similar to that of our generations, which we suspect encourages our model to ignore retrievals. A second explanation is the large amount of train-validation overlap (Section 3.2), which eliminates the need for retrieval.
Why does our model do well compared to other systems despite not using retrievals? While our model has similar capacity as the BART/RAG baselines (comparison in Appendix A.3), we hypothesize that our improvements in ROUGE-L are due to a different pretraining objective. BART is pretrained on a masked infilling task on short sequences. Instead, we pretrain our model to perform next-word prediction on long sequences from Project Gutenberg, which encourages long & fluent generations. To illustrate this length effect, in Appendix A.6 we show that truncated outputs from our model get lower ROUGE-L scores on ELI5.[11] Prior summarization literature (Sun et al., 2019) has also shown that ROUGE scores vary heavily by length. To compare the same systems on shorter length outputs, we also tried finetuning the pretrained model on Wizard of Wikipedia (Dinan et al., 2019), an unconstrained dialogue generation task with single sentence dialogues (much shorter than ELI5). As seen on the public KILT leaderboard,[12] our system has lower ROUGE-L scores than the BART / RAG baselines. Another possible explanation is issues with ROUGE-L itself, as discussed in Section 3.3.
Takeaway (better evaluation of grounding): For evaluating LFQA, it is important to run control experiments with random retrievals & measure grounding of generations in retrieval. While the KILT benchmark does attempt to measure the com
bined retrieval + generation performance via KILT RL, it does not check whether the generations actually used the retrievals. In other words, one can submit independent retrieval & generation systems, but still perform well on the combined score. This may not be an issue for short-form QA tasks like Natural Questions, since the gold answer is often exactly contained as a span in the gold retrieval. Also, as retrieval might be less important for large language models with parametric knowledge (Roberts et al., 2020), the KILT-RL strategy of simply aggregating top-1 retrieval score with ROUGE-L unfairly penalizes systems not relying on retrieval.[13]
3.2 Training / Validation Overlap
Our experiments in Section 3.1 show that model performance is mostly unchanged by conditioning generation on randomly sampled retrievals instead of predictions from C-REALM. Despite not using retrievals, we observe qualitatively that our model displays a large amount of parametric knowledge ("Faraday Cage" in Figure 1c), which is surprising since it was pretrained on novels from Project Gutenberg (not Wikipedia). In this section, we discover that a major reason for ignoring retrievals is the large amount of train / validation overlap in ELI5. While Fan et al. (2019) attempted to fix this issue through TF-IDF overlap, this method is insufficient to identify all question paraphrases, as we find significant overlap between the training set and the KILT validation set of ELI5.[14] ELI5 is not the only dataset with substantial train / test overlap: Lewis et al. (2020d) identify similar issues with short-form QA datasets like Natural Questions.
Finding similar questions & measuring overlap:
We use our retriever C-REALM to retrieve similar questions from the training set, since it has learned to map questions to a feature-rich embedding space. For each validation question, we retrieve the 7 most similar training set questions. We use both human and automatic evaluation to calculate the amount of overlap. For human evaluation, we show annotators on Amazon Mechanical Turk<sup>15</sup> a validation set question and a retrieved training set question,
10https://huggingface.co/qa 11While we do not have access to generations from baselines on the KILT leaderboard, example generations from the demo of the BART model in Jernite (2020) are significantly shorter (59 words avg.) than our generations (187 words avg.). 12https://eval.ai/web/challenges/challenge-page/689/leaderboard/1909
\(^{13}\)Another issue of KILT-RL is ignoring non top-1 retrievals, penalizing models using multiple retrievals together in context. \({ }^{14}\)The ELI5 demo from Jernite (2020) also retrieves the top-1 similar training set question. Qualitatively, we found many validation examples had near-identical train paraphrases. \({ }^{15}\)We pay workers 4 cents per question pair (\(8 - 12 / hr). We only hire workers from USA, UK and Australia with a \(95 \%\) or higher approval rating and at least 1000 approved HITs.
qns with at least one train set paraphrase	81%
qns with at least one train set topically similar	100%
% of all pairs marked paraphrases	39.5%
% of all pairs marked topically similar	47.8%
% of all pairs marked as non-paraphrases	12.7%
and ask them to annotate the pair as 0: No paraphrase relationship; 1: on similar topics, but different questions; 2: approximately the same question (an adaptation of the paraphrase evaluation of Kok and Brockett, 2010). We take 300 validation set questions and ask three crowd-workers to rate them against retrieved training questions on this scale, and consider the label with majority rating. To improve quality, we manually verify their annotations.
Table 5 shows that  $81\%$  of validation set questions have at least one paraphrase in the training set, while all annotated questions have at least one topically similar question in the training set, which indicates substantial training / validation overlap. The experiment had "fair agreement" with a Fleiss  $\kappa$  of 0.29 (Fleiss, 1971; Landis and Koch, 1977).
As manually annotating question overlap can be expensive and time-consuming, we also experiment with automatic overlap detection methods. In particular, we use a RoBERTa-large binary classifier (Liu et al., 2019) fine-tuned on the Quora Question Paraphrase (QQP) dataset (Iyer et al., 2017) from the GLUE benchmark (Wang et al., 2019). For  $43.6\%$  of the ELI5 validation set, this classifier marked at least one retrieved question as a paraphrase ( $46\%$  for the 300 questions we annotated). Qualitatively, we notice that this classifier often mis-classifies retrieved questions that are valid paraphrases but exhibit significant lexical or syntactic divergence. This observation, along with the smaller fraction of valid paraphrases in the QQP training set ( $37\%$ ), partially explains the gap between automatic & human evaluations.
Using retrieved QA for generation: Since ELI5 contains significant amount of overlap between the training and validation sets, a system can simply copy the answers of retrieved training set questions instead of actually doing generation. Table 7 shows that by using the longest answer within the top-  $K$  retrieved questions, we outperform two prior systems (RAG, BART + DPR) that use retrieval-augmented generation. As an upper
Table 5: A human evaluation measuring the amount of overlap between validation set questions (qns) and retrieved questions from the training set.
Split	Retrieval	Generation
RPrec	R@5	F1	R-L
QQP classifier (1.5k examples)
overlap (43.6%)	17.0	25.8	26.0	24.6
not overlap (56.4%)	10.4	17.7	25.2	24.2
AMT evaluation (300 examples)
overlap (81%)	14.0	20.0	25.0	24.3
not overlap (19%)	5.3	17.9	24.5	24.8
Table 6: ELI5 performance difference (for the  $p = 0.6$  model) between subsets of validation QA having a question paraphrase (overlap) and not having a question paraphrase (not overlap) in the training set. We see the overlap subset has much better retrieval performance and slightly better generation performance.
bound, we also consider a system which uses the best possible answer to retrieved training set questions in terms of ROUGE-L (best top-K train answer). This system gets 28.5 ROUGE-L, outperforming all others.
ELI5 performance on overlapping QA: Finally, we measure the performance difference between validation questions that overlap with the training set vs. those that do not. Since we only have human annotations for 300 questions (the no-overlap subset has only 53 samples), we present this analysis using the QQP classifier's outputs as well. In Table 6, we notice large differences of 6.6 RPrec, 8.1 R@5 in retrieval performance favoring the overlap subset, but only a small generation score gain of 0.8 F1, 0.4 R-L (which may be misleading as discussed in Section 3.3).
Takeaway (careful held-out curation): Based on our findings, we suggest that more careful dataset curation for LFQA tasks is needed to prevent duplicates. While we acknowledge the efforts of Fan et al. (2019) to fix this issue, we also suggest alternative methods to control overlap and focus on evaluating generalization in held-out sets: (1) automatically retrieving paraphrases and then running human validation to eliminate them; or (2) holding out entire genres or domains to reduce the possibility of overlap — for example, keeping Q/A on Sports only in the held-out sets. Note that simply pruning the existing splits using these criteria will significantly reduce the size of the held-out datasets; so we suggest re-splitting the train/Validation/test splits from the entire pool of collected questions.
3.3 ROUGE-L Bounds on EL15 Performance
We have seen that simply copying the answer of a close question paraphrase from the training set achieves 28.5 ROUGE-L with an optimal selection among retrieved questions and outperforming all computational models. But how "good" is this absolute number? What are some suitable upper & lower bounds to ROUGE-L scores on ELI5? Is ROUGE-L an informative metric for LFQA?
Lower bounds are trivial baselines used to test the vulnerability of datasets or metrics to simple heuristic strategies that do not actually perform the task. Recent examples include hypothesis-only baselines for natural language inference (Gururangan et al., 2018) and passage-only baselines for reading comprehension (Kaushik and Lipton, 2018). We evaluate two ROUGE-L lower bounds on ELI5:
(1) copy the question 5 times and concatenate, as longer outputs boost ROUGE-L (Appendix A.6);
(2) retrieve a random training set answer.
Our first baseline contains entities often present in the gold answer, but without actually answering the question. Our second baseline follows the "style" of an answer but is completely off-topic.
As an upper bound, we estimate the ROUGE-L of gold answers themselves. On an average, there are 12 gold answers per question, so we measure the ROUGE-L of the longest gold answer with respect to the other gold answers. We also measure the maximum pairwise ROUGE-L between two gold answers for the same question.[16] We only calculate upper bounds for the validation set, since the gold answers of the KILT test set are hidden.
Lower bounds beat prior work, upper bounds have low ROUGE-L: We compare our bounds with actual retrieval augmented generation systems in Table 7. Both our lower bounds (random training answer, copy input) are quite competitive, outperforming RAG (Lewis et al., 2020c) and performing close to BART + DPR (Petroni et al., 2020) without actually answering the question! This shows that ROUGE-L is fairly sensitive to simply copying entities from the question
Scheme	Validation	Test
F1	R-L	F1	R-L
random train answer (↓)	17.8	16.2	17.1	15.5
copy input (↓)	16.6	20.0	14.8	16.9
RAG (2020c)	17.2	16.1	14.5	14.1
BART + DPR (2020)	18.8	18.5	17.9	17.4
longest top-1 train answer	25.2	20.7	21.6	18.7
longest top-7 train answer	26.9	21.1	22.0	18.5
RT + c-REALM (ours)	25.6	24.4	22.9	23.2
best top-1 train answer (↑)	25.9	22.4	-	-
best top-7 train answer (↑)	31.5	28.5	-	-
longest gold answer (↑)	26.7	21.2	-	-
best gold answer (↑)	29.5	26.2	-	-
Table 7: Upper (↑) and lower (↓) bounds to performance on ELI5. Lower bounds have been submitted to the public KILT leaderboard, as "Metrics Test".
as well as stylistic properties of ELI5. On the other hand, upper bounds (longest gold answer) perform worse than our system (21.2 vs 24.4). Suspecting that this result is misleading, we run another human A/B test by showing volunteers a question and asking them to choose between answers generated by our system and the longest gold answer, shuffled at random.[17] As seen in Table 3, the majority of humans prefer the gold reference answers vs generations (68% vs 14% for  $p = 0.6$ ). In interviews with human annotators after completing the task, they reported that both answers were often fluent and stylistically similar, but one eventually veered off-topic.
Takeaway (better automatic metrics needed): Our experiments demonstrate that computing the ROUGE-L of generations against gold answers is not a meaningful way to evaluate LFQA systems, since it is not selective enough to differentiate between valid/invalid answers. There is a very small margin of improvement between trivial lower bounds and strong upper bounds, with the absolute scores of upper bounds being quite low. We suspect this is due to the long length of answers and fairly unconstrained and large output space. The ELI5 dataset has several open-ended questions with many plausible answers (like What causes traffic?), often involving analogies. A possible fix is a sentence-level evaluation and then aggregating scores across generated sentences, but appropriate penalties are needed for lack of diversity (Zhu et al., 2018) and short lengths. Other possible fixes
<sup>16</sup>Note that different gold answers were not written independently as Reddit users writing answers can read existing answers and may want to provide a non-overlapping perspective. Due to the high train-valid overlap, the best top-7 retrieved answer could be a better upper bound since it is from another Reddit post (and performs better than best gold answer).
$^{17}$ Human A/B testing details in Appendix A.5.
include learning task-specific metrics to measure semantic overlap (Sellam et al., 2020) or metrics to check factual correctness (Zhang et al., 2020) and faithfulness to input (Wang et al., 2020; Durmus et al., 2020; Zhou et al., 2020). Ultimately, all automatic metrics have their limitations, and human evaluation is necessary (Celikyilmaz et al., 2020).
3.4 Difficulty of Human Evaluation
To better understand the inherent difficulty of evaluation in ELI5, we interviewed human annotators (of Table 3) and found two challenges:
(1) Unfamiliarity with question topics: While most annotators found the Q/A interesting, they were often unfamiliar with the technical topics discussed in the questions. This made it hard for them to assess answer correctness. The ELI5 dataset has questions in a wide variety of topics (History, Politics, Biology etc.), while most annotators were Computer Science graduate students. While we did allow annotators to use Wikipedia, they mentioned domain-experts will be better judges of factual correctness of answers.
(2) Length of Answers: Annotators mentioned the paragraph-long length of answers made the task quite challenging. Annotators reported taking an average of 2 minutes per answer pair, many of which required careful thought & concentration. This was especially difficult when only part of the answer was correct and the rest had contradictions or repetitions, a common theme in our generations.
Takeaway: Human evaluation is challenging but necessary for evaluating LFQA. Crowd-workers are unlikely to spend time reading & analyzing long text (Akoury et al., 2020). Hence, it is imperative to design simpler evaluations. One effort in this direction is Dugan et al. (2020), who reveal one generated sentence at a time and estimate system quality based on the number of sentences which fooled humans. Another promising direction is extrinsic evaluation (Celikylmaz et al., 2020) where humans actually interact with systems in real-world scenarios such as the Alexa Prize (Ram et al., 2018) or STORIUM (Akoury et al., 2020).
We present a "retrieval augmented" generation system that achieves state-of-the-art performance on
the ELI5 long-form question answering dataset. However, an in-depth analysis reveals several issues not only with our model, but also with the ELI5 dataset & evaluation metrics. We hope that the community works towards solving these issues so that we can climb the right hills and make meaningful progress on this important task.
First and foremost, we thank the twenty people who volunteered to help out with the human annotation experiments. We are very grateful to Vidhisha Balachandran, Niki Parmar, and Ashish Vaswani for weekly meetings discussing progress and the REALM team (Kenton Lee, Kelvin Guu, Ming-Wei Chang and Zora Tung) for help with their codebase and several useful discussions which helped us improve our experiments. We are grateful to Tu Vu for help with the QQP classifier. We thank Jules Gagnon-Marchand and Sewon Min for suggesting useful experiments on checking ROUGE-L bounds. Finally, we thank Shufan Wang, Andrew Drozdov, Nader Akoury, Andrew McCallum, Rajarshi Das, and the rest of the UMass NLP group for helpful discussions and suggestions at various stages in the project. This work was primarily done during KK's internship at Google Brain, mentored by AR. MI and KK are supported by award IIS-1955567 from the National Science Foundation (NSF).
Our system faces a similar set of issues as most modern text generation technology, like fabrication of facts (Zellers et al., 2019), potential for misuse (Brown et al., 2020) and reflecting biases prevalent on Reddit (the ELI5 dataset has been built using the r/ELI5 subreddit). In our work, we attempted to make text generators more factually grounded by conditioning generations on retrieved Wikipedia articles, hoping to reduce fact fabrication. Unfortunately, a thorough analysis (Section 3.1) has revealed that our system is still not grounding its generations in retrievals, and we have recommended the design of better metrics to measure factual correctness to tackle this issue.
Our final models were trained using 64 Google Cloud TPUs for a total of 32 hours. As mentioned in the Google 2019 environment report, $^{18}$
$^{18}$ https://www.gstatic.com/gumdrop/sustainability/google-2019-environmental-report.pdf
"TPUs are highly efficient chips which have been specifically designed for machine learning applications". These accelerators run on Google Cloud, which has "matched  $100\%$  of its electricity consumption with renewable energy purchases, and has committed to fully decarbonize its electricity supply by 2030" (https://cloud.google.com/sustainability). More details on training time are provided in Appendix A.1.
Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, Michael Isard, et al. 2016. Tensorflow: A system for large-scale machine learning. In 12th {USENIX} symposium on operating systems design and implementation (\{OSDI\} 16), pages 265-283.
Nader Akoury, Shufan Wang, Josh Whiting, Stephen Hood, Nanyun Peng, and Mohit Iyyer. 2020. Storium: A dataset and evaluation platform for machine-in-the-loop story generation. In Proceedings of Empirical Methods in Natural Language Processing.
Tom B Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems.
Asli Celikyilmaz, Elizabeth Clark, and Jianfeng Gao. 2020. Evaluation of text generation: A survey. arXiv preprint arXiv:2006.14799.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In Proceedings of the International Conference of Machine Learning.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Conference of the North American Chapter of the Association for Computational Linguistics.
Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2019. Wizard of wikipedia: Knowledge-powered conversational agents. In International Conference on Learning Representations.
Liam Dugan, Daphne Ippolito, Arun Kirubarajan, and Chris Callison-Burch. 2020. RoFT: A tool for evaluating human detection of machine-generated text. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations. Association for Computational Linguistics.
Esin Durmus, He He, and Mona Diab. 2020. Feqa: A question answering evaluation framework for faithfulness assessment in abstractive summarization. In Proceedings of the Association for Computational Linguistics.
Angela Fan, Yacine Jernite, Ethan Perez, David Grangier, Jason Weston, and Michael Auli. 2019. ELI5: Long form question answering. In Proceedings of the Association for Computational Linguistics.
Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5):378.
Ruiqi Guo, Philip Sun, Erik Lindgren, Quan Geng, David Simcha, Felix Chern, and Sanjiv Kumar. 2020. Accelerating large-scale inference with anisotropic vector quantization. In Proceedings of the International Conference of Machine Learning.
Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A Smith. 2018. Annotation artifacts in natural language inference data. In Conference of the North American Chapter of the Association for Computational Linguistics.
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Papat, and Ming-Wei Chang. 2020. REALM: Retrieval-augmented language model pre-training. In Proceedings of the International Conference of Machine Learning.
Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text degeneration. In International Conference on Learning Representations.
Shankar Iyer, Nikhil Dandekar, and Kornél Csernai. 2017. First quora dataset release: Question pairs.
Gautier Izacard and Edouard Grave. 2020. Leveraging passage retrieval with generative models for open domain question answering. arXiv preprint arXiv:2007.01282.
Yacine Jernite. 2020. Explain anything like i'm five: A model for open domain long form question answering. https://yjernite.github.io/lfqa.html.
Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open-domain question answering. In Proceedings of Empirical Methods in Natural Language Processing.
Divyansh Kaushik and Zachary C Lipton. 2018. How much reading does reading comprehension require? a critical investigation of popular benchmarks. In Proceedings of Empirical Methods in Natural Language Processing.
Stanley Kok and Chris Brockett. 2010. Hitting the right paraphrases in good time. In *Conference of the North American Chapter of the Association for Computational Linguistics*.