Knowledge Corpus Error in Question Answering
Authors: Yejoon Lee
Philhoon Oh
James Thorne
Venue: EMNLP 2023 Findings (2023)
Keywords: large language models
open-domain question answering
retrieval
qa
odqa
TL;DR: We study how generated contexts may be more helpful than retrieved contexts in question answering

Abstract: Recent works in open-domain question answering (QA) have explored generating context passages from large language models (LLMs), replacing the traditional retrieval step in the QA pipeline. However, it is not well understood why generated passages can be more effective than retrieved ones. This study revisits the conventional formulation of QA and introduces the concept of $\textit{knowledge corpus error}$. This error arises when the knowledge corpus used for retrieval is only a subset of the entire string space, potentially excluding more helpful passages that exist outside the corpus. LLMs may mitigate this shortcoming by generating passages in a larger space. We come up with an experiment of paraphrasing human-annotated gold context using LLMs to observe knowledge corpus error empirically. Our results across three QA benchmarks reveal an increased performance (10\% - 13\%) when using paraphrased passage, indicating a signal for the existence of knowledge corpus error.

Knowledge Corpus Error in Question Answering
Yejoon Lee" Seoul National University leeyejoonesnu.ac..
Philhoon Oh
James Thorne KAIST AI
KAIST AI
philhoonohekaist..acc kr
thorne@kaist.cm ac.
Abstract
Recent works in open-domain question answer- ing (QA) have explored generating context pas- sages from large language models (LL.Ms), re- placing the traditional retrieval step the QA pipeline. However, is not well understood why generated passages can be more effective than retrieved ones. This study revisits the con- ventional formulation of QA and introduces the concept of knowledge corpus error. This error arises when the knowledge corpus used for re- trieval is only subset of the entire string space, potentially excluding more helpful passages that exist outside the corpus. LLMs may miti- gate this shortcoming by generating passages in a larger space. We come up with an experiment of paraphrasing numan-annotated gold context using LLMs to observe knowledge corpus error empirically. Our results across three QA bench- marks reveal increased performance (10% 13%) when using paraphrased passage, indi- cating signal the existence of knowledge corpus error.
knowledge corpus emor
paraphrased
gold
retrieved
retrieval crror
Figure I: Illustration of our framework. Each dot repre- sents a passage. Retrieval error refers to an error from failing retrieve the gold passage. Knowledge corpus enoor refers to an error from discarding better passages outside the knowledge corpus, c.g., Wikipedia, which is inevitable in any retrieval setting. See for details.
retrieved passages. These recent advancements lack robust links to prior research in QA posing a challenge to a holistic understanding.
1 Introduction
Large language models (LLMs) generate surpris- ingly fluent and informative texts. This led to many works utilizing the text data generated by these models for purposes such as instruction tun- ing (Honovich et al., 2022; Wang et al., 2022) and improving reasoning capability (Zelikman et al., 2022; Ho et al., 2023; Magister et al., 2023).
By revisiting the formulation of answer gener- ation with retrieved passages (Guu et al., 2020; Lewis et al., 2020; Singh et al., 2021), we identify the then-overlooked gap, which has become signifi- cant now due the advance in LLMs (Brown al., 2020). Our discussion starts with the observation that the knowledge corpus from which the passages are retrieved is only subset of the possible string space. More helpful passages to the reader may exist outside the knowledge corpus. Unfortumately. retrieval, by definition, cannot utilize passages out- side the knowledge corpus, potentially causing shortfall. We refer this knowledge corpus error. In contrast, LLMs can generate passages from the entire string space, which may mitigate the inherent limits of retrieval.
Open-domain question answering (QA) (Chen al., 2017) task where retrieving relevant pas- sages from a corpus of factual information such as Wikipedia is standard practice. Recent works have attempted to generate such passages from LLMs, replacing the retrieval step of the traditional pipeline (Sun et al., 2023; Yu et al., 2023). Despite their success, it is not well understood why these generated passages could be more effective than
Work done during internship at KAIST
'Our code is available at this repository
We empirically demonstrate the presence

---

knowledge corpus error where passage from outside of Wikipedia outperforms the human- annotated gold passage inside Wikipedia ques- tion answerin. We design an experiment of paraphrasing human-annotated gold context with LLMs. Experiments with four QA benchmarks, NQ (Kwiatkowski et al., 2019). HotPotQA (Yang et al., 2018), StrategyQA (Geva et al., 2021), and QASC (Khot et al., 2019) result in 10% 13% gain in reader performance across three benchmarks, when using paraphrased passages. This gain sup- ports our hypothesis that there exist more helpful passages than the gold passage outside the knowl- edge corpus.
condition answer generation from reader (Izacard and Grave 2021, inter alia).
Mcanwhile, the success pre-trainee language models (Raffel et al., 2020) and the associative memory properties learned during training (Petroni et al., 2019) has allowed researchers to revisit closed-bor book QA, in which models answer the ques- tions without being provided a passage. Closed- book QA was demonstrated be effective both in in-context learning (Brown et al., 2020) and super- vised learning (Roberts et al., 2020).
Recent works on chain-of-thought thoughh prompting has shown that generating intermediate steps before giving an answer improves the reasoning capability of LLMs (Wei et al., 2022b; Wang et al., 2023). Inspired by this, recent works prompt LLMs generate the intermediate step of QA, which is the passages (Sun et al., 2023; Yu et al., 2023). These passages are subsequently fed into the reader, ei- ther supervised FiD (Izacard and Grave, 2021) or in-context learning LLMs (Brown et al., 2020). De- spite their success, these methods require very large scale and risk generated passages containing stale or non-factual information. Moreover, it is not fully explained why generating passages may have advantages over retrieval.
2 Related Work
2.1
Leveraging LLM-generated text
As the quality of text generated from LLMs has im- proved through larger models (Kaplan et al., 2020) and instruction tuning (Sanh et al., 2022; Wei et al., 2022a), many works have sought to use these mod- els as data sources in other NLP tasks. Text gen- erated from LLMs has been used for generating datasets for instruction finetuning (Honovich et al., 2022; Wang al., 2022), improving reasoning (Ze- likman et al., 2022; Ho et al., 2023; Magister et al.,
2023), and many other purposes (Liu et al., 2022a; Ye et al., 2022; Haluptzok et al., 2023).
3 Analytic Discussion
Recently, there has been growing attention to- wards open-source LLMs (Touvron et al., 2023) finetuned instructions generated from propri- etary LLMs, such as Alpaca (Taori al., 2023), Koala (Geng al., 2023) and Vieuna (Chiang et al.. 2023). Text generated from these models purport- edly match quality of those from proprietary LLMs (Chiang et al., 2023), but this assertion remains disputed (Gudibande et al., 2023). Understanding the role of LLM-generated text will serve as an important aspect of this discourse.
Our task formulation follows retrieval augmented models for QA (Guu et al., 2020; Lewis et al., 2020; Singh et al., 2021). These works view contexts latent variable for the QA model (Lee et al., 2019).
3.1 Setup
Let V be the infinite set of all possible strings over vocabulary tokens in V. including the empty string. An instance of a QA dataset consists of a triple (q. a, c): question q, answer a, and context c, where q. c E V*. Typically, the context c is retrieved from the knowledge corpus 2, such as Wikipedia, where 2 c V*.
2.2 Knowledge-intensive NLP and retrieval
Knowledge-intensive NL., such as open-domain QA (Chen et al., 2017) and fact verification (Thorne et al., 2018) 8), requires substantial factual knowledge that may change over time. Therefore, these tasks originally envisioned to incorporate the re- trieval of relevant passages from the knowledge corpus (Chen al., 2017). In the typical retrieve- then-read pipeline (Karpukhin et al. 2020, inter alia), a pipeline of models, first selects & passages from a retrieval function which are then used to
3.2 QA Task Formulation
The goal of QA is to learn distribution p(a[g), where models decode string that acts as an abstractive answer to the query (Lewis et al., 2020; Izacard and Grave, 2021). One can directly prompt language model to obtain an answer a, given question q (where context e implicitly the empty string), relying only on model parameters in closed- book QA (Roberts et al., 2020; Brown et al., 2020).

---

for Topk(Z, q). For contexts outside the knowledge corpus, ē & Z, applying Bayes' rule. p(clg) x p(glo)p(o), where the retrieval-based task formulation setting the prior p(²) =0 Knowl- edge corpus error may explain why reader mod- els can benefit from generated contexts (Sun et al., 2023; Yu et al., 2023) as LLMs can generate strings from the set V* O Z.
arg aax p(alg)
(1)
However, direct prompting is often difficult to learn and barely discloses its inner working. There- fore, a popular approach is to marginalize p(al4) over contexts in the knowledge corpus (Guu et al., 2020: Lewis et al., 2020; Singh et al., 2021). As it is intractable to calculate the probability for all the contexts in the knowledge corpus, p(alg) is approx- imated to the sum of probabilities for top & contexts from Z. TZp q) denotes the set of resulting top k passages after the retrieval with a query q.
Empirical Observation
To observe knowledge corpus error, we study the effect of paraphrasing human-annotated gold con- texts from QA dataset. Gold context Cyold E Z is what humans annotated as the supporting passage for a given question-answer pair. While human annotation may be imperfect, we assume that this Cgold acts as the best available passage from the knowledge corpus 2, i.c., there is no retrieval error. In our experiment, Cguld is paraphrased into Cporaphs by prompting LLMs with Cgold and g. Then, Coold and Cparaph are separately fed into the reader compare the performance. As Ggold is the best avail- able context, any gains from paraphrasing should be attributed to reduced knowledge corpus error.
p(c(2)= ~
Σ (ET)
p(ala, c)p(clg)
(2)
The gap in this formulation is that relevant con- text è may exist outside the knowledge corpus 2. This makes the sum of marginal probabilities over 2 only an approximation. The true proba- bility would require the summation of marginal probabilities over the infinite string space V*.
p(alg)= Σ p(ala, c)p(c(o)
4.1 Experimental setup
p(ala, clp(cl)) cE2
For a single instance of QA dataset (q, Cgold: a) and paraphrased context Paraph(Cgold- 4). we compare model performance two settings without and with paraphrasing: Read(g, Cgoid) and Read(g, Cparaph). Both Paraph() and Read() are function calls to LLMs, GPT-3.5 (gpt-3.5- turbo²) and Claude (claude- 1²t). Experiments were conducted in June 2023.
(3)
Σ
Oporaph
eET (ETopk(Z..))
3.3 Knowledge corpus error
Equation details two steps of approximation, which results in two sources of potential error in QA using contexts. The first source of error is introduced when the entire knowledge corpus Z is approximated to top k retrieved contexts, Topk(2,g) q) This error, which we denote retrieval error, can be mitigated by better retrieval meth- ods or increasing k, the number of contexts. On the other hand, the second source of error intro- duced when the entire string space V* is approxi- mated to knowledge corpus Z. This error is rooted in the use of knowledge corpus itself, hence we denote it as knowledge corpus error. To elabo- rate, for some ē & Z, p(ēlq) > plc E Zlg), but Pretriver (ő)q) 0 whereas (ālg) > 0.
4.2 Benchmarks
For benchmarks, we used NQ (Kwiatkowski et al., 2019), HotPotQA (Yang et al., 2018), Strategy (Geva al., 2021), and QASC (Khot et al., 2019). NQ consists of factual questions which can be an- swered with single passage. Unlike NQ Hot- PotQA consists of questions that require reasoning across multiple passages, known as multi-hop QA. Strategy QA and QASC further extend this multi- hop setting by requiring more implicit reasoning.
For gold context Ggold, we use the paragraph(s) from Wikipedia, which are part of the annotations in NQ. HotPotQA, and StrategyQA. For QASC,
For a query 4, p(clg) is sufficiently small for most contexts c. This allows these terms be ig- nored by setting p(clq) to zero. For instance, top-k retrieval is essentially setting p(clq) to zero
https://pppi openai htt.i/ooomm//l//oo/oiii Iichatcoopllesions htttps://api. wnt.ppic.aoroooccoooooo 1/complete

---

Benchmarks
Reader: GPT
Reader: Claude
Average gap between
Gold
GPT
Claude
Gold
GPT
Claude
gold and paraphrased
NQ exact match
40.9
39.9
44.3
18.3
21.3
35.5
3.125
HotPotQA exact match (%)
36.3
38.6
43.4
47.6
50.9
54.2
4.825
StrategyQA accuracy (%)
54.6
56.4
70.5
68.9
75.5
76.5
7.975
QASC accuracy (%)
95.7
92.4
91.1
86.3
75.7
76.9
6.975
Table 1: Performance of each reader when given original gold context ("Gold"), paraphrased context with GPT ("GPT"), and paraphrased context with Claude ("Claude"). Red indicates an increase in performance after para- phrasing, implying knowledge corpus error has been observed. Blue indicates decrease in performance after paraphrasing, implying knowledge corpus error has not been observed.
where such paragraph does not exist, we treat the seed facts that were used to create questions gold context. In multi-hop QA, concatenate all the contexts into a single context. See Appendix B.1 for details.
Wikipedia, with the communicative intent to gen- erally inform about the subject. Therefore, gold passages inevitably include information that is irrel- evant the question. LLMs can only filter the help- ful information from the gold passage during the paraphrasing. In fact, has been shown that when both retrieved and generated passages contain the correct answer, the FiD reader can produce more correct answers when reading the generated pas- sages (Yu et al., 2023). And furthermore, models sensitive to related but irrelevant information in phenomena called damaging retrieval (Sauchuk et al., 2022; Oh and Thorne, 2023). Query-focused paraphrasing acts as an information filter mitigating damaging retrieval.
4.3 Results
We report the results in Table 1. Paraphrased context outperforms the original gold context for most cases in NQ, HotpotQA, and StrategyQA. This means that paraphrased passages were more helpful than the gold passages, implying the ex- istence of knowledge corpus error. Moreover, us- ing the context paraphrased by different model did not cause any performance depredation, indicat- ing some level of universality in the helpfulness of the passages. We provide further analysis of this finding in Appendix E.
2. Chain-of-thought Some questions require a composition of facts to answer the question. In such case, we observe that paraphrasing is acting in manner similar to chain-of-thought (Wei al., 2022b). This also highlights the inherent limit of corpus such Wikipedia, where the explicit com- position information is seldom given.
QASC. We attribute the degradation in QASC for two reasons. First, the seed facts, which we considered gold contexts in QASC, not from raw corpus. The seed facts are manually selected from cleaned knowledge sources like WorldTree corpus. This is problematic as the gold contexts we are using represent the best-case scenario in retrieval, thereby eliminating any retrieval error. Second, distractor options in multiple-choice ques- tion confuses the model to generate passage rel- evant to those options. This results in a passage containing distracting information for answering the question. Examples in Table 8 illustrate these two points well.
In the second example Table 2, the question requires combining two distinct facts, one about the military unit (VMAQT-1) and another about Irish mythology (Banshee). The paraphrased context acts somewhat akin to chain-of- th-thh, resulting in more helpful context.
3. Incorporation of commonsense knowledge Commonsense knowledge plays a crucial role in understanding the world (Davis and Marcus, 2015), but not often explicitly stated, especially in a corpus such as Wikipedia. Language models are known to possess degree of tacit knowledge (Petroni et al., 2019), which can be utilized by knowledge generation (Liu al.. 2022b). We observe that during paraphrasing, commonsense knowledge is clicited, aiding the reader.
4.4 Qualitative Analysis
After manually examining a sample of results from the empirical study, we identify three common fac- tors contributing to knowledge corpus error.
Increased focus on the question Gold passages are a very small subset of facts from
The third example of Table 2 illustrates how com-

---

Souree
Question Answer
Gold
Paraphrased
Increased focus
Q. who plays
Title: The Haves and the Have Nots (TV series)
Charles, played by Nick Sager, of the main characters the show The Haves and the Have Nots. He is por- trayed the president-elect and love interest of the character Candace. (...)
the question
charles have and have nots A. Nick Sager
The Road to Hell", was revealed that David had a pulse while Erica's fate was casualty and died from the explo- Her death was the fuel for David's warpath of vengeance
Veronica. Charles Frederickson (Nick Sager): (Season 4 present) The president-elect and love interest to Candace, whom she uses to help her get her money back. (..)
Chain-of- thought
Q. The VMAQT-
Title: VMAQT-I
The VMAQT-I logo features a ban- shee, which a female spirit in Irish mythology. According to Irish mythol- ogy, banshees shriek or keen to foretell the death of a family member. (...)
logo is a fe- male spirit in Irish mythology who her- alds the death of what?
(..) The VMAQT-1 logo the Ban- shee, an Irish mythological figure fore- telling death. Its motto is "Tairngreacht Bas," Gaelic for "Death Foretold."
Title: Banshee
A. a family member
A banshee (: Modern Irish "bean si". from Old Irish: "han síde" ,1, "woman of the fairy mound" or "Fairy woman") is female spirit in Irish mythology who heralds the death of family member, usually by shrieking or keening. (..) Title: Bruce Lee
Incorporation of commonsense knowledge
Q. Was Bruce Lee absent from the 1964 University of Washington gradua- tion ceremony? A. True
There is mention of him attending or participating in the graduation in ci- ther document. The documents instead mention that in early 1964, Lee dropped out of college and moved Oakland (...) While this evidence is not definitive. it suggests that Lee probably did not attend participate the University of Washington's 's 1964 graduation mony.
martial artist, but he also studied drama and Asian and Westem philosophy while dent the University of Washington and throughout his life. (...) Title: Bruce Lee
Lee is best known
Lee dropped out of college in early 1964 and moved Oakland to live with James Yimm Lee. (...)
Table 2: Qualitative analysis on the source of knowledge corpus error Examples are respectively from NQ. HotPotQA, and StrategyQA (from top to bottom). All paraphrases are done by Claude. The bold part highlights how paraphrased context may be more helpful than gold context. See 4.4 for detailed explanation.
monsense knowledge - someone who dropped out of college will probably not attend graduation cer- emony- - is induced during paraphrasing.
Acknowledgments
This work was supported by Institute of Informa- tion & communications Technology Planning & Evaluation (UTP) grant funded by the Korea gov- ernment (MSIT) (No. (.6.2019-00000775 Artificial In- telligence Graduate School Program (KAIST)) and Artificial intelligence industrial convergence clus- ter development project funded by the Ministry of Science and ICT(MSIT, Korca) & Gwangju Metropolitan City.
5 Conclusion
In this work, we demonstrate that generated con- texts may be more helpful than retrieved contexts in open-domain question answering. By revisiting the formulation of question answering, we identify a gap where retriever inevitably ignores potentially helpful contexts outside of the corpus. We call this knowledge corpus error, and design experiment in order observe knowledge corpus error em- pirically. Paraphrasing the human-annotated gold contexts with L.MSs led increased reader perfor- mance in out of 4 QA benchmarks, implying the existence of knowledge corpus error.
Limitations
The first limitation of this work is that it did not em- ploy it in retrieval setting. We used a gold context, which assume is the best-case scenario in trieval. However, retrieved contexts in real retrieval setting (Karpukhin al., 2020), i.c., contexts in knowledge corpus other than the gold context, may deviate significantly from the gold context. There- fore, it is hard to discuss the effect of paraphras-

---

ing and the degree of knowledge corpus within retrieval.
domain questions. In Proceedings of the 55th Annual Meeting of the Association, for Computational Lin- guistics (Volume I: Long Papers), pages 1870-1879, Vancouver, Canada. Association for Computational Linguistics.
The second limitation of this work that it did not address the practical way to marry retrieval and generation via LLMs. Regardless of the seeming benefit of context generation, this approach suffers from issues such as information staleness and hallu- cination. Contemporaneous works explore various methods to leverage benefits of both retrieval and generation (He et al., 2022; Jiang et al., 2023; Xu et al.. 2023). This work is primarily concerned with the analytic understanding of how generation may have advantages over retrieval. We believe our work can inspire future contributions on empirical method of incorporating retrieval and generation.
Wei-Lin Chiang, Zhuohan Li, Zi Lin. Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang. Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: An open- source chatbot impressing gpt-4 with 90%* chatgpt quality.
Ernest Davis and Gary Marcus. 2015. Commonsense reasoning and commonsense knowledge in artificial intelligence. Commun. ACM, 58(9)922-103.
Xinyang Geng, Arnav Gudibande, Hao Liu, Eric Wal- lace, Pieter Abbeel, Sergey Levine, and Dawn Song, 2023. Koala: A dialogue model for academic re- search. Blog post.
The third limitation of this work is that its scope was limited to question answering. Conditioning generation on retrieved context is a well-studied approach in language modeling (Khandelwal et al., 2020; Yogatama et al., 2021; Borgeaud et al., 2022). It will be worth exploring how knowledge corpus error manifests within language modeling.
Mor Geva, Daniel Khashabi, Elad Segal, Tushar Khot, Dan Roth, and Jonathan Berant. 2021. Did aristotle use laptop? question answering benchmark with implicit reasoning strategies. Transactions of the Association for Computational Linguistics, 9:346- 361.
References
Arnay Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song. 2023. The false promise of imitating proprietary 1lms.
Sebastian Borgeaud, Arthur Mensch, Jordan Hoff- mann, Trevor Cai, Eliza Rutherford, Katie Milli- can, George Bm Van Den Driessche, dean-Baptiste Lespiau, Bogdan Damoc, Aidan Clark, Diego De Las Casas, Aurelia Guy, Jacob Menick, Roman Ring, Tom Hennigan, Saffron Huang, Loren Mag- giore, Chris Jones, Albin Cassirer, Andy Brock, Michela Paganini, Geoffrey Irving, Oriol Vinyals, Simon Osindero, Karen Simonyan, Jack Rae, Erich Elsen, and Laurent Sifre. 2022. Improving language models by retrieving from trillions of tokens. In Proceedings of the 39th International Conference on Machine Learning, volume 162 of Proceedings of Machine Learning Research, pages 2206-2240. PMLR.
Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang- 2020. Retrieval Augmented Language Model Pre-Training. In Proceedings of the 37th International Conference on Machine Learning, pages 3929-3938. PMLR. ISSN: 2640-3498.
Patrick Haluptzok, Matthew Bowers, and Adam Tauman Kalai. 2023. Language models can teach themselves to program better. In The Eleventh International Conference on Learning Representations.
Hangfeng He, Hongming Zhang, and Dan Roth. 2022. Rethinking with retrieval: Faithful large language model inference.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss Voss, Gretchen Krueger, Tom Henighan. Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Ma- teusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Leamers. In Ad- vances in Neural Information Processing Systems, volume 33, pages 1877-1901. Curran Associates, Inc.
Namgyu Ho, Laura Schmid, and Se- Young Yun. 2023. Large language models are reasoning teachers.
Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. 2022. Unnatural instructions: Tuning lan- guage models with (almost) no human labor.
Gautier Izacard and Edouard Grave 2021. Leveraging passage retrieval with generative models for open do- main question answering. In Proceedings of the 16th Conference of the European Chapter of the Associ- ation for Computational Linguistics: Main Volume, pages 874-880, Online. Association for Computa- tional Linguistics.
Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading Wikipedia to answer open-

---

Zhengbao Jiang, Frank F. Xu, Luyu Gao, Zhiqing Sun, Qian Liu, Jane Dwivedi- Yu, Yiming Yang. Jamie Callan, and Graham Neubig. 2023. Active retrieval augmented generation.
In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 6826-6886 Abu Dhabi, United Arab Emirates. Association for Com- putational Linguistics.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess. Rewon Child, Scott Gray. Alec Radford, Jeffrey Wu, and Dario Amodei. 2020. Scaling laws for neural language models.
Jiacheng Liu, Alisa Liu, Ximing Lu, Sean Welleck, Pe- ter West, Ronan Le Bras, Yejin Choi, and Hannaneh Hajishirzi. 2022b. Generated knowledge prompting for commonsense reasoning. In Proceedings of the 60th Anmual Meeting of the Association for Compu- tational Linguistics (Volume I: Long Papers), pages 3154-3169, Dublin, Ireland. Association for Compu- tational Linguistics.
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov. Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for open- domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769-6781, Online. Association for Computational Linguistics.
Lucic Charlotte Magister. Jonathan Mallinson, Jakub Adamek, Eric Malmi, and Aliaksei Severyn. 2023, Teaching small language models to reason.
Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2020. Generalization through memorization: Nearest neighbor language models. In International Conference Learning Representations.
Philhoon Oh and James Thome. 2023. Detrimental con- texts in open-domain question answering. In Find- ings of the Association for Computational Linguistics: EMNLP 2023, Singapore. Association for Computa- tional Linguistics.
Tushar Khot, Peter Clark, Michal Guerquin, Pe- ter Alexander Jansen, and Ashish Sabharwal. 2019. Qasc: A dataset for question answering via sentence composition. In AAAI Conference on Artificial Intel- ligence.
Fabio Petroni, Aleksandra Piktus. Angela Fan, Patrick Lewis, Majid Yazdani, Nicola De Cao, James Thorne, Yacine Jernite, Vladimir Karpukhin, Jean Maillard, Vassilis Plachouras, Tim Rocktäschel, and Sebastian Riedel. 2021. KILT: a benchmark for knowledge intensive language tasks. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2523-2544, Online. Association for Computational Linguisties.
Takeshi Kojima, Shixiang (Shane) Gu, Machel Reid, Yu- taka Matsuo, and Yusuke Iwasawa. 2022. Large lan- guage models are zero-shot reasoners. In Advances in Neural Information Processing Systems, volume 35, Curran Associates, Inc.
pages
Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. Language models as knowl- edge bases?
Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red- field, Michael Collins. Ankur Parikh, Chris Alberti, Danielle Epstein, Iilia Polosukhin. Jacob Devlin, Ken- ton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming- Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoe Le. and Slav Petrov. 2019. Natu- ral questions: A benchmark for question answering research. Transactions of the Association for Compu- tational Linguistics, 7:452 466.
Proceedings of the 2019 Confer- ence on Empirical Methods in Natural Language Pro- cessing and the 9th International Joint Conference on Natural Language Processing (EMNLP-ICCLP)), pages 2463-2473, Hong Kong, China. Association for Computational Linguistics.
Colin Raffel, Noam Shazeer, Adam Roberts, Kather- ine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140):1-67.
Kenton Lee, Ming-Wes Wei Chang, and Kristina Toutanova. 2019. Latent retrieval for weakly supervised open domain question answering. In Proceedings of the 57th Annual Meeting of the Association for Computa- tional Linguistics, pages 6086-6096. Florence, Italy. Association for Computational Linguisties.
Adam Roberts, Colin Raffel, and Noam Shazeer. 2020. How much knowledge can you pack into the param- eters of a language model? In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5418-5426, Online. Association for Computational Linguistics.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Hein- rich Küttler. Mike Lewis, Wen-tau Yih. Tim Rock- täschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augnee -uggmented generation for knowledge- intensive nlp tasks. In Advances in Neural Infor- mation Processing Systems, volume 33, pages 9459- 9474. Curran Associates, Inc.
Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szezechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti
Alisa Liu, Swabha Swayamdipta, Noah A. Smith, and Yejin Choi. 2022a. WANLI: Worker and Al collabo- ration for natural language inference dataset creation.

---

Datta, Jonathan Chang, Mike Tian-Jian Jiang, Han Wang, Matteo Manica, Sheng Shen, Zheng Xin Yong, Harshit Pandey, Rachel Bawden, Thomas Wang, Tr- ishala Neeraj. Jos Rozen, Abheesht Sharma, An- drea Santilli, Thibault Fevry, Jason Alan Fries, Ryan Teehan, Teven Le Scao, Stella Biderman, Leo Gao. Thomas Wolf, and Alexander M Rush. 2022. Multi- task prompted training enables zero-shot task gener- alization. In International Conference on Learning Representations.
Hajishirzi. 2022. Self-instruct: Aligning language model with self generated instructions.
Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoe V Le. 2022a. Finetuned language models are zero-shot learners. In International Con- ference on Learning Representations.
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Brian Ichter. Fei Xia, Ed Chi, Quoc V. Le, and Denny Zhou. 2022b. Chain-of-Thought Prompt- ing Elicits Reasoning in Large Language Models. Advances in Neural Information Processing Systems. 35:24824-24837.
Artsiom Sauchuk, James Thorne, Alon Halevy, Nicola Tonellotto, and Fabrizio Silvestri. 2022. On the role of relevance in natural language processing tasks. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Infor- mation Retrieval, SIGIR '2, page 18891789, New York, NY, USA. Association for Computing Machin- ery.
Fangyuan Xu, Weijia Shi, and Eunsol Choi. 2023. Re- comp: Improving retrieval-augmennt Ims with com- pression and selective augmentation.
Zhilin Yang. Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christo- pher D. Manning. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering, In Proceedings of the 2018 Conference on Empiri- cal Methods in Natural Language Processing, pages 2369-2380, Brussels, Belgium. Association for Com- putational Linguistics.
Devendra Singh, Siva Reddy, Will Hamilton, Chris Dyer, and Dani Yogatama. 2021. End-to End-to-Ene -nd Train- ing of Multi-Document Reader and Retriever for Open-Domain Question Answering. In Advances in Neural Information Processing Systems, volume 34, pages 25968- 25981. Curran Associates, Inc.
Zhiqing Sun, Xuezhi Wang. Yi Tay, Yiming Yang, and Denny Zhou. 2023. Recitation -ugmenttdd language models. In The Eleventh International Conference on Learning Representations.
Jiacheng Ye, Jiahui Gao, Qintong Li, Hang Xu, Jiangtao Feng. Zhiyong Wu, Tao Yu, and Lingpeng Kong. 2022. ZeroGen: Efficient zero-shot learning via dataset generation. In Proceedings of the 2022 Con- ference on Empirical Methods in Natural Language Processing, pages 11653-11669, Abu Dhabi, United Arah Emirates. Association for Computational Lin- guistics.
Rohan Taori, Ishaan Guirajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. 2023. Stanford alpaca: An cnstruction-folloiing Ilaa model. https:// github. www. watpuulaaasttaadd_.aao
Dani Yogatama, Cyprien de Masson d' Autume, and Lingpeng Kong. 2021. Adaptive semiparametric lan- guage models. Transactions the Association for Computational Linguisticss 9:362-373.
James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. FEVER: large-scale dataset for fact extraction and VERification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pupers), pages 809-819, New Orleans, Louisiana. Association for Computational Linguistics.
Wenhao Yu, Dan Iter, Shuohang Wang, Yichong Xu, Mingxuan Ju, Soumya Sanyal, Chenguang Zhu, Michael Zeng, and Meng Jiang. 2023. Generate rather than retrieve: Large language models are strong context generators. In The Eleventh Inter- national Conference on Learning Representations.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13977
Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Good- man. 2022. STaR:
Bootstrapping Reasoning With Reasoning. In Advances in Neural Information Pro- cessing Syxtems, volume 35, pages 15476-15488. Curran Associates, Inc.
A Dataset
Xuezhi Wang. Jason Wei, Dale Schuurmans, Quoe V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery. and Denny Zhou. 2023. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations.
NQ: We use the KILT (Petroni et al., 2021) version of NQ We use the dev split after excluding the instances where context does not include answers. which results in 2532 samples.
HotPotQA: We use the dataset from its original
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh

---

source". We use the dev split, which includes 7405 samples. To reduce the inference cost, we only use the subset of first 1531 samples.
StrategyQA: Max tokens generate was
300 in paraphrase and 10 in read.
QASC: Max tokens to generate was set to 100 paraphrase and 10 in read. Temperature during generation was set to 0.8 in paraphrase and 0.4 in read.
StrategyQA- QA: We use the dataset from its original source 6, We use the training split, which includes 2290 samples. We use the training split because the dev split contained too few (490) examples. QASC: We use the dataset from its original source 7. We use the dev split, which includes 926 sam- ples.
We used 3-shot prompting for reading NQ but otherwise used zero-shot prompting. Other generation keyword arguments are set to default if not specified. For the prompts used, see Table 4.
Hyperparameters related to generation are de- cided mainly through trial-and-error, For example, max tokens was adjusted according to few prelim- inary samples. We tried to tweak temperature for QASC after observing deviant result, but only had minor impact. 3-shot setup was chosen for NQ because the performance was too low in zero-shot.
B Detailed experimental setup
B.1 Selection of gold passage
NQ contains set of provenances for possible answer contexts. For the experiments, we select the gold passages from the provenances that include at least one of the candidate answers. When there multiple good passages, employ the very first one.
NQ:
C Details on evaluation
HotPotQA: HotPotQA contains 2 gold paragraphs from Wikipedia for each question. gold passage is simply the concatenation of these two. Note that we do not utilize tnn-ggrined sentence-level anno- tation in 2 paragraphs.
Evaluating exact match or accuracy may be non- trivial in a generative setting. Hence, we follow the previous works (Kojima et al., 2022; Yu et al., 2023).
Strategy QA: StrategyQA contains decomposition steps solve the question. Each of these steps may be attached with supporting paragraph from Wikipedia. A gold passage the concatenation of all these paragraphs throughout the whole steps. Among three different annotated decomposing steps in the dataset, we use the first one.
NQ (Exact Match): Following Yu et al. 2023, we measure exact match of the output string after nor- malization.
HotPotQA (Exact Match): Similarly as Yu et al. 2023, we measure exact match of the output string after normalization.
Strategy QA (Accuracy): Following Kojima et al. 2022, we measure accuracy by picking up the first "yes" or "no" encountered in the text after remov- ing unnecessary letters.
QASC: QASC contains two facts that are com- bined to create a question. These facts are selected from a cleaned knowledge source. A gold passage is simply the concatenation of these two facts.
QASC (Accuracy): Similarly as Kojima al. 2022, we measure accuracy by picking up the first large letter out of to H encountered the text.
The title of the passage is prepended the pas- sage in cases where titles are available (NQ, Hot- PotQA, and StrategyQA).
D Inference Cost
B.2 Details on generation
NQ: During reading, we used 3-shot prompting, where the 3-shot demonstrations are sampled from GPT-3.5 with questions from the dev split of NQ. Note that these questions are excluded from the experiment. Max tokens to generate was set to 500 in paraphrase and 25 in read.
We used OpenAl and Anthropic's API to use their LLMs. The cost for OpenAl's API is estimated to be around $40 to $50. Anthropic's API has not cost any we were on the free version.
E Accordance between heterogeneous reader models
HotPotQA: Max tokens to generate was set to 300
in paraphrase and 10 in read.
For majority of the examples, two readers accord with each other, i..., both are correct or wrong, and this ratio is even higher in paraphrased contexts.
"huts://hotpookaagituuu https://allenai.or/daaaa data/qque

---

Both
Only correct for
Given context
correct
wrong
GPT
Claude
Gold
21
GPT-paraph
18
Claude-paraph
42
15
Table 3: Accordance between two heterogencous reader models, GPT and Claude. The table based on the reader performance (exact match (%)) in HotPoiQA when given gold context, GPT-paraphrased context, and hauueepaaraprasee context (from top to bottom).
F Examples
For examples of paraphrased context, see Table through 8.

---

Benchmark
Paraphrase
Read
NQ
Paraphrase background document in your words answer the given question.
Referring to the passage, find the correct answer (just one entity) to the given question. will first show you few examples.
Question: {q)
Passage: [c1]
Document: {c}
Question: {ql)
Answer: (al)
example continues
Passage: (c]
Question: {q}
Answer:
HotPotQA
Paraphrase the background documents into single document in your own words answer the given question.
Referring to the passage, find the correct answer (just one entity) to the given question.
Question: {q}
{c}
Documents: {c)
(q)
StrategyQA QA
Paraphrase the background documents into single document in your own words to answer the given question.
Read the passage and answer the question with yes or no.
(c)
Question: {q}
{q}
Documents: (c)
QASC
Paraphrase the background documents into a single document in your own words to answer the given question.
Read the passage and answer the question with one of A, B, E.
G. or H.
Question: (q)
(c)
Documents: (c)
(q)
Table 4: Prompts used for paraphrasing and reading. (q), [c], and [a] each indicate question, context, answer from the test instance. (q1), [c1), and (al) indicate of the demonstrations used for few-shot prompting. Note that the full few-shot prompt format is not displayed for conciseness.

---

Benchmark
Gold
Paraphrased
NQ
Question: what does the pearl symbol- ize in the bible
The Pearl of Great Price is a parable told by Jesus that is found in the book of Matthew. It teaches us about the immense worth of the Kingdom of Heaven. This parable is situated just before the Parable of the Dragnet and comes after the Parable of the Hidden Treasure, which has similar meaning. (...)
GPT:
Title: Parable of the Pearl
Parable of the Pearl The Parable of the Pearl (also called the Pearl of Great Price) one of the parables of Jesus. It appears in and illustrates the great value of the Kingdom of Heaven. It is the penultimate parable in Matthew 13, just before the Parable of the Dragnet. It immediately follows the Parable of the Hidden Treasure, which has similar theme. does not appear in the other synoptic gospels but a version of this parable does appear in the non-canonical Gospel of Thomas
GPT: X-> X, Claude: X X~>0 0
In summary, the pearl symbolizes the Kingdom of Heaven in the Bible. This illustrated in the Parable of the Pearl told by Jesus, where merchant finds a pearl of great value and sells all his possessions so that he can buy it. The pearl represents the Kingdom of Heaven, which is so valuable that one should sacrifice everything else to obtain it.
Claude:
GPT: X-> O, Claude: X X
Question: when did the song here comes the boom come out
GPT: "Here Comes the Boom" is song by P.O.D., an American rock band. It was released in May 2002 as the third single from their album "Satellite." " Although it didn't 't perform as well as the previous singles from the album. it has been featured in numerous movies and TV shows. (...) GPT: X-> O, Claude: X->
Title: Boom (P.O.D. song)
Boom (P.O.D. song) "Boom" is song by American rock band P.O.D. It released in May 2002 as the third sin- gle from their second major label studio album "Satellite". While it did not chart as well as the albumá previous singles, the song has appeared significantly in film and television. "Boom (The Crystal Method remix)" was included on the remix album "Community Service" and as bonus track on the special edition re-release of "Satellite" available August 27
The song 'Here Comes the Boom' by the rock band P.O.D. was released in May 2002. It was the third single from their album 'Satellite' released that year. Though the song did not chart highly, it has been featured in various films and TV shows. A remixed version of the song by The Crystal Method was included in P.O.D.'s remix album CCommunity Service' as well as in special re-release of the 'Satellite' album in August 2002.
Claude:
GPT: X -> O, Claude: X->
Table 5: Examples of gold contexts and paraphrased contexts in NQ. Answers are highlighted. Below the paraphrased context are results of each reader. GPT: O indicates that GPT reader got wrong with original context, but right with paraphrased context.

---

Benchmark
Gold
Paraphrased
HotPotQA
Question: The VMAQT-1 logo female spirit in Irish mythology who heralds the death of what?
GPT: VMAQT-I is US Marine Corps training squadron consisting of EA-6B Prowler jets based in North Carolina. Their logo is the Banshee, a female spirit from Irish mythology who predicts death in a family by screaming or keening. The Banshees name is associated with the mounds in the Irish countryside known as "side." The squadrons motto is "Tairngreacht Bas," meaning "Death Foretold" Gaelic. GPT: 0 -> X, Claude: X-> X
Title: VMAQT-1
Marine Tactical Electronic Warfare Training Squadron (VMAQT-1) is United States Marine Corps electronic warfare training squadron consisting of EA-6B Prowler jets. The squadron is based at Marine Corps Air Station Cherry Point, North Carolina and falls under the command of Marine Aircraft Group 14 (MAG-14) and the 2nd Marine Aireraft Wing (2nd MAW). The VMAQT-1 logo is the Banshee, an Irish mythological figure foretelling death. Its motto "Tairn- greacht Bas," Gaelic for "Death Foretold."
The VMAQT-1 logo fea- tures a banshee, which is female spirit in Irish mythology. According Irish mythology, banshees shrick or keen foretell the death of family member. The banshee's name comes from síde, the mythological mounds in the Irish countryside. Therefore, the banshee in the VMAQT-I logo heralds the death of family member.
Claude:
Title: Banshee
A banshee ( Modern Irish "bean sf". from Old Irish: "ban síde" "woman of the fairy mound" "fairy woman")
GPT: O -> O, Claude; X -> O
female spirit in Irish mythology who heralds the death of family member, usually by shricking or keening. Her name is connected to the mythologically-imporraa tumuli or "mounds" that dot the Trish countryside, which are known síde (singular "síd") in Old Irish.
Table 6: An example of gold context and paraphrased contexts in HotPotQA. Answers highlighted. Below the paraphrased context are the results of each reader. GPT: X -> O indicates that GPT reader got wrong with original context, right with paraphrased context.

---

Benchmark
Gold
Paraphrased GPT: The Lamborghini Veneno is a limited production high performance sports car created to commemorate Lamborghini's 50th anniversary. It is based the Lamborghini Aventador and was first showcased the 2013 Geneva Motor Show with price $4 million, making it one of the most expensive cars the world. It has a top speed of 356 km/h, can accelerate from 0-97 km/h in 2.8 seconds, and can brake from 97-0 km/h in 30 meters while cornering at 1.41 Gs. The Porsche 911 GTI is a race car developed in 1996 for the GT1 class in the 24 Hours of Le Mans. In order to qualify for GT racing, 25 woad-going models were built to achieve type homologation. The engine produces 608 PS (600 hp) for the race version and 544 PS (537 hp) for the road version. It accelerate from 0-97 km/h in 3.3 seconds and has top speed of 330 km/h.
StrategyQA
Question: Can Lamborghini' fastest model win a race against a Porsche 911?
Answer: yes
Title: Lamborghini Veneno
The Lamborghini Veneno (Spanish pronunciation:(be nenol) a limited production high performance sportscar manufactured by Italian automobile manufacturer Lamborghini. (...) The Lamborghini Veneno has top speed of 356 km/h (221 mph) and has 0-97 km/h (0-60 mph) acceleration time of 2.8 seconds. The car has braking distance of 30 m (98.0 ft) from 97-0 km/h (60-0 mph), and can produce 1.41 G while cornering.
Title: Porsche 911
Porsche 911 GTI on display at the
(...)
Porsche Museum The Porsche 911 GTI a race car that was developed 1996 for the GT1 class in the 24 Hours of Le Mans. In order to qualify for GT racing. 25 road-going models were built to achieve type homologation. The engine in the GTI is rated at 608 PS (447 kW; 600 hp) (544 PS (400 kW; 537 hp) for the road version) and accelerated from 0-97 km/h in 3.3 seconds. The top speed stood at 330 km/h (205 mph). (...)
GPT: X Claude: X
Claude: Based on the background
documents. seems that Lamborghini Veneno would likely win race against a Porsche 911. The Veneno has higher top speed of 356 km/h compared to 330 km/h for the Porsche 911 GTI. The Veneno also has faster acceleration, able to go from 0 to km/h in just 2.8 seconds compared to 3.3 seconds for the Porsche. Although the Porsche 911 GTI was designed primarily as race car and the Veneno was more of a limited production sportscar, the Veneno still has superior performance specifications. Given its faster speed, acceleration and handling, the Lamborghini Veneno model would probably win a race against a Porsche 911. GPT: X O, Claude: X
Table 7: An example of a gold context and paraphrased contexts in Strategy Below the paraphrased context are the results each reader. GPT: indicates GPT reader got wrong with original context, but right with paraphrased context.

---

Benchmark
Gold
Paraphrased
QASC
Question:
important for What pollens? (A)
GPT: Allergens substances that cause allergies, and they include common sources such as pollens and dusts. Pollens are not unicellular organisms or bees, but rather they are building blocks of organisms composed of nuclcotides that form polymers. They are not diploid or microscopic, but they are known to be antigens that can trigger allergic reactions.
eukaryotes (B) unicellular organisms
(C) building blocks of organisms (D) antigens that cause allergy (E) bees (F) microscopic organisms (G) nucleotides composing polymers (H) diploid Answer: (D)
Any antigen that causes allergy is called allergen. Many of the common allergens are pollens and dusts..
GPT: O Claude: X
Here is paraphrased summary of the background documents:
Claude:
Pollens are microscopic cells that contain the male gametes of plants and facilitate reproduction. They are produced by the anthers of stamens in flowering plants. When pollen grains get dispersed by wind, water, or insects and come contact with the stigma of a flower, they germinate and produce a pollen tube to transfer the sperm cells to the ovule.
Though pollens plant
GPT: 0 X, Claude: X
Question: Climate is generally de- scribed in terms of what? (A) sand (B) occurs over a wide range (C) forests (D) Global warming (E) rapid changes occur (F) local weather conditions (G)
GPT: The climate typically defined by its temperature and moisture levels. The behavior of fires is influenced by local weather conditions, which may include wind, temperature, and moisture. GPT: X O, Claude: X 0
measure of motion (H) city life
Answer: (F)
Based on the background documents, climate is generally described in terms of temperature and moisture or local weather conditions, not the other options.
Claude:
Climate is generally described in
terms of temperature and moisture.
Fire behavior is driven by local weather conditions such winds, temperature and moisture.
GPT: X -> O, Claude: X-> 0
Table 8: Examples of gold contexts and paraphrased contexts in QASC. Below the paraphrased context are the results of each reader. GPT: X> 0 indicates that GPT reader got wrong with original context but right with paraphrased context. The first example shows a failure case, where distractor option misleads the paraphraser generate irrelevant information (see the red-colored part). The second example, on the other hand, shows success case.


Reviews:
[{"review_id": "piTqQAztTC", "reviewer": "Reviewer_bwTs", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback. We will first provide a general comment regarding the reasons to accept, before discussing responses to the three main points from “Reasons To Reject” and then close with an additional analysis that may find useful.\n\n\n# Reasons to accept\n\n\nThank you for identifying the ease of implementation of our approach. We hope that it can be used to aid reproducibility for other researchers. However, we wish to assert that our contributions are more than a “simple method to gain performance advantages”. \n\n\nWhat we pose is a new framework to help researchers reason about how knowledge is incorporated in open-domain NLP tasks combining retrieval and reasoning. We hope that the empirical findings of our paper help justify our argument. However, our principle argument centers around the source of the error. While we show that the LM can aid with smoothing the errors with pipeline, we wish to remind the reviewer that our contributions also include the analytical framework to reason with these errors.\n\n\n# Reasons to reject\n## Analysis on QASC\n\n\nWe apologize for the lack of clarity in our explanation for performance degradation for QASC (§4.3). Here we provide examples to better illustrate our explanation. We will include further details in the final version of the manuscript.\n\n\n1. **Gold passages we used are manually created, instead of being extracted from raw Wikipedia.**  \nGold passages we used represent the best-case scenario in retrieval, thereby eliminating any retrieval error. However, gold passages we used for QASC do not reflect this setting, as they are **manually selected** from cleaned knowledge sources like WorldTree corpus. Such choice was done because there were no annotated supporting passage from raw source such as Wikipedia. An example below illustrates how manually created \"gold\" in QASC differs from other benchmarks.  \n\n\n\n\n*QASC*:  Any antigen that causes an allergy is called an allergen. Many of the common allergens are pollens and dusts.    \n*NQ*:  \nTitle: Boom (P.O.D. song)  \nBoom (P.O.D. song) \"Boom\" is a song by American rock band P.O.D. It was released in May 2002 as the third single from their second major label studio album \"Satellite\". While it did not chart as well as the album's previous singles, the song has appeared significantly in film and television. \"Boom (The Crystal Method remix)\" was included on the remix album \"Community Service\" and as a bonus track on the special edition re-release of \"Satellite\" available August 27  \n\n\n---\n\n\n\n\n2. **Distractor options in multiple-choice question confuses the model to generate passage relevant to those options.**   \nThis is best illustrated through an example. See the bold parts to see how distractor option (C) misled the paraphraser, GPT.\n  \n*Question*: What are pollens? (A) eukaryotes (B) unicellular organisms **(C) building blocks of organisms** (D) antigens that cause allergy (E) bees (F) microscopic organisms (G) nucleotides composing polymers (H) diploid  \n\n\n*Answer*: D  \n\n\n*Gold passage*: Any antigen that causes an allergy is called an allergen. Many of the common allergens are pollens and dusts.  \n\n\n*GPT-paraphrased passage*: Allergens are substances that cause allergies, and they include common sources such as pollens and dusts.  Pollens are not unicellular organisms or bees, but rather **they are building blocks of organisms** composed of nucleotides that form polymers. They are not diploid or microscopic, but they are known to be antigens that can trigger allergic reactions.  \n\n\n---\n\n\n## Details on our “paraphrase” experiment\nWe believe our intent was not delivered well, so we will first reiterate our argument, which will help answer the questions.\n> \"The authors suggest that paraphrasing allows the model to incorporate knowledge that lies outside the knowledge corpus\"\n\n\n\n\nThis is **not** our argument. Our argument on *knowledge corpus error* is rooted in the formulation rather than intuition. Knowledge corpus error arises when some context outside of knowledge corpus is ignored even though it is better than the context inside the knowledge corpus (=context being retrieved). Formally speaking, for question $q$ and knowledge corpus $\\mathcal{Z}$, $p_{retriever}(\\tilde{c} \\notin \\mathcal{Z}|q) = 0$ where $p(\\tilde{c}|q) > p(c \\in \\mathcal{Z}|q)$.  \n\nThis concept by itself doesn't assert anything about the source of knowledge corpus error (= source of improvement from paraphrasing), leaving a room for further works to contribute explanations specific to model types or tasks. From our manual inspection, we concluded three main factors contributing to knowledge corpus error (developed from §3.3). \n1. More focus on the question\n2. Chain-of-thought\n3. Commonsense knowledge\n\n\nFor conciseness, we defer detailed explanation to the very last section and answer the questions.\n\n\n\n\n> (1) \"doing something akin to chain-of-thought prompting\"  \n\n\nChain-of-thought definitely explains the benefit from paraphrasing. In fact, successful composition of multiple facts is often observed in cases where paraphrasing benefits the reader. \n\n\n> (2) \"simply rephrasing the passage into a more usable format\"\n\n\nThis is exactly our point, hence we named our experiment *paraphrase*.\n\n\n> (3) \"can also lead to hallucination\"\n\n\nHallucination is definitely a downside of using passages outside of knowledge corpus, as they usually require generation from LMs. Nonetheless, we’d like to emphasize that paraphrasing results in less hallucination compared to generation from scratch such as GenRead (Yu et al., 2023). Still, it will be important to find out whether LMs are faithfully paraphrasing the given gold passage, not adding any information (no matter it is factually correct or not).We will address this question in the future study.\n\n\n## Instances where retrieval hurts / Comparison with pure LM generation\nFirst, we want to be clear that we did not actually retrieved the passages. We rather **used gold passages as the best-case scenario in retrieval**. Mallen et al., 2023 has shown that “non-parametric memories can mislead LMs”. However, they explain this with low recall@1 of retrieved passages for the questions the reader failed. In contrast, for the passages we used, recall@1 is 100%, because they are the gold which contain the answer.\n\n\n> Another type of analysis that would strengthen this work is understanding what types of questions this method works for.\n\n\nThis analysis will be helpful to understand why knowledge corpus error arises. In fact, we suggested a short comparison between benchmarks in the second paragraph of §4.3. We found out that the benefit of paraphrasing is greater in HotPotQA(4.8) and StrategyQA(8.0) than NQ(3.1). This is probably because the former requires reasoning and commonsense, whereas NQ mostly requires simple fact recall.  \nIn addition to our benchmark-wise analysis, question-wise analysis may also be useful. We will consider it in the future version.\n\n\n> It is very possible that the proposed method is simply mitigating the negative effects of retrieval in such instances, and eliminating retrieval in these instances would achieve a similar effect.\n\n\nThis is an interesting suggestion. There may be a chance that gold passaegs are harming the readers. If so, “non-parametric knowledge misleading LMs* will require further explanation that lower recall@1 in retrieved passages (Mallen et al., 2023), as recall@1 is 100% for gold passages. We look forward to delve into this question in further study.\n\n\n## Appendix: Qualitative analysis on the source of knowledge corpus error\nIn §3.3, we posited two factors as the source of knowledge corpus error (= why paraphrasing improves the reader performance). Through further analysis, we identified three factors contributing to knowledge corpus error and have provided illustrative examples for each. \n\n1. **More focus on the question**  \nGold passages are a fraction of Wikipedia, where the written communicative intent is to generally inform about the subject. Therefore, gold passages inevitably include irrelevant information to the question.   \nIn fact, it has been shown that “when both retrieved and generated documents contain the correct answer, the FiD reader can produce more correct answers when reading the generated documents” (Yu et al., 2023 - §4.4). It suggests that retrieved passages are more likely to contain noisy information that is irrelevant to the question.  \nBelow is our example from NQ. The gold passage focuses on general information about the TV series, while paraphrased ones has shifted their focus to the character the actor played, which is directly tied to the question.\n\n\n*Question/Answer*: who plays charles on have and have nots  / Nick Sager\n\n*Gold*:  \nTitle: The Haves and the Have Nots (TV series)  \nThe Road to Hell\", it was revealed that David had a pulse while Erica's fate was a casualty and died from the explosion. Her death was the fuel for David’s warpath of vengeance on Veronica. - **Charles Frederickson (Nick Sager)**: (Season 4–present) The president-elect and the love interest to Candace, whom she uses to help her get her money back. (...)   \n\n*GPT-paraphrased*:  \ncrew of \"The Haves and the Have Nots\": In the TV series, \"The Haves and the Have Nots\", **Charles Frederickson is played by Nick Sager**. He plays the character of the president-elect and Candace's love interest. (...)  \n\n*Claude-paraphrased*:   \n**Charles, played by Nick Sager**, is one of the main characters on the show The Haves and the Have Nots. He is portrayed as the president-elect and love interest of the character Candace. (...)  \n\n---\n2. **Chain-of-thought**  \nSome questions require composition of facts to answer the question. In such case, we saw that paraphrasing is doing something similar to chain-of-thought.  \nBelow is an example from HotPotQA (Table 4 in the paper). Paraphrased passage is combining two distinct facts into a single fact.  \n\n*Question/Answer*: The VMAQT-1 logo is a female spirit in Irish mythology who heralds the death of what?  /  a family member  \n\n*Gold*:   \nTitle: VMAQT-1  \n(...) The VMAQT-1 logo is the Banshee, an Irish mythological figure foretelling death. Its motto is “Tairngreacht Bas,” Gaelic for “Death Foretold.”  \nTitle: Banshee  \nA banshee ( ; Modern Irish \"bean sí\", from Old Irish: \"ban síde\" , ] , \"woman of the fairy mound\" or \"fairy woman\") is a female spirit in Irish mythology who heralds the death of a family member, usually by shrieking or keening. (...)\n\n*Claude-paraphrased*:   \nThe VMAQT-1 logo features a banshee, which is a female spirit in Irish mythology. According to Irish mythology, banshees shriek or keen to foretell the death of a family member. (...)  \n\n---\n3. **Commonsense knowledge**  \nCommonsense knowledge plays a crucial role in understanding the world, but not often explicitly stated (especially in Wikipedia). During paraphrasing, commonsense knowledge is expressed in the passage, which aids the reader to answer better.  \nBelow is an example from StrategyQA. *Someone who dropped out of college will probably not attend graduation ceremony* is an important commonsense knowledge here.\n\n*Question/Answer*: Was Bruce Lee absent from the 1964 University of Washington graduation ceremony? / True  \n\n*Gold*:   \nTitle: Bruce Lee  \nLee is best known as a martial artist, but he also studied drama and Asian and Western philosophy while a student at the University of Washington and throughout his life. (...)  \nTitle: Bruce Lee  \nLee dropped out of college in early 1964 and moved to Oakland to live with James Yimm Lee. (...)  \n\n*Claude-paraphrased*:   \nBased on the background documents, it appears that Bruce Lee was likely absent from the 1964 University of Washington graduation ceremony. There is no mention of him attending or participating in the graduation in either document. The documents instead mention that in early 1964, Lee dropped out of college and moved to Oakland, California. (...) While this evidence is not definitive, it suggests that Lee probably did not attend or participate in the University of Washington's 1964 graduation ceremony.\n\n\n### References\nYu et al., 2023, [Generate rather than Retrieve: Large Language Models are Strong Context Generators](https://openreview.net/forum?id=fB0hRu9GZUS), ICLR.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The proposed method is simple, intuitive and easy to implement. It also shows strong performance gains in a variety of settings.", "reasons_to_reject": "The authors suggest several explanations for the poor performance on QASC. Further analysis is required to identify and understand why this method fails in these settings.\n\nWhile the proposed method improves in a variety of settings, it's unclear why this is the case. The authors suggest that paraphrasing allows the model to incorporate knowledge that lies outside the knowledge corpus, but there is no analysis justifying this fact. Other possible explanations are that the model in summarizing the passage given the question is (1) doing something akin to chain-of-though prompting or (2) is simply rephrasing the passage into a more usable format, without introducing any additional information. Furthermore, it's unclear whether (3) \"knowledge corpus error\", the suggested reason for this improved performance, is a desirable feature as it can also lead to hallucination. Further analysis is required to determine the source of these gains and justify the author's proposed reasoning.\n\nAnother type of analysis that would strengthen this work is understanding what types of questions this method works for. [1 from missing references] demonstrates that retrieval hurts performance for questions about popular entities. It is very possible that the proposed method is simply mitigating the negative effects of retrieval in such instances, and eliminating retrieval in these instances would achieve a similar effect.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper introduces a new prompting method for performing open-retrieval QA with LLMs. In their method, after performing retrieval the authors use the LLM to first paraphrasing the retrieved passage before using the summarized passage in a separate prompt to perform QA. The authors demonstrate that performing this prompting method improves QA performance across a variety of LLMS (Claude + GPT3) and QA datasets.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "1. Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, Hannaneh Hajishirzi. When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories. The Annual Conference of the Association for Computational Linguistics (ACL), 2023. "}}, {"review_id": "T8sIUT9Nru", "reviewer": "Reviewer_dTRR", "summary": "", "questions": "Question A: Is it possible to distinguish between “good documents, bad reader” and “good reader, bad documents”? It seems you define knowledge corpus errors as the latter, but how could you know if the readers are good enough to extract information from the documents? Or how do you know if you are prompting the reader properly?  \nQuestion B: Per “Reason to reject” 1, how are the hyperparameters decided (e.g. Max token, temperature, etc.)?  \nQuestion C: In the definition of knowledge corpus error (L184), it is mentioned that knowledge corpus error happens when p(a|q, c˜) > p(a|q, c). However, when measuring the errors, the exact match metric is used instead of probability. Is there a reason for this, or could the authors show that the two are equivalent?  \nQuestion D: Is there a reason why an example for QASC is not provided?  \nQuestion E: Why is the training split for StrategyQA used, and how is the subset of HotPotQA selected?  \n", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your valuable feedback. We will first respond to the three main points from “Reasons To Reject”, then answer the queries.\n\n## From \"Reasons To Reject\"\n\n> 1. The experiments are conducted only once, without generating multiple samples or using different hyperparameters. The reason for choosing the hyperparameters are not mentioned.  \n\nThank you for your observation and we apologize for the confusion. We respond to this as part of our answer to Question B. However, we would like to reiterate that the empirical findings are a secondary contribution of the paper after the analytical framework and introduction of the knowledge corpus error framework. \n\n> 2. Lacking detailed analysis: The authors do show that knowledge corpus errors could exist in some settings. However, they never discuss the reasons behind them. (e.g. How are the paraphrased documents different from the gold documents?) Also, the reasons they provide for the accuracy degradation in QASC are weak, and the authors did not provide any experiments to back them up. Seed facts are not necessarily better than Wikipedia documents, and no examples are shown to prove the paraphrased contexts are relevant to the distractors.\n\n### Qualitative analysis on the source of knowledge corpus error\n\nIn §3.3, we posited two factors as the source of knowledge corpus error (= why paraphrasing improves the reader performance). Through further analysis, we identified three factors contributing to knowledge corpus error and have provided illustrative examples for each. \n\n1. **More focus on the question**  \nGold passages are a fraction of Wikipedia, where the written communicative intent is to generally inform about the subject. Therefore, gold passages inevitably include irrelevant information to the question.   \nIn fact, it has been shown that “when both retrieved and generated documents contain the correct answer, the FiD reader can produce more correct answers when reading the generated documents” (Yu et al., 2023 - §4.4). It suggests that retrieved passages are more likely to contain noisy information that is irrelevant to the question.  \nBelow is our example from NQ. The gold passage focuses on general information about the TV series, while paraphrased ones has shifted their focus to the character the actor played, which is directly tied to the question.\n\n\n*Question/Answer*: who plays charles on have and have nots  / Nick Sager\n\n*Gold*:  \nTitle: The Haves and the Have Nots (TV series)  \nThe Road to Hell\", it was revealed that David had a pulse while Erica's fate was a casualty and died from the explosion. Her death was the fuel for David’s warpath of vengeance on Veronica. - **Charles Frederickson (Nick Sager)**: (Season 4–present) The president-elect and the love interest to Candace, whom she uses to help her get her money back. (...)   \n\n*GPT-paraphrased*:  \ncrew of \"The Haves and the Have Nots\": In the TV series, \"The Haves and the Have Nots\", **Charles Frederickson is played by Nick Sager**. He plays the character of the president-elect and Candace's love interest. (...)  \n\n*Claude-paraphrased*:   \n**Charles, played by Nick Sager**, is one of the main characters on the show The Haves and the Have Nots. He is portrayed as the president-elect and love interest of the character Candace. (...)  \n\n---\n2. **Chain-of-thought**  \nSome questions require composition of facts to answer the question. In such case, we saw that paraphrasing is doing something similar to chain-of-thought.  \nBelow is an example from HotPotQA (Table 4 in the paper). Paraphrased passage is combining two distinct facts into a single fact.  \n\n*Question/Answer*: The VMAQT-1 logo is a female spirit in Irish mythology who heralds the death of what?  /  a family member  \n\n*Gold*:   \nTitle: VMAQT-1  \n(...) The VMAQT-1 logo is the Banshee, an Irish mythological figure foretelling death. Its motto is “Tairngreacht Bas,” Gaelic for “Death Foretold.”  \nTitle: Banshee  \nA banshee ( ; Modern Irish \"bean sí\", from Old Irish: \"ban síde\" , ] , \"woman of the fairy mound\" or \"fairy woman\") is a female spirit in Irish mythology who heralds the death of a family member, usually by shrieking or keening. (...)\n\n*Claude-paraphrased*:   \nThe VMAQT-1 logo features a banshee, which is a female spirit in Irish mythology. According to Irish mythology, banshees shriek or keen to foretell the death of a family member. (...)  \n\n---\n3. **Commonsense knowledge**  \nCommonsense knowledge plays a crucial role in understanding the world, but not often explicitly stated (especially in Wikipedia). During paraphrasing, commonsense knowledge is expressed in the passage, which aids the reader to answer better.  \nBelow is an example from StrategyQA. *Someone who dropped out of college will probably not attend graduation ceremony* is an important commonsense knowledge here.\n\n*Question/Answer*: Was Bruce Lee absent from the 1964 University of Washington graduation ceremony? / True  \n\n*Gold*:   \nTitle: Bruce Lee  \nLee is best known as a martial artist, but he also studied drama and Asian and Western philosophy while a student at the University of Washington and throughout his life. (...)  \nTitle: Bruce Lee  \nLee dropped out of college in early 1964 and moved to Oakland to live with James Yimm Lee. (...)  \n\n*Claude-paraphrased*:   \nBased on the background documents, it appears that Bruce Lee was likely absent from the 1964 University of Washington graduation ceremony. There is no mention of him attending or participating in the graduation in either document. The documents instead mention that in early 1964, Lee dropped out of college and moved to Oakland, California. (...) While this evidence is not definitive, it suggests that Lee probably did not attend or participate in the University of Washington's 1964 graduation ceremony.\n\n---\n\n### Analysis on QASC\n\nWe apologize for the lack of clarity in our explanation for performance degradation for QASC (§4.3). Here we provide examples to better illustrate our explanation. We will include further details in the final version of the manuscript.\n\n\n1. **Gold passages we used are manually created, instead of being extracted from raw Wikipedia.**  \nGold passages we used represent the best-case scenario in retrieval, thereby eliminating any retrieval error. However, gold passages we used for QASC do not reflect this setting, as they are **manually selected** from cleaned knowledge sources like WorldTree corpus. Such choice was done because there were no annotated supporting passage from raw source such as Wikipedia. An example below illustrates how manually created \"gold\" in QASC differs from other benchmarks.  \n\n\n*QASC*:  Any antigen that causes an allergy is called an allergen. Many of the common allergens are pollens and dusts.    \n*NQ*: Title: Boom (P.O.D. song)  \nBoom (P.O.D. song) \"Boom\" is a song by American rock band P.O.D. It was released in May 2002 as the third single from their second major label studio album \"Satellite\". While it did not chart as well as the album's previous singles, the song has appeared significantly in film and television. \"Boom (The Crystal Method remix)\" was included on the remix album \"Community Service\" and as a bonus track on the special edition re-release of \"Satellite\" available August 27  \n\n---\n\n\n2. **Distractor options in multiple-choice question confuses the model to generate passage relevant to those options.**   \nThis is best illustrated through an example. See the bold parts to see how distractor option (C) misled the paraphraser, GPT.\n  \n*Question*: What are pollens? (A) eukaryotes (B) unicellular organisms **(C) building blocks of organisms** (D) antigens that cause allergy (E) bees (F) microscopic organisms (G) nucleotides composing polymers (H) diploid  \n\n*Answer*: D  \n\n*Gold passage*: Any antigen that causes an allergy is called an allergen. Many of the common allergens are pollens and dusts.  \n\n*GPT-paraphrased passage*: Allergens are substances that cause allergies, and they include common sources such as pollens and dusts.  Pollens are not unicellular organisms or bees, but rather **they are building blocks of organisms** composed of nucleotides that form polymers. They are not diploid or microscopic, but they are known to be antigens that can trigger allergic reactions.  \n\n---\n\n> 3. Unsupported Claims: L308-L311  \n\nThank you for pointing this out.  This stems from the analytical framework in Section 3.  We did include a statement in the limitations section L323-L327 adding a caveat. However,  we can see how it would be unclear.  While we have introduced an analytical framework backed up by empirical results, there are many factors that contribute to this error. We will remove this claim from the final version of the paper, focusing on our concrete contributions instead.\n\n\n> 4. Results on more LMs could be included. Is this phenomenon consistent for all kinds of LMs, including ones that are not instruction-tuned or not trained with RLHF? What about LMs of different sizes?   \n\nFirst of all, we want to distinguish the use of LMs for *paraphrasing* and *reading* (answering questions). LMs used for *paraphrasing* are the means of observation, but LMs used for *reading* are the subjects of observation. The purpose of paraphrasing the gold passage is to obtain passage even better than gold. Therefore, we believe that only using the best LLMs available for paraphrasing is justified.\n\nHowever, we definitely notice the benefit of including more models for readers. This will make our argument more robust. We are especially eyeing on the recent open-source LMs such as Llama-2 and Falcon. We are looking forward to add more LMs, both base and instruction-tuning+RLHFed, in the future version.\n\nWe'd like to further mention that the reader can be a supervised model trained on QA datasets, e.g., FiD (Izacard and Grave, 2021). From our experiment, FiD trained on retrieved passages, $c \\in \\mathcal{Z}$, performed worse when given paraphrased passages, $c \\notin \\mathcal{Z}$ ($\\mathcal{Z}$ refers to knowledge corpus such as Wikipedia). This indicates a distribution shift after paraphrasing. Training FiD on paraphrased passages would be required to see a true effect of paraphrasing for FiD.\n\n\n## From \"Questions For The Authors\"\n### Question A\n> Is it possible to distinguish between “good documents, bad reader” and “good reader, bad documents”? It seems you define knowledge corpus errors as the latter, but how could you know if the readers are good enough to extract information from the documents? Or how do you know if you are prompting the reader properly?  \n\n*Good is in the eye of the reader*. We think of *good* documents as documents beneficial to the reader, i.e., a reader-dependent concept.\n$$\np(a|q) = \\sum p(a|q,c)p(c|q)\n$$  \nFrom above, we can think of the probability on the right-hand side being modeled by different models. It is possible for some $c$, $p_{1}(a|q,c) \\neq p_{2}(a|q,c)$. The case of using different prompts with the same model can be thought equivalent to using different models.  \n\nHowever, as we are not conditioning on the reader model when paraphrasing the context, we are assuming some level of **universality of *good* documents** across various readers. Although this universality is presumable, it will be worth exploring the difference between *good* documents depending on the readers, from LLMs and supervised FiD to human.  \n\nIn fact, we found out that there seems to be some universality of *goodness* across the readers we used. Table 1 shows exact match in HotPotQA for two readers (GPT-3.5 and Claude) . For majority of the examples, two readers accord with each other (both correct or wrong), and this ratio is even higher in paraphraed contexts (70% for gold, 77% for GPT-paraphrased, 81% for Claude-paraphrased).   \n\n| Who is correct? (%)   | Both | Only GPT | Only Claude | None (both wrong) |\n|--------------------|------|----------|-------------|-------------------|\n| Gold               | 27   | 9        | 21          | 43                |\n| GPT-paraphrased    | 33   | 5        | 18          | 44                |\n| Claude-paraphrased | 39   | 4        | 15          | 42                |  \n<Table 1: Exact Match in HotPotQA> \n\n> Question B: Per “Reason to reject” 1, how are the hyperparameters decided (e.g. Max token, temperature, etc.)?  \n\nGeneration hyperparameters (max tokens, temperature, zero-shot vs few-shot) are decided through trial-and-error. For example, max tokens was adjusted according to few preliminary samples. We tried to tweak temperature for QASC after observing deviant result, but only had minor impact. 3-shot setup was chosen for NQ because the performance was too low in zero-shot.  \n\n> Question C: In the definition of knowledge corpus error (L184), it is mentioned that knowledge corpus error happens when p(a|q, c˜) > p(a|q, c). However, when measuring the errors, the exact match metric is used instead of probability. Is there a reason for this, or could the authors show that the two are equivalent?  \n\nThanks for pointing out an important gap. We used EM / accuracy for two reasons.  \n\n1. The reader models we used, GPT-3 and Claude, are black-box APIs where probability cannot be directly known. Therefore, we used end metrics (exact match / accuracy) as proxies. \n\n2. In experiments, end metrics (exact match / accuracy) are far more intuitive than probability. This helps the reader to grasp the effect of paraphrasing. In addition, it allows comparison with other works as these are the most widely reported figures.\n\nWe acknowledge the gap between our analytic formulation (probability) and empirical experiments (exact match / accuracy). However, we believe that this gap is negligible, especially given that we adopt greedy decoding. Furthermore, we believe the aforementioned benefits outweigh this limitation.\n\n> Question D: Is there a reason why an example for QASC is not provided?  \n\nWe apologize for our negligence. In addition to an example provided in #2 above, we here provide an example where paraphrasing was successful.  \n\n---\n*Question*: Climate is generally described in terms of what? (A) sand (B) occurs over a wide range (C) forests (D) Global warming (E) rapid changes occur (F) local weather conditions (G) measure of motion (H) city life  \n*Answer*: F    \n\n*Gold*: Climate is generally described in terms of temperature and moisture.\\n\\nFire behavior is driven by local weather conditions such as winds, temperature and moisture.  \n*GPT-paraphrased*: The climate is typically defined by its temperature and moisture levels. The behavior of fires is influenced by local weather conditions, which may include wind, temperature, and moisture.  \n*Claude-paraphrased*: Based on the background documents, climate is generally described in terms of temperature and moisture or local weather conditions, not the other options.    \n\n*GPT reader given gold / GPT-paraph / Claude-paraph*: X / O / O  \n*Claude reader given gold / GPT-paraph / Claude-paraph*: X / O / O    \n\n---\n> Question E: Why is the training split for StrategyQA used, and how is the subset of HotPotQA selected?  \n  \nWe used the training split for StrategyQA because the dev split contained too few (490) examples. For HotPotQA, we used the first 1531 samples in the dataset. \n\n## References\nIzacard and Grave, 2021, [Leveraging Passage Retrieval with Generative Models\nfor Open Domain Question Answering](https://aclanthology.org/2021.eacl-main.74/), EACL.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. “Knowledge Corpus Error” is a novel concept that can explain current LM errors and why generated context might be superior to retrieved ones in knowledge-intensive tasks. This work could inspire future work in locating errors coming from corpus that are not good enough. \n2. The paper empirically shows that knowledge corpus errors exist in some of the current open-domain question answering settings. \n3. The mathematical formulation of knowledge corpus is clear, easy to follow and mostly correct.\n", "reasons_to_reject": "There are four points in total, ranked from major to minor:\n1. The experiments are conducted only once, without generating multiple samples or using different hyperparameters. The reason for choosing the hyperparameters are not mentioned.  \n2. Lacking detailed analysis: \nThe authors do show that knowledge corpus errors could exist in some settings. However, they never discuss the reasons behind them. (e.g. How are the paraphrased documents different from the gold documents?) Also, the reasons they provide for the accuracy degradation in QASC are weak, and the authors did not provide any experiments to back them up. Seed facts are not necessarily better than Wikipedia documents, and no examples are shown to prove the paraphrased contexts are relevant to the distractors. \n3. Unsupported Claims: \nL308-L311 “We further track down the source of knowledge corpus error and suggest that reasoning and commonsense required in the benchmarks are closely tied to knowledge corpus error.”  \nHowever, the paper’s second limitation (as mentioned in the Limitation section) is “that the exact source of knowledge corpus error is unclarified”. I would suggest editing the last few sentences in the conclusion.\n4. Results on more LMs could be included. Is this phenomenon consistent for all kinds of LMs, including ones that are not instruction-tuned or not trained with RLHF? What about LMs of different sizes?\n", "questions_to_authors": "Question A: Is it possible to distinguish between “good documents, bad reader” and “good reader, bad documents”? It seems you define knowledge corpus errors as the latter, but how could you know if the readers are good enough to extract information from the documents? Or how do you know if you are prompting the reader properly?  \nQuestion B: Per “Reason to reject” 1, how are the hyperparameters decided (e.g. Max token, temperature, etc.)?  \nQuestion C: In the definition of knowledge corpus error (L184), it is mentioned that knowledge corpus error happens when p(a|q, c˜) > p(a|q, c). However, when measuring the errors, the exact match metric is used instead of probability. Is there a reason for this, or could the authors show that the two are equivalent?  \nQuestion D: Is there a reason why an example for QASC is not provided?  \nQuestion E: Why is the training split for StrategyQA used, and how is the subset of HotPotQA selected?  \n", "paper_topic_and_main_contributions": "The paper introduces and formally describes “Knowledge Corpus Error”, which happens when the knowledge corpus for retrieval excludes more relevant content. The authors argue that knowledge corpus errors exist in three out of four selected datasets by showing reader models could achieve higher accuracy / exact match score with paraphrased documents. ", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. I think the \"not in Z\" is not needed in L185. "}, "extra_text": {}}, {"review_id": "X9lDPyNai4", "reviewer": "Reviewer_JpPN", "summary": "", "questions": "- Did you validate or have any conclusions on the generated paraphrases? are they different that what is included in the index?\n- Why was is neccessary to use few-shot in NQ experiments?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks for your thorough review and valuable feedback. We here go through three main points -- analysis on paraphrased passages, contribution of this paper, and use of few-shot in NQ.\n\n## 1. Analysis on paraphrased passages\n\n> The main experiment suffers from a major drawback that it is unclear from the setup of the evaluations how well paraphrasing using a LLM generates good representations, that is, good examples of external knowledge outside of the corpus. The method in the paper does not validate, or convince, that the paraphrases are indeed outside of the corpus knowledge. (Reasons To Reject 1)\n\n> Did you validate or have any conclusions on the generated paraphrases? are they different that what is included in the index? (Question 1)\n\nWe first want to clarify that we are **not** arguing that paraphrasing improves the reader performance as a result of incorporating \"*external knowledge* outside of the corpus\".  \n\nAs the name ***paraphrase*** and the prompts (Table 2 from the paper) suggest, we are not trying to generate new knowledge by LLMs. The role of paraphrasing is to smooth the textual information from the gold passage (= best-case scenario in retrieval). This would likely result in more factual and helpful passages compared to generation from scratch, like GenRead (Yu et al., 2023).\n\n### Qualitative analysis on the source of knowledge corpus error\nIn §3.3, we suggested two factors as the source of knowledge corpus error (= why paraphrasing improves the reader performance). Through further analysis, we identified three factors contributing to knowledge corpus error and have provided illustrative examples for each. \n\n\n---\n1. **More focus on the question**  \nGold passages are a fraction of Wikipedia, where the written communicative intent is to generally inform about the subject. Therefore, gold passages inevitably include irrelevant information to the question.  \nIn fact, it has been shown that “when both retrieved and generated documents contain the correct answer, the FiD reader can produce more correct answers when reading the generated documents” (Yu et al., 2023 - §4.4). It suggests that retrieved passages are more likely to contain noisy information that is irrelevant to the question.  \nBelow is our example from NQ. The gold passage focuses on general information about the TV series, while paraphrased ones has shifted their focus to the character the actor played, which is directly tied to the question.\n\n\n*Question/Answer*: who plays charles on have and have nots  / Nick Sager\n\n*Gold*:  \nTitle: The Haves and the Have Nots (TV series)  \nThe Road to Hell\", it was revealed that David had a pulse while Erica's fate was a casualty and died from the explosion. Her death was the fuel for David’s warpath of vengeance on Veronica. - **Charles Frederickson (Nick Sager)**: (Season 4–present) The president-elect and the love interest to Candace, whom she uses to help her get her money back. (...)   \n\n*GPT-paraphrased*:  \ncrew of \"The Haves and the Have Nots\": In the TV series, \"The Haves and the Have Nots\", **Charles Frederickson is played by Nick Sager**. He plays the character of the president-elect and Candace's love interest. (...)  \n\n*Claude-paraphrased*:   \n**Charles, played by Nick Sager**, is one of the main characters on the show The Haves and the Have Nots. He is portrayed as the president-elect and love interest of the character Candace. (...)  \n\n---\n2. **Chain-of-thought**  \nSome questions require composition of facts to answer the question. In such case, we saw that paraphrasing is doing something similar to chain-of-thought.  \nBelow is an example from HotPotQA (Table 4 in the paper). Paraphrased passage is combining two distinct facts into a single fact.  \n\n*Question/Answer*: The VMAQT-1 logo is a female spirit in Irish mythology who heralds the death of what?  /  a family member  \n\n*Gold*:   \nTitle: VMAQT-1  \n(...) The VMAQT-1 logo is the Banshee, an Irish mythological figure foretelling death. Its motto is “Tairngreacht Bas,” Gaelic for “Death Foretold.”  \nTitle: Banshee  \nA banshee ( ; Modern Irish \"bean sí\", from Old Irish: \"ban síde\" , ] , \"woman of the fairy mound\" or \"fairy woman\") is a female spirit in Irish mythology who heralds the death of a family member, usually by shrieking or keening. (...)\n\n*Claude-paraphrased*:   \nThe VMAQT-1 logo features a banshee, which is a female spirit in Irish mythology. According to Irish mythology, banshees shriek or keen to foretell the death of a family member. (...)  \n\n---\n3. **Commonsense knowledge**  \nCommonsense knowledge plays a crucial role in understanding the world, but not often explicitly stated (especially in Wikipedia). During paraphrasing, commonsense knowledge is expressed in the passage, which aids the reader to answer better.  \nBelow is an example from StrategyQA. *Someone who dropped out of college will probably not attend graduation ceremony* is an important commonsense knowledge here.\n\n*Question/Answer*: Was Bruce Lee absent from the 1964 University of Washington graduation ceremony? / True  \n\n*Gold*:   \nTitle: Bruce Lee  \nLee is best known as a martial artist, but he also studied drama and Asian and Western philosophy while a student at the University of Washington and throughout his life. (...)  \nTitle: Bruce Lee  \nLee dropped out of college in early 1964 and moved to Oakland to live with James Yimm Lee. (...)  \n\n*Claude-paraphrased*:   \nBased on the background documents, it appears that Bruce Lee was likely absent from the 1964 University of Washington graduation ceremony. There is no mention of him attending or participating in the graduation in either document. The documents instead mention that in early 1964, Lee dropped out of college and moved to Oakland, California. (...) While this evidence is not definitive, it suggests that Lee probably did not attend or participate in the University of Washington's 1964 graduation ceremony.\n\n---\n## 2. Limited contribution\n> The paper's contribution is limited. The results of the evaluations just verify the known phenomena that is also addressed by the authors related work, and it quite intuitive to understand since Wikipedia is far smaller than the data large LLMs are trained on. (Reasons To Reject 2)\n\nWe will first discuss our contribution in general, and then address specific limitations mentioned.  \n\nEven though the phenomenon is intuitive to understand and presumable, **we believe our formulation and its follow-up experiment setup are novel**. Our analytic formulation (§3 in the paper) connects the use of generated passage to classic formulation of *retrieve-then-read* in QA. This helps understanding why generated passages can be more beneficial than retrieved passages (Yu et al., 2023). Further, it can serve as a cornerstone to understand the role LLM-generated text plays, compared to pre-existing text. As we mentioned in §2.1, this discourse is becoming increasingly important, as many LMs, especially the smaller open-source ones, are being trained on LLM-generated text.  \n\nWe notice the focus on our evaluations in the comment. While evaluation is an important part, we want to emphasize that those are rather an empirical follow-up to our formulation. We believe future works can utilize it to build further contributions.\n\n> The results of the evaluations just verify the known phenomena that is also addressed by the authors related work\n\nAlthough it is well-understood that LLM-generated text can benefit a lot of tasks including QA, there has been no controlled experiment to measure the limits of existing text (knowledge corpus). Analysis closest to ours is the “readability analysis” from §4.4 in Yu et al., 2023 (more details in \"Qualitative analysis - 1\" from above). However, not all passages containing the answer are guaranteed good, leaving a chance that the gap in reader performance is due to a suboptimal retrieval. In contrast, our experimental setup is focused to observe the inherent limit of the knowledge corpus, disregarding the performance of retrieval.\n\n> it quite intuitive to understand since Wikipedia is far smaller than the data large LLMs are trained on\n\nWe believe that this phenomenon is not mainly about the size of the knowledge corpus vs the size of the data LM is trained on. Instead, we’d like to emphasize the inherent limit any fixed corpus would hold and language models’ generalization capability to output meaningful text beyond it. For further analysis, it will be interesting to use larger non-parametric memory than Wikipedia.  \n\n## 3. Use of few-shot prompting for NQ\n> Why was is neccessary to use few-shot in NQ experiments?  \n\nWe used 3-shot prompting for NQ because EM in zero-shot Claude reader was too low to deduce meaningful results. Performance gap between zero-shot and few-shot is explained by the answer format -- only single entity are counted correct whereas LLMs may output a whole sentence. Note that for such reason, the use of few-shot in NQ is prevalent (Gudibande et al., 2023; Touvron et al., 2023). HotPotQA faces similar problem, but we concluded that EMs were sufficiently high in zero-shot.\n\n### References\nGudibande et al., 2023, [The False Promise of Imitating Proprietary LLMs](https://arxiv.org/abs/2305.15717).  \nTouvron et al., 2023, [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://arxiv.org/abs/2307.09288).  \nYu et al., 2023, [Generate rather than Retrieve: Large Language Models are Strong Context Generators](https://openreview.net/forum?id=fB0hRu9GZUS), ICLR.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper focuses on an important understanding gap of QA systems and LLMs which has not been addressed yet.\n- Evaluation results showing non-corpus memory can lead to increase in model performance, tested positively on 3 out of 4 QA datasets and 2 LLMs.\n- Setup of experiments are detailed well for reproduction and provided in the appendix.\n- Writing style, explanations and focused contribution.", "reasons_to_reject": "* The main experiment suffers from a major drawback that it is unclear from the setup of the evaluations how well paraphrasing using a LLM generates good representations, that is, good examples of _external knowledge_ outside of the corpus. The method in the paper does not validate, or convince, that the paraphrases are indeed outside of the corpus knowledge.\n* The paper's contribution is limited. The results of the evaluations just verify the known phenomena that is also addressed by the authors related work, and it quite intuitive to understand since Wikipedia is far smaller than the data large LLMs are trained on.", "questions_to_authors": "- Did you validate or have any conclusions on the generated paraphrases? are they different that what is included in the index?\n- Why was is neccessary to use few-shot in NQ experiments?", "paper_topic_and_main_contributions": "The work examines the phenomena of why generated passages, instead of retrieved passages from a corpus, can be more effective in the conventional definition of QA tasks. The work sets a target to find out why this gap occurs.\n\nThe gap: the index does not cover all the available knowledge, more relevant data/passages can exist outside of the index space. LLMs can look at the whole information space thus not missing out relevant information. The authors coin the term knowledge corpus error which defines a subspace of the knowledge space which the index of documents does not refer to, potentially missing relevant info when trying to answer a question in a QA task.\n\nThe paper includes one main focused benchmark the aims to evaluate the knowledge corpus error by paraphrasing golden passages of 4 different QA tasks, and evaluating the reading capabilities of 2 LLMs, and compared to setups using the actual gold passages representing the best possible retrieval function. The evaluation results show quite significant improvement in EM and accuracy is 3 out of the 4 tasks.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work provides notable improvements in performance by exploring an interesting question around “knowledge corpus error”. Reviewers expressed concerns about the impression that the method is augmenting “external knowledge” (presumably from your use of “passages outside the knowledge corpus”). We strongly recommend this wording is adjusted and clarified to bring the intended claims in line with the reviewers’ perceived claims. Additional analysis into why and how this works, as suggested by reviewer bwTs would also greatly strengthen this work. However, we do believe overall this set of experiments yield interesting and noteworthy results for a short paper.
Decision: Accept-Findings