Understanding Retrieval Augmentation for Long-Form Question Answering

Venue: colmweb_org_COLM_2024_Conference
Published: 2024-03-22 00:07:31

Abstract: How retrieved documents are used in language models (LMs) for long-form generation task is understudied. We present two controlled studies on retrieval-augmented LM for long-form question answering (LFQA): one fixing the LM and varying evidence documents and the other fixing evidence documents and varying the LMs.
We study various attributes of generated answers (e.g., fluency, length, variance), with an emphasis on the attribution of generated answers to in-context evidence documents. We collect a dataset (SALAD) containing human annotations of sentence-level answer attribution in LFQA and evaluate existing methods for automatically judging attribution.
We find that while LMs can leverage relevant in-context documents, the generated answer is only partially attributable towards the documents, especially for LMs trained without retrieval augmentation.
Together, our analysis reveals how retrieval augmentation impacts long knowledge-rich text generation and provide directions for future work.

Main paper: 
 introduction: 
 Long-form question answering (LFQA) is designed to address any type of question that could be asked. Instead of extracting spans in the evidence document, LFQA systems generate paragraphlong, complex answers to questions by leveraging parametric knowledge in large language models (LLMs) and retrieved documents provided at inference time. In recent years, we learned surprisingly impressive -yet brittle (Ji et al., 2023; Liu et al., 2023b) -LFQA capabilities of large-scale LLMs.Recent work (Nakano et al., 2021) proposes retrieval as a powerful tool to provide up-to-date, relevant information to LMs. Yet, our understanding of how retrieval augmentation impacts generation in LMs is limited, and retrieval augmentation does not always affect LMs the way we anticipate. Liu et al. (2023a) discovered how information placed in the middle of contexts is not used by LMs and a line of work (Chen et al., 2022; Longpre et al., 2021) showed parametric knowledge continues to affect generation even when relevant document is provided in-context for factoid QA task.We study how retrieval impacts answer generation for LFQA, a complex long text generation task. We present two controlled study settings (illustrated in Figure 1 ): one fixing the LM and varying evidence documents and the other fixing evidence documents and varying the LMs. As evaluating the quality of LFQA is notoriously difficult (Krishna et al., 2021) , we start our analysis by measuring surface features (e.g. length, perplexity) that correlate with specific answer qualities such as coherence (Xu et al., 2023) . One desirable property of retrieval augmented LFQA system is whether the generated answer can be attributed to provided evidence documents. To evaluate this, we newly collect human annotations on sentence-level attribution (Rashkin et al., 2021) and evaluate off-the-shelf models for detecting attributions (Schuster et al., 2021) on our collected dataset (Section 7).Our analysis on surface patterns reveals that retrieval augmentation changes LM's generation substantially. Some effects, e.g., change in the length of generated answers, were pronounced even when provided documents are not relevant. Relevant in-context evidence documents lead to more substantial changes, leading LMs to generate more unexpected sentences (measured by higher perplexity), while irrelevant document does not have the same effects. The impact of retrieval augmentation, even with the same set of evidence documents, can result in opposite effects for different base LMs.We provide an in-depth analysis of attribution with our newly annotated dataset, which can serve as a benchmark for evaluating attribution. We observe NLI models that performed well in detecting WebGPT GPT-3 AlpacaQuestion: If I need pain relief, how do I know whether to choose between aspirin, Tylenol, Advil, etc.?• Aspirin and ibuprofen are both NSAIDs which work by inhibiting the body's cyclooxygenase (COX) enzymes to reduce pain and swelling. … • Therefore, if you are suffering from a fever, acetaminophen would be the better choice. …•Answer: • Aspirin, ibuprofen, and naproxen are NSAIDs that reduce pain, swelling, and fever by inhibiting the body's COX enzymes. • Acetaminophen is an analgesic that works in the heat center of the brain to reduce fever and also affects serotonin pathways in the central nervous•Aspirin is an NSAID (nonsteroidal antiinflammatory drug) and is generally used to reduce inflammation and pain. • Tylenol is an acetaminophen and is used to reduce pain and fever. • Advil is a combination of ibuprofen and acetaminophen and is used for both pain relief and fever reduction. …•Aspirin is an NSAID that works to reduce inflammation and pain. • Tylenol is an acetaminophen-based drug that works to reduce pain and fever. attribution in factoid QA (Bohnet et al., 2022) performs competitively in LFQA setting as well, significantly outperforming chance, yet falls behind human agreement by 15% in accuracy. Our study reveals that attribution quality varies significantly across base LMs, even when they are provided with the same set of documents. Further, we find that a model Nakano et al. (2021) that are trained with retrieval augmentation are more faithful to the evidence documents, and that LMs can ignore irrelevant evidences when needed. We provide new insights on attribution patterns for long text generation. For instance, the last generated sentence is substantially less attributable than earlier sentences, and the generated text roughly follows the order of the in-context evidence documents, even when the in-context document is a concatenation of multiple documents. Taken together, our analysis provides a better understanding of how LMs use in-context evidence documents for long-form question answering and concrete directions for future work in this domain. 
background and related work: 
 LFQA (Fan et al., 2019; Stelmakh et al., 2022) requires models to generate paragraph-length answers to complex, open-ended questions. Combining the challenges of information retrieval and text generation, LFQA remains difficult and an under explored topic in NLP. Prior work (Krishna et al., 2021) suggested retrieval augmented models largely ignore retrieved documents during generation. More recent work, WebGPT (Nakano et al., 2021) , introduced a web agent that searches the web and integrate the information to LMs. We evaluate the behavior of this model closely.Retrieval augmented generation has received attention as a way to provide up-to-date, relevant documents to language models at inference time (Ram et al., 2023) , showing consistent performance gains across multiple tasks (Shi et al., 2023) . A line of work investigates how LMs incorporate in-context documents (Mallen et al., 2023; Liu et al., 2023a) with their parametric knowledge on simpler tasks such as factoid QA. Wang et al. (2023) studies the impact of retrieval in open-ended text generation with kNN-LM (Khandelwal et al., 2019) . In this work, we focus on LFQA, which requires factual, attributable generation over long sequences of tokens.Our study put an emphasis on attribution of long-form answers with respect to the prepended evidence document set. We follow the AIS framework Rashkin et al. ( 2021), an evaluation framework for whether a system generated text can be derived by a given knowledge source. Bohnet et al. (2022) and Yue et al. (2023) study automatically evaluating attribution; the former uses entailment models, while the latter prompts LLMs. Gao et al. (2023b) builds QA models that generate text along with citations and evaluates the citation quality of the generations automatically. Related to our work, Bohnet et al. (2022) presents a controlled study of attribution (e.g., varying evidence documents and how it impact attribution) on factoid QA with Wikipedia as retrieval corpus.Recent work (Liu et al., 2023b) annotates attribution quality in long-form answers generated from commercial generative search engines. While they provide a comprehensive study on attribution quality with manual annotations, their study on black box models is limited, as they do not have knowledge of how the cited documents were integrated into the language models. For instance, cited documents could have been retrieved post hoc (Gao et al., 2023a) . We instead present a controlled study involving open sourced models, and analyze their data in Section 6. 
study setting: 
 We plan a controlled study on how retrieval augmentation impacts long-form answer generation for LMs, observing surface features and attribution while varying evidence document sets and LMs. In this section, we describe our experimental setting.Dataset We source questions from ELI5 dataset (Fan et al., 2019) , which contains questions from the Reddit forum "Explain Like I'm Five". We use the entire test set released by WebGPT (Nakano et al., 2021) (271 questions) for automatic evaluation (Section 4, 7.2), and randomly sample 100 questions to collect manual attribution annotations (Section 5).Knowledge Source: Evidence Documents For each question, we compile four evidence document sets to examine how models use documents of varying degree of relevance. Each document set D contains roughly 3-4 document snippets {d 1 , d 2 , ..., d M }, each snippet containing roughly 100 words. The statistics on each set can be found in Appendix A.1. We describe each document set below:• Human Demonstration Trained annotators from prior study (Nakano et al., 2021) used commercial search engine (Bing) to gather evidence documents to answer questions. We include these as "gold" documents that are considered relevant for answering questions by humans. • WebGPT Retriever We consider documents retrieved by the WebGPT (175B) (Nakano et al., 2021) model. Their study found using these documents result in high-quality answer generation. • Bing Search We retrieve relevant documents using Bing Search API with the question as the query, and obtain the top 10 pages returned by the API, and retrieve four 100-word segments from aggregate search results. Post-processing details can be found in Appendix A.2. • Random To simulate a set of irrelevant documents, we randomly sample another question in the test set and take the corresponding documents retrieved by WebGPT.We evaluate the relevance of first three sets of documents manually by sampling 20 questions and examine the document set of each type. We find that WebGPT and Bing documents contains sufficient information to answer the question for 85% and 45% of the examples, respectively. More details on the manual study is in Appendix B.5.We mainly consider three LMs: WebGPT(175b) (Nakano et al., 2021) , GPT-3 (text-davinci-003) (Brown et al., 2020) and Alpaca (Taori et al., 2023) . The WebGPT model is trained to interact with a commercial search engine (Bing) and compose long-form answers based on the information gathered from the output of the search engine for questions from the ELI5 dataset. 1 We experimented with a range of open-sourced LMs (GPT-J (Wang & Komatsuzaki, 2021) (6B), Flan-T5 (Chung et al., 2022) (11B), Llama (Touvron et al., 2023 ) (7B, 13B, 30B) and Alpaca 7B (Taori et al., 2023) ) and found Alpaca to be the best-performing one upon manual examination. 2The prediction examples for all other LMs can be found in Table 9 in Appendix B.4.We prepend the concatenated evidence document set to the question and provide it as a prompt to LMs with a brief instruction. We sample three answers for each setting to study answer variability.The decoding hyperparameters and prompts can be found in Appendix A.3.  
how in-context documents impact surface answer statistics: 
 Metrics Unlike evaluating short, mostly entity answers (Rajpurkar et al., 2016; Fisch et al., 2019) , evaluating overall quality of long-form answers (Krishna et al., 2021; Xu et al., 2023) is notoriously difficult for both humans and machines. In this section, we look at metrics that has been shown to correlate with specific aspects (e.g., fluency, coherence) (Xu et al., 2023) of answers, to quantify the differences between answers generated from different LMs or with different evidence documents.• Length: We report the number of sentences in the answer as well as number of words. The length is shown as a significant confounding factor in human evaluation for various tasks, with humans often preferring longer answer (Sun et al., 2019; Liu et al., 2022; Xu et al., 2023 ). • Self-BLEU (Zhu et al., 2018) is a metric that measures the lexical diversity of generated text. An answer is less diverse and contains more repetition if it has a higher Self-BLEU score. Prior work Xu et al. (2023) also found that lower self-bleu score correlates to better coherence. • RankGen (Krishna et al., 2022) is an encoder (based on T5-XXL) trained with large-scale contrastive learning, ranking generation given a prefix. Higher RankGen score signifies more likely continuation of the prefix. We measure RankGen score between the question and answers. • Perplexity: We report perplexity of the answer measured with GPT-2-XL (Radford et al., 2019) .Lower perplexity generally indicates more fluent generated text, though human-written texts (Holtzman et al., 2019) do not necessarily exhibit lower perplexity compared to model generated text.Results Table 1 presents the statistics for answers generated with three base LMs with various evidence documents. We present statistics on seven other LMs in Appendix B.4. Overall, prepending relevant documents yields bigger changes for both models compared to prepending random documents.Prepending unrelated documents has little effect on the automatic metrics for Alpaca, but impacts the generation of GPT-3, especially in length and Self-BLEU. This might be related to instruction tuning enables LMs (Alpaca in this case) to be more robust to irrelevant prompts (Webson & Pavlick, 2022) .Using the same set of evidence documents brings different effects on two LMs. On GPT-3, providing documents results in shorter generations and less repetitions, while on Alpaca, it results in longer generations and more repetitions. Yet, on both models, adding relevant documents cause bigger changes in length than adding random documents. Overall, GPT-3 generates longer answers with less variability across examples. Alpaca answers exhibit higher variance across examples, consistently across all metrics.In both models, RankGen scores decrease when document set is more relevant. This can be as model incorporates new information from retrieved documents, generated answers become less predictable from the question alone. Perplexity also shows similar trends, with relevant documents increasing perplexity. This might be because it copies rare tokens from evidence documents, which will get assigned high perplexity when evaluating answer alone.Our finding diverges from Krishna et al. (2021) which showed conditioning on ramdom vs. relevant documents does not bring differences in smaller-scale, fine-tuned retrieval-augmented LM, which fails to incorporate relevant information from retrieved documents into the answer. We will compare attribution patterns in Section 7.2, which again shows substantial difference between two settings. 0.12 0.12 0.12 0.12 0.13 0.12 0.12 0.14 0.14 Figure 2 : Comparing an answer generated without evidence document with an answer generated with diverse evidence document set.Similarities Among Answer Generated with Different In-Context Settings Retrieval-augmented LM combines its parametric knowledge and nonparametric knowledge from evidence documents to address the question (Longpre et al., 2021; Mallen et al., 2023; Zhou et al., 2023) . We aim to understand the impact of combining information from evidence documents on generated answers, as opposed to relying solely on parametric knowledge. We thus compare lexical similarities (measured by bigram overlap) and embedding similarity (measured by SimCSE (Gao et al., 2021) ) between answers generated with various evidence document settings and answers generated without documents.Figure 2 reports the results (results on all answer pairs can be found in Appendix B.1). To contextualize similarity scores, we provide an upper bound (0.19 for bigram, 0.875 for SimCSE) by computing average similarity between three pairs of samples generated without documents, and a lower bound (0.19 for bigram overlap and 0.15 for SimCSE) by computing the similarity between answers to different questions. According to both metrics, the answers generated without evidence document are most similar to the answers generated with random documents, followed by Bing documents, suggesting more relevant evidence set change answers more substantially.0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 Avg Answer Index Fraction Unlike the other two models, We-bGPT was fine-tuned for LFQA with evidence document prepended. This suggests that updating LMs under retrieval augmented setting might be helpful for LFQA, echoing findings from prior work in factoid QA (Bohnet et al., 2022 ) that retrievethen-read systems which are trained with a retrieval component achieve more faithful generation.Unsurprisingly, answers generated without documents (last two rows) are largely irrelevant to reference document set (WebGPT docs). This does not necessarily mean the model generated answers are not factual, as valid answers to the same question can be very different (Krishna et al., 2021; Xu et al., 2023) . Nonetheless, over 20% of sentences were supported by reference documents, suggesting LLMs exhibit level of some parametric knowledge that matches information in the reference documents.Comparing the same base model (GPT-3) provided with different evidence document sets (WebGPT docs vs. Human docs), we find that the model can use WebGPT docs more efficiently. This might be caused by WebGPT evidence documents being longer (about 10%) than human demonstration documents, providing more comprehensive information to copy from. Nonetheless, even with WebGPT docs, 15% of the answer sentences are not supported, suggesting that GPT-3 generates information that lie beyond what can be inferred from evidence documents.Does the order of information presented in the evidence documents impact the order of information presented in the generated answer? If LM is synthesizing information based on content alone, there should be little correlation, considering we provide a concatenated set of evidence documents, not a coherent single document. We plot the correspondence between answer sentence Question: Why does it seem like when I watch something the second time around, it goes by faster than the first time I watched it? Documents: ... Basically, the busier you are during a time interval, the faster that time interval will feel like it passed. ... (more about time goes by faster when you are not bored...) Answer Sentence: However, when we watch something for the second time, our brains have had a chance to process the information and are able to make more efficient use of the information. Explanation:The documents explain why time goes by faster when you are having fun, but the question is asking watching something the second time.Facts (72%): contents that are never mentioned in the documents.Question: How does money go from my pocket, through the stock market, and to support the business I've bought stock from? Documents: Stocks, or shares of a company, represent ownership equity in the firm, which give shareholders voting rights as well as a residual claim on corporate earnings in the form of capital gains and dividends. ... (more about how stock market works) Answer Sentence: You can purchase shares of the stock from a broker or through an online trading platform. Explanation: The documents never mention where you can buy stock from.Incorrect Synthesis (14%) synthesizes the content from separate documents incorrectly.Question: Seismologists: How do you determine whether an earthquake was naturally occurring or if it was human induced? Documents: Studies of the numerous nuclear tests that took place during the Cold War show that explosions generate larger P waves than S waves when compared with earthquakes. Explosions also generate proportionally smaller Surface waves than P waves. Answer Sentence: Natural earthquakes generate larger P waves and smaller Surface waves compared to nuclear tests. Explanation: Explosion generate larger P waves, not natural earthquakes. The answer sentence is thus incorrect. Most of it is copied from the documents. location and their supporting sentences in the evidence document set in Fig. 3 (a)(b) , by aggregating the supporting sentences sets annotated for each answer sentence. We report supporting sentences locations on both answers generated with documents (Fig. 3(a) ) and without documents (Fig. 3(b) ), with the focus on the former and the latter being a reference. We identify linear correspondence on answers generated with documents, with information mentioned earlier in the evidence document appears earlier in the generated answer. This suggests the order of evidence documents will be reflected in the order of generated contents. Recent study (Liu et al., 2023a ) also showed order sensitivity of in-context augmentation for factoid QA, showing that models ignore information in the middle. We also find that later half of the evidence documents, except for the last portion, are less cited by the generated answer (see Avg. column in Fig. 3 ).Which parts of the answer are less supported by the evidence documents? Generated long-form answers consist of 5-10 sentences. Would sentences generated earlier more likely to be supported by evidence documents? Fig. 3(c)(d ) report the percentage of unsupported sentences by the relative position of the answer sentence on our data and attribution annotation on long-form answers from commercial generative search engines from Liu et al. (2023b) respectively. We find that the last sentence is almost twice as likely to be unsupported compared to other sentence in the answer. This phenomenon is even more pronounced on Liu et al. (2023b) .What causes the model to produce unsupported sentences? We manually examine 30 answer sentences labeled as Not Supported for each setting that has access to evidence documents. 3 We identify three categories of unsupported sentences: retrieval failure, hallucinated facts, and incorrect synthesis. 4 Table 3 provides description for each category along with an example. In Table 6 in the appendix, we further provide breakdown of error types for each generation setting. During our analysis, we found that about 14% of errors corresponds to annotation error.We found that attribution error happens more frequently when the retrieved documents do not provide sufficient evidences for answering the question. Generating ungrounded concepts is a more common cause of unsupported sentences than incorrectly synthesizing information from incompatible documents. However, incorrect synthesis happens relatively more frequently in WebGPT model, Under review as a conference paper at ICLR 2024 potentially as it grounds its information more heavily from the documents. This suggests multidocument summarization and synthesis is an important direction for future work, especially for more faithful retrieval-augmented LMs. 
automatically identifying unsupported sentences: 
 Annotating attribution requires careful reading over multiple documents and comparison between two texts. Recent prior work (Bohnet et al., 2022; Gao et al., 2023a; 2022) showed that fine-tuned models from NLI datasets can successfully automate this process. We investigate automatic identification of unsupported answer sentences in LFQA domain with our collected dataset.Setting Given a question q, reference documents D and answer sentence y i , the evaluated system should predict if each answer sentence y i is supported by D. We merge Partially Supported and Not Supported into a single class and consider it as a target label. We report micro average F1 score, which is computed over the set of predictions and labels of all the answer sentences for each generation setting in Section 5 separately, as model performances vary greatly per dataset. We report accuracy in Appendix B.3, which shows similar trends.We evaluate methods for automatically evaluating attribution. First we establish lower and upper bounds, and introduce existing methods. No model is fine-tuned for our task, but we chose one hyperparameter, threshold for deciding supportedness or not.• Baselines As a lower bound, we provide random (which randomly assigns labels for each answer sentence according to the label distribution in each dataset) and majority baseline (which assigns the majority label for all instances). • Human Performance We report the human performance by taking one set of the annotations as the prediction set and another set of annotations as the label set. We compute the F1 score, and take an average across three possible pairs. • NLI models Prior works (Schuster et al., 2022; Laban et al., 2022; Gao et al., 2023b) showed off-the-shelf NLI models can be used to identify generated sentences that are not supported by the evidence document set. We evaluate four NLI model variants: two RoBERTa-large (from Nie et al. 2022)) trained on a combination of NLI datasets. While most NLI models compare a pair of sentences, our setting compares a set of documents (as hypothesis) and a sentence (as premise). For the models except the RoBERTa-large trained on DocNLI (Yin et al., 2021) , we follow Schuster et al. (2022) , which makes entailment decisions for each document sentence and answer sentence pair and aggregates the results by taking the maximum value over all the pairs. The details on training the NLI models can be found in Appendix A.4. • QAFactEval (Fabbri et al., 2022) : is a QA-based factual consistency metric for summarization.It indicates how consistent the generations are with respect to the given documents. We use long-form answer in place of the summary, measuring whether the questions generated from long-form answer are answerable by the given documents.We report model performances in Figure 4a , with each box representing the performance of an approach. Each dot in the box is the score on each answer generation setting. We report the exact scores in Table 7 in the appendix. We find all methods outperform simple baselines (majority, random) by a large margin, but none comes close to human agreements. As in factoid QA setting (Bohnet et al., 2022) , we find that T5 model achieves very competitive performances across datasets, achieving an average F1 over 60 and accuracy over 80%. While developed for a different domain (summarization), QAFactEval performs relatively well.Having discovered that trained T5 model achieves competitive performances in predicting attribution, we use the T5 model as a approximation for human judgement on attribution in generation settings evaluated in Table 1 , following (Gao et al., 2023b; Bohnet et al., 2022) , as human annotations are costly. This complement human annotation results in Section 6. We quantify how frequently the answer sentences are supported by different types of documents using the T5 model.In Figure 4b , we present the results of attribution predicted by the T5 model (along with gold human attribution score if exists). We find answers generated with random documents as evidence (last row in each block) exhibit similar attribution pattern with answers generated without documents (first row in each block). This suggests that models successfully ignore irrelevant documents, and retain similar level of attribution to relevant document, especially for GPT-3 base model. Providing noisy, yet relevant document set (+Bing docs) still does not meaningfully change attribution pattern with respect to the other documents (Human docs, WebGPT docs, Random docs), yet increases supportedness towards provided evidence document set (Bing). Adding WebGPT docs brought the highest change in both models, both in terms of attribution towards other relevant documents (Human) and towards the provided document set. Adding human document also shows similar trends but less impact, potentially as it contains less information than WebGPT docs. 
conclusion / suggestions for future work: 
 We present an extensive study on retrieval-augmented LM generation in the context of LFQA. Our analysis suggests concrete directions for future work. First, LMs trained without retrieval and attribution in mind does not always generate sentences that can be attributed to in-context evidence documents, even when provided relevant documents only. This motivates designing and training LMs after introducing in-context evidence documents. Analyzing patterns of unsupported sentences, we find that injecting faithful multi-document synthesis ability to LLM can be an important direction for future work. Second, we find evidence document should be carefully added to LMs. For example, the order of information presented in evidence documents will impact the order of information presented in the generated answer. And even prepending irrelevant documents meaningfully change the surface statistics of generated answers, though attribution percentage to relevant documents remains somewhat stable. We find attribution error is more common when prepended documents without sufficient information, motivating improving retriever. Also, retrieval failure being one of the main cause of attribution error suggests that the retriever component could be further improved. Third, off-the-shelf NLI models show promising performance at identifying generated sentences unsupported by evidence document, but fall behind human agreements. With our annotated new dataset as well as other related datasets (Liu et al., 2023b) , one can investigate improving automatic attribution methods. Together, we present a comprehensive study of retrieval augmented models for the long-form question answering task, a challenging yet important problem in NLP. Generate a long answer to the following question, using information from the documents. 
 
 Appendix: Answer:A EXPERIMENTAL DETAILSWe report the lengths of each document type in terms of numbers of documents, sentences and words, in Table 4 .We use Bing Search API v7.0. 5 We post-process the raw HTML of the retrieved pages with tools such as html2text 6 and readability 7 . We split each page into 100-word segments, merge segments from all pages, and retrieve the top four segments with BM25 retriever (Robertson et al., 2009) .The prompts we used for answer generation can be found in Table 5 . For Alpaca, we use sampling with a temperature of 0.9, top p = 1 and a maximum length of 1024. For GPT-3, we use sampling with a temperature of 0.7, top p = 1 and a maximum length of 512.Out of the four models, one of the RoBERTa-large is trained on DocNLI (Yin et al., 2021) , which encodes all the documents at once and outputs a prediction. taking the maximum out of entailment scores with every document sentences as the premises, following (Schuster et al., 2022) . More specifically, for each answer sentence y i and document sentence s j , we consider c(i, j) = p(entailed|y i , s j ) to be the entailment score between pair y i and s j . Then we take e i = max sj ∈D c(i, j) to be the entailment score of y i , and consider y i Supported if e i > threshold ϵ. We perform a grid search on ϵ = {0.01, 0.03, 0.05, 0.07, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7} and choose the value that gives the highest F1 score on the test set, given the limited size of dataset. We settle on ϵ = 0.1 for RoBERTa-L (M,S,A,F), ϵ = 0.5 for RoBERTa-L (D), ϵ = 0.2 for ALBERT-xl (M,V), and ϵ = 0.03 for T5.Figure 5 compares answers generated from two LMs (GPT-3, Alpaca) under five evidence settings (including no documents and four evidence types describe in Section 3). The answers generated with random documents prepended are the most similar to answers generated without documents. Answers generated with WebGPT documents are the most similar to ones generated with human documents and vice versa (and thus less similar to the others). This indicate high-quality documents might elicit slightly different behaviors out of LMs compared to when they are relying only on parametric knowledge. Surprisingly, answers generated with Bing documents are the most similar to answers generated without documents.We report the occurrence of each attribution error types for 30 randomly sampled unsupported answer sentences (17 for WebGPT) for the settings with access to evidence documents in Table 6 . We present the accuracy of each evaluate approach in Figure 6 . We also present the exact numbers of F1 score and accuracy in Table 7 . We show the datasets which the models are trained on in acronyms: M -MNLI (Williams et al., 2018) , S -SNLI (Bowman et al., 2015) , A -ANLI (Nie et al., 2020) , F -FEVER (Thorne et al., 2018) , D -DocNLI (Yin et al., 2021) , and V -VitaminC (Schuster et al., 2021) .We report automatic metrics for answers generated by series of GPT-3 models (davinci-001, davinci-002) and other open-sourced models (GPT-J, FLAN-T5-XXL, Llama and Alpaca) in Table 8 . We additionally include generation examples for all the above LMs in Table 9 .We randomly sample 20 questions from the ELI-5 (Fan et al., 2019) test set and annotate if the documents are sufficient for answering the questions. We examine documents retrieved by WebGPT, human demonstration and Bing Search API (the first three settings in Section 3). The results are presented in Table 10 . The WebGPT documents are sufficient for answering the question in the most number of examples (85%), while human documents and Bing documents are less relevant, with only about half of them being sufficient for answering the question. Human documents are often insufficient for answering the questions because human do not cite documents extensively, as shown Question Why did we decide that certain words were "bad" and shouldn't be used in social settings?GPT-J Bad words are used today to express the deepest emotions of a person, such as anger, disgust, fear, or joy. Our English language is full of words that would make an adult in our culture, no doubt, blush. There are some phrases that in the past were considered acceptable to use with people even though, on their own, they are considered, "bad". For example, I'm sure many of us have witnessed and used the phrase "God bless you" without a raised eyebrow or a second thought. However, there are those people, in social situations, who feel the need to state, or even ask, that they were, "praying for you." It could be construed as somewhat odd to someone that a man, who was simply standing in a store eating his sandwich, was "praying for his food." ...Because they convey a negative message about themselves and their behavior.Llama-7B This is because we don't want to offend people. Would you please give me your answer? """ import re # Don't use raw_input()! Type your data into the script, and have the script log it. This way raw_input() doesn't limit the user's input. response = raw_input("Question: Why did we decide that certain words were " "'bad' and shouldn't be used in social settings?" "Answer: This is because we don't want to offend people. " "Would you please give me your answer?") answer = response.strip() print answer # If the user entered something that was not a question, then print a message telling the user that they entered something that was not a question. if not answer: print "You did not enter a question." else: ...Words that are not considered appropriate in social settings are the words that make people feel uncomfortable when they are used, words that make people feel bad, that make people feel sick, that make people feel uncomfortable. Answer: There are many words that are "bad" and shouldn't be used in social settings. They're words that make people feel uncomfortable when they're used. Words that make people feel bad. Words that make people feel sick. Words that make people feel uncomfortable.It has always been socially acceptable to use certain words in private settings. However, when communicating with the public, it has become acceptable to use "clean" versions of those words in order to avoid upsetting the people with whom you are interacting. # Very Simple String Reversal # I worked on this by myself. # I spent 1.5 hours on this. # https://github.com/timothyshen/phase-0/blob/master/week-4/my_solution.rb =begin def reverse(string) string.split(").reverse end reverse("") == "" reverse("cat") == "tac" reverse("race car") == "race car" reverse("the rain in spain") == "ain in Spain raeht" =end # Refactored Solution def reverse(string) string.split(").reverse.join(") end reverse("") == "" reverse("cat") == "tac" reverse("race car") == "race car" reverse("the rain in spain") == "ain in Spain raeht" # Reflection # The major difference in the refactoring was the inclusion of the .join method. This allowed the strings to be joined together after being reversed, creating the complete original string. ...The history of the words "bad" and "taboo" is closely related. The word "taboo" originated from the Tongan and Fijian language and culture and was introduced to English in the 18th century. The concept of taboos were meant to ensure the continuity of a culture by restricting certain behaviors and activities that could bring disruption or harm to a community. Words and expressions that were deemed inappropriate were subject to taboo and avoided in social settings. This was an effort to ensure that people refrained from speaking or engaging in behavior that could cause offense or distress. As time passed, the concept of taboo evolved and some words became increasingly associated with negative energy, misfortune, or danger, and thus became labeled "bad words". This negative association with certain words and expressions is what led to the idea that some words were too "bad" to be spoken in polite company.There is no one answer to this question -it is a complex issue with many contributing factors. Some people might say that certain words are "bad" because they are considered to be impolite or rude. Others might say that these words are offensive or derogatory, and that they should be avoided because they can be hurtful to others. There are also a number of words that have been labelled as "bad" because they are considered to be vulgar or obscene.There are a few reasons why we decided that certain words were "bad" and shouldn't be used in social settings. First, some of these words can be seen as offensive or hurtful to certain groups of people. Second, using these words can make other people feel uncomfortable or unwelcome. Finally, using these words can be seen as a sign of disrespect.The decision to deem certain words as "bad" or inappropriate for use in social settings is based on the belief that certain words can be offensive, hurtful, or disrespectful to the people around us. By avoiding the use of such words, we show respect and consideration to others, and create a more welcoming and inclusive environment. Additionally, certain words can be seen as symbols of certain oppressive systems or ideologies, and using them can make people feel unwelcome or even threatened. In order to create a safer and more welcoming space, we strive to avoid using words that carry negative connotations or are associated with oppressive systems. Furthermore, by avoiding the use of such words, we can foster a more positive and respectful atmosphere, and show consideration and respect to people of different backgrounds and experiences.In general, words that are considered "bad" relate to parts of life that people don't like talking about in public, like bathroom functions.[1] Other words considered bad relate to negative ways of talking about people and their religion, skin color, ability, gender, etc.[1] These words are called slurs and many people consider them the worst kind of bad words.[1] Some words are considered "bad" because they offend others.[2] They are words that refer to impolite things, like body parts or bodily functions and most people don't like to hear about it.[2] Some people believe there are no "bad" words, just inappropriate times and places to say certain words.[2] Question Why do benches empty when there is a fight in baseball?WebGPT Documents Document 1: Bench-clearing brawl -Wikipedia (en.wikipedia.org) A bench-clearing brawl is a form of ritualistic fighting that occurs in sports, most notably baseball and ice hockey, in which every player on both teams leaves their dugouts, bullpens, or benches, and charges the playing area in order to fight one another or try to break up a fight. Penalties for leaving the bench can range from nothing to severe.In baseball, brawls are usually the result of escalating infractions or indignities,[2] often stemming from a batter being hit by a pitch, especially if the batter then charges the mound. They may also be spurred by an altercation between a baserunner and fielder, such as excessive contact during an attempted tag out.Unlike most other team sports, in which teams usually have an equivalent number of players on the field at any given time, in baseball the hitting team is at a numerical disadvantage, with a maximum of five players (the batter, up to three runners, and an on-deck batter) and two base coaches on the field at any time, compared to the fielding team's nine players. For this reason, leaving the dugout to join a fight is generally considered acceptable in that it results in numerical equivalence on the field, a fairer fight, and a generally neutral outcome, as in most cases, managers and/or umpires will intervene to restore order and resume the game. Document 2: Rule 629 | Leaving the Players' Bench or Penalty Bench (www.usahockeyrulebook.com) A major plus a game misconduct penalty shall be assessed to any player who leaves the players' bench or the penalty bench during an altercation or for the purpose of starting an altercation. These penalties are in addition to any other penalties that may be assessed during the incident. Substitutions made prior to the altercation shall not be penalized under this rule provided the players so substituting do not enter the altercation. For purpose of this rule, an altercation is considered to be concluded when the referee enters the referee's crease or, in the absence of penalties, signals a face-off location. Document 3: BASEBALL; A Game of Many Rules Has None on Fighting -The New York Times (www.nytimes.com)The first player to leave either bench or penalty box to join or start a fight is automatically suspended without pay for 10 games. The second player to do that is suspended for five games without pay. The players' teams are fined $10,000 for the first incident, and the coaches of the teams face possible suspension and a fine based on review of the incident by the commissioner.Document 1: Bench-clearing brawl (en.wikipedia.org) A bench-clearing brawl is a form of ritualistic fighting that occurs in sports, most notably baseball and ice hockey, in which every player on both teams leaves their dugouts, bullpens, or benches, and charges the playing area in order to fight one another or try to break up a fight. Penalties for leaving the bench can range from nothing to severe. Document 2: Unlike MLB, the NHL stamped out bench-clearing brawls (www.si.com) Although this particular brawl earned a place in history, a similar sort of all-hands-on-deck silliness remains a fairly regular feature in baseball. Document 3: Bench-Clearing Brawls Just Not The Same Without Amphetamines (www.thebrushback.com)In the glory days of bench clearing brawls, real punches were thrown and real blood was shed, mostly because the players were so incredibly high all the time.Bing Documents Document 1: What does it mean to clear the bench? -TimesMojo not to mention dangerous, for the batter to charge the mound with a bat and has resulted in criminal charges). When was the last NHL bench clearing brawl? The last bench clearing brawl in the NHL was 1987-88. Fifty percent of the players that suited up in the 1980s had at least one fight. Why do benches empty? Most fights in baseball turn into what is known as a bench-clearing brawl. This is when an entire team's bench, sometimes Document 2: What does it mean to clear the bench? -TimesMojo position in the field, then he may return to the mound although that rarely happens. Is bat flipping illegal?Canada and the United States. In Canada, and the United States, bat flips have traditionally been considered rude and inconsistent with baseball etiquette. Traditional etiquette and the unwritten rules of baseball espouse humility and discourage actions which may be interpreted as arrogant or showing up the opponents. Why do catcher's throw to third base after a strikeout? Around the Horn If Document 3: Baseball Fighting Rules -rookieroad.com players that leave their sideline benches will be subject to some pretty hefty fines. The MLB does not feel like it can afford to automatically fine players for joining a fight because of the way the game is structured. In baseball, when there is a fight between an offensive player and a defensive player, the offense is always going to be outnumbered. That is because unless there are offensive players on base, it will always be on Document 4: WATCH: Benches clear between Cincinnati Reds and Chicago ... -Sportsnaut two sides met again in 2018, Garrett got his shot at revenge. With Cincinnati leading 4-2 in the seventh inning, he struck out Baez to end the inning. They stared each other down after Garrett celebrated the strikeout then started exchanging words. After Baez invited a fight, the two rivals charged at one another and the benches cleared out. Related: MLB trade rumors -Latest MLB rumors entering July. Needless to say, there is no love lost between these two. Fortunately, a fight didn't section. Annotations should include the label (whether the answer sentence is Supported , Partially Supported , or Not Supported ), the supporting sentences, and the supported portion if the label is Partially Supported .The collected dataset contains labels of whether each sentence in the answer is suppported by the evidence documents, providing benchmark for studying automatic attribution methods (). We compare our dataset with recent attribution efforts in Table 12 . WICE (Kamoi et al., 2023b ) is a multidocument entailment dataset where the hypothesis is a sub-claim from Wikipedia. AttrScore (Yue et al., 2023) creates data from existing QA datasets using heuristics, and annotates attribution on outputs from generative search engines. Liu et al. (2023b) is the most closest to our work. TheyWhile their model is not released, the model outputs were provided in https://openaipublic.blob.core. windows.net/webgpt-answer-viewer/index.htmlThis is likely because ELI5 was one of the seed task used to generate training data for Alpaca.COLLECTING ATTRIBUTION ANNOTATIONWhile automatic metrics show that in-context documents influence generation substantially, we lack deeper understanding on how the answers change. In this section, we focus on attribution(Rashkin et al., 2021), which measures how much of generated answer can be entailed from the evidence documents. As automatically measuring attribution is nontrivial, we first collect human annotations. We compare our collected dataset with recent attribution datasets in Appendix C.4. Unlike prior work which conduct annotations on full-fledged system without altering evidence documents to the same LM, our annotation presents multiple evidence document for the same base LM.Setup Given a question x, generated answer y, which consist of n sentences y 1 , y 2 , • • • y n and a set of reference documents D, we aim to label each answer sentence y i with one of the following: Supported, Partially Supported, Not Supported by D. If the sentence is Supported or Partially Supported, the annotator also provides a minimal subset of sentences from D that support the answer sentence. Lastly, the annotator highlights the unsupported span if the sentence is Partially Supported.Data CollectionWe collect annotations for 100 questions randomly sampled from the ELI-5 test set on six LM -retrieval document set configurations, namely WebGPT + {WebGPT docs}; GPT-3 + {No docs, WebGPT docs, Human docs} and Alpaca + {No docs, WebGPT docs}. We use the prepended document set as the reference document, and use WebGPT documents for answers generated without documents. We collect three annotations per example as the task is somewhat subjective and take the majority label, discarding 3.4% of examples without majority vote. The interannotator agreement is reasonably high, with Krippendorff's alpha at 0.71. More details about crowdsourcing, including recruitment and disagreement patterns, can be found in Appendix C.INSIGHTS FROM ATTRIBUTION ANNOTATION RESULTSEquipped with manual annotation, we analyze how much of long-form answers can be attributed to evidence documents. Table2summarizes the annotation results. We first compare attribution performance of different models using the same evidence document set (the top section in Table2). We observe that generations from the WebGPT model is most faithful to the evidence documents. When we look at the Alpaca model, even with the same evidence document setting, the percentage of unsupported sentences is ten times more than WebGPT.We analyze all unsupported answer sentences generated by WebGPT, as there are only 17 of them in total.Categories are not mutually exclusive (one can contain irrelevant documents and combine facets from each).An example annotation interface can be found at https://lfqa-test-1.herokuapp.com/id=0. 


Decision: The paper explores how retrieval-augmented language models (RAGs) utilize evidence documents for long-form question answering (LFQA). The study introduces the SALAD dataset, containing human annotations for sentence-level answer attribution, and evaluates existing methods for automatic attribution judgment. The findings reveal that while LMs can leverage relevant documents, the generated answers are only partially attributable to the documents, with unsupported sentences being a significant issue.

The paper is well-executed and offers valuable insights into the use of RAGs for LFQA. Despite some concerns about the clarity of the discussion and the relevance of certain experiments, the comprehensive study and novel findings contribute significantly to the field. Therefore, the paper is accepted with minor revisions. 

To enhance the paper, the authors should provide a clearer explanation of why their results differ from earlier works like Krishna et al., ensuring readers understand and trust their findings. The discussion section should be rewritten for better clarity, making the key takeaways more accessible. Improving the readability of Figure 2 captions is also recommended.

Reviews: review: paper_summary: paper_summary: The paper focuses on the topic of retrieval-augmented long-form question answering (LFQA). The two controlled studies show how the evidence documents or the LLMs impact the generated answers. The performance is evaluated in terms of both surface features and attribution to the evidence documents. It also includes an human-annotated dataset named SALAD, which could facilitate the research in LFQA. Several key findings help the committee better understand the status of retrieval augmentation for LFQA.
strength_weakness: strength_weakness: 1. The size of the dataset is relatively small, which might affect the conclusion of this paper.  How do you select the 100 questions? The question types could affect the results as well.

2. According to the results in FIgure 3b, it is interesting  that the results have a large variance when comparing Human docs and WebGPT docs. It seems Human only works well with Human docs, so does WebGPT (WebGPT docs) and Bing (Bing docs).
questions: questions: 1. Will you open-source the dataset?
2. How do you know GPT3.5  trained without retrieval and attribution? Is it a guess?
paper_summary: paper_summary: This works presents a study on retrieval augmentation for long-form question answering. They perform 2 controlled studies : first they fix the LM and varying evidence documents, second they fix evidence documents and varying the LMs. This is a good setup as it resulted in interesting results/discussion useful for a better understanding of how retrieval augmentation work.

They also present a new dataset -SALAD, containing human annotations of sentence-level answer attribution in long-form question answering.
strength_weakness: strength_weakness: 1. They authors argue that their results differ significantly from earlier works such as Krishna et al. I would have liked a better argument as to why that is happening and we should trust their results as compared to previous works. It might leave readers in a dilemma of whose results to trust.
2. The discussion of results such as in Section 6 is difficult to follow. I understand that the reviewers had a lot of points to discuss and space constraint might have reduced some discussion, but a better written discussion section would help the readers get the key takeaways from this work more easily
3. Figure 2 plot captions are difficult to read and a better readability will def help
questions: questions: 1. Why do you think some of your results differ significantly from other works? Is it because of a different setting or some limitations of earlier works that you solved?
paper_summary: paper_summary: This paper presents a comprehensive analysis and study on retrieval-augmented long-form question answering (LFQA). Different from short answers, the LFQA receives less study on how the retrieved documents affect the long-form answers in various perspectives, which is the focus of this work. Through extensive experiments, the authors made several observations including different LMs' capabilities towards attributing their generated answers to the retrieved contexts, the effect of different sets of documents under retrieval for common metrics such as perplexity, self-BLEU, etc, and comparing different automatic models for fine-grained attribution evaluations.

Overall, this paper is well written and clearly delivers the objective. The LFQA problem is indeed an under-explored area with much room for improvement. The findings given from this work provide useful insights and suggest potential directions for future work. The authors also conduct a comprehensive study with extensive experiments, testing various perspectives ranging from different LMs, different sets of retrieval corpus, diverse metrics, and attribution evaluations, which make the conclusion more convincing.

A few limitations are observed:
1. Some of the experiments/analysis are not so constructive and hence are less meaningful from my understanding. For example, Table 1 compares RankGen which focuses on the correlation between the question only and the generation. This does not seem to be relevant under the retrieval-augmented setting, because the answer could incorporate more contents from the context, making it less corresponding with the question. And the results do show this pattern, without any surprise. Further, two of the metrics in Table 1 demonstrate inverse effect given by relevant documents compared to no or random documents. I do not see how this could bring any meaningful insights from these results. Other more relevant metrics could be considered such as measuring the coherence, coverage, diversity etc.
2. The last observation in Section 4 could be made more precise and meaningful - The divergence between this work and Krishna et al. (2021) is probably due to the model capacity. One possible conclusion made is that higher-capacity LMs are more robust to random contexts and pay more attention to relevant context. To better show this point, it could be more interesting to evaluate smaller LMs for emprirical comparison.
3. Some experiments in Table 2 also seem less meaningful. I don't see the value of conducting experiments without documents and evaluate the supportedness. First, the documents from WebGPT might not be relevant for the question. Second, since the documents are not presented, it is not fair to force the model to generate answers relevant to the documents.
4. The setting in section 7.1 merge partially supported with not supported. This is not entirely intuitive to me. Is there any reason why the partially supported should be treated as the negative class for NLI? What if this is merged with the supported label?
strength_weakness: strength_weakness: Some of the experiments and conclusions are not constructive.
Some experimental settings and observations are not clearly explained.
The dataset being evaluated is less diverse.
questions: questions: The setting in section 7.1 merge partially supported with not supported. This is not entirely intuitive to me. Is there any reason why the partially supported should be treated as the negative class for NLI? What if this is merged with the supported label?
paper_summary: paper_summary: This paper looks at how in-context examples affect long-form answers (more than a word or phrase) in LLM generation.  The authors vary both the model and the documents given, and measure aspects of the answer.  They find that the answers are impacted by the documents given (even by random documents), and answers are often related to the input documents (even corresponding to the order of the documents given), but answers are not always attributable to the documents (unless the model was trained for that behavior).  They evaluate automatically using several statistics that indicate higher quality (e.g., looking at fluency, length, variance), but no automatic metric is used that really targets a correct answer.

This paper covers and interesting topic and delves into it a bit, but the description of the task and the dataset (SALAD) leaves me a bit unclear.  Are the answers in the SALAD dataset given scores of correctness or just how much their answer is supported by the source document?  I would like to understand exactly what is in the SALAD dataset and how it is useful.  Is it useful for evaluating new LLMs or only the LLMs used to generate it?

Notes:
- Figure 1 references sections 3 and 5 for attribution, but I think it should be sections 5 and 7 (as stated in the caption to the same Figure)
- Figure 2 - the heat maps are hard to interpret.  What are they intended to show... just an idea of approximately where the answers are coming from?  Also the text of parts (a) and (b) runs together... would be more readable if spaced farther apart.

[After author discussion]
I am raising my score to 7 - Good paper, accept
strength_weakness: strength_weakness: - Lack of clarity with methodology and dataset [Addressed in Author Discussion]

- Weak automatic evaluation method for most LLMs (only WebGPT, GPT-3.5, and Alpaca get human evaluations)
questions: questions: - Are the answers in the SALAD dataset given scores of correctness or just how much their answer is supported by the source document?  I would like to understand exactly what is in the SALAD dataset and how it is useful.  Is it useful for evaluating new LLMs or only the LLMs used to generate it?

- What are Figure 2's heat maps intended to show?

Comments: title: Response
Thank you for clarification
Response to authors’ rebuttal
Gentle Reminder for Discussion
comment: Thanks for the clarification.
I am clearer on the paper now, I will raise the score to 7.
I thank the reviewers for their response and the paper would definitely merit from including discussion regarding Krishna et al. I am inclined towards keeping my score and think this paper should be accepted.
I appreciate the authors’ replies to all my questions. I will keep my original evaluation and am leaning positive towards the acceptance of this paper.
Hi! We thank all the reviewers for your reviews and hope that you have read our rebuttal. Do you have any thoughts regarding our rebuttal?