RARR: Researching and Revising What Language Models Say, Using Language Models
Year: 2023

Abstract:
Language models (LMs) now excel at many tasks such as question answering, reasoning, and dialog. However, they sometimes generate unsupported or misleading content. A user cannot easily determine whether their outputs are trustworthy or not, because most LMs do not have any built-in mechanism for attribution to external evidence. To enable attribution while still preserving all the powerful advantages of recent generation models, we propose RARR (Retrofit Attribution using Research and Revision), a system that 1) automatically finds attribution for the output of any text generation model, and 2) post-edits the output to fix unsupported content while preserving the original output as much as possible. When applied to the output of several state-of-the-art LMs on a diverse set of generation tasks, we find that RARR significantly improves attribution while otherwise preserving the original input to a much greater degree than previously explored edit models. Furthermore, the implementation of RARR requires only a handful of training examples, a large language model, and standard web search.1

heading: 1 Introduction
text: Generative language models (LMs) and other text generation models are now the backbone of many AI systems. For example, large language models can perform multi-step reasoning (Nye et al., 2021; Wei et al., 2022), generate plans (Ahn et al., 2022), use tools and APIs (Shin et al., 2021; Thoppilan et al., 2022), and answer open-domain questions (Petroni et al., 2019; Roberts et al., 2020).
Despite these incredible advances, state-of-theart LMs still frequently produce biased, misleading,
∗Lead contributors. Please see Contributions section for details. ⋄Work done during an internship at Google Research.
1We release open-source implementations of RARR, the evaluation pipeline, and the evaluation sets at https:// github.com/anthonywchen/RARR.
or unsupported content, colloquially called “hallucinations” (Maynez et al., 2020; Menick et al., 2022). To make LMs more trustworthy, we want to justify each generation by an attribution report (Rashkin et al., 2021; Bohnet et al., 2022) that contains supporting evidence from trusted sources (e.g., encyclopedia or articles) where appropriate.
Most existing LMs, such as those based on sequence-to-sequence architectures, lack a builtin mechanism for attribution. Even retrievalaugmented models (Guu et al., 2020; Lewis et al., 2020), which retrieve relevant documents and then condition on them to generate text, still do not guarantee attribution. Prior work has shown that
16477
retrieval-augmented models generate text that either includes additional information outside the retrieved documents (Dziri et al., 2022), ignores the documents altogether (Krishna et al., 2021), or even contradicts the documents (Longpre et al., 2021). In fact, occasionally ignoring the retrievals can make the models more robust to bad retrievals (Khandelwal et al., 2020), illustrating that end-task performance and attribution are not always aligned.
Instead of constraining LMs to generate attributed text, we propose a model-agnostic approach to improve the attribution of any existing LM: Retrofit Attribution using Research and Revision (RARR). The approach is inspired by works on fact-checking2 where simple research-and-revise workflows are effective at attributing or correcting unattributed claims made by humans (Thorne et al., 2018; Schuster et al., 2021; Thorne and Vlachos, 2021). As shown in Figure 1, after generating text with the LM, RARR does research to retrieve relevant evidence, and then revises the text to make it consistent with the evidence while preserving qualities like style or structure, enabling the revised text to be seamlessly used in place of the original. RARR can be viewed as a retrievalaugmented model where retrieval happens after generation rather than before. This allows RARR to stand on the shoulders of giant LMs without having to modify them to support attribution.
In our effort to expand the scope of Research & Revision models to handle the output of arbitrary LMs, we make the following contributions. First, we formalize the Editing for Attribution task and propose new metrics that evaluate revision models not just on their ability to produce well-attributed revisions, but also on their ability to otherwise preserve original properties of the text. Second, we use these metrics to benchmark how existing revision models perform on various types of LM outputs such as knowledge-intensive statements, reasoning chains, and dialog responses. Finally, we find that existing revision models do not always generalize across many tasks (and were not originally intended to), and therefore propose a new research-and-revise model that leverages the power of few-shot prompting in large language models to robustly generalize across domains.
2In this paper, we generally avoid the term “fact-checking” other than to reference relevant literature, because we only address attribution, and attribution does not entail correctness. Even if a claim is attributed to a particular source, it does not guarantee that the source is “correct” (Menick et al., 2022).

heading: 2 Task formulation
text: We propose the task of Editing for Attribution as follows. As Figure 1 shows, the input to the system is a text passage x produced by a generation model. The output is a revised text passage y along with an attribution report A, which contains evidence snippets e1, . . . , eM that support the content in y. Optionally, the attribution report can contain additional information such as the alignment between evidence snippets and relevant parts in y.
We propose to measure the quality of the revised text y and attribution report A along two dimensions: (1) attribution: how much of the revised text y can be attributed to the evidence in A, and (2) preservation: how much the revised text y preserves aspects of the original text x.

heading: 2.1 Measuring attribution
text: Previously, Rashkin et al. (2021) proposed Attributable to Identified Sources (AIS), a human evaluation framework which considers a binary notion of attribution. Roughly speaking, a text passage y is attributable to a set A of evidence if a
generic hearer would affirm the statement “According to A, y” under the context of y. A system either receives full credit (1.0) if all content in y can be attributed to A, and no credit (0.0) otherwise.
We propose a more fine-grained, sentence-level extension of AIS. We ask annotators to give an AIS score for each sentence s of y, and then report the average AIS score across all sentences:
AttrAIS(y,A) = avg s∈y AIS(s,A). (1)
Since the AIS score is binary, this effectively measures the percentage of sentences in y that are fully attributed to A. When judging each sentence, we also give annotators access to the surrounding sentences and other necessary context, such as the question that the text passage responded to. We also impose the maximum number of evidence snippets in the attribution report A to make it concise enough for both the annotator and downstream users. By manually inspecting 30 examples from our benchmarks, we found M = 5 snippets to be sufficient for full attribution.
During model development, we define an automated metric, auto-AIS (Attrauto), that approximates human AIS judgments. We utilize the natural language inference (NLI) model from Honovich et al. (2022), which correlates well with AIS scores. For each sentence s of y, and for each evidence snippet e in A, let NLI(e, s) be the model probability of e entailing s. We then define
Attrauto(y,A) = avg s∈y max e∈A NLI(e, s). (2)
To improve accuracy, we decontextualize (Choi et al., 2021) each sentence based on the entire context of y before computing the scores. See Appendix B for implementation details.

heading: 2.2 Measuring preservation
text: To measure preservation, we first ask annotators to decide if the revision preserves the text’s original intent (completely, somewhat, or not at all — see Appendix C for exact rubrics). Like AIS evaluation, we give annotators the necessary surrounding context. We define the binary metric Presintent(x, y) to be 1.0 if the revision completely preserves the original intent, and 0.0 otherwise.
However, even if a revision preserves intent, it may still make superfluous modifications, such as reordering words, changing textual style, or including unnecessary additional information (Thorne
and Vlachos, 2021). Different tasks have different requirements for what should be preserved. Here, we desire a simple metric that can be readily computed for many tasks and that generally penalizes unnecessary changes. We thus define a metric based on the character-level Levenshtein edit distance (Levenshtein, 1965) between x and y:
PresLev(x, y) = max ( 1− Lev(x, y)
length(x) , 0
) (3)
This metric is 1.0 if x and y are the same, and 0.0 if y completely overwrites all parts of x. PresLev is generally sensitive to any kind of change, but certainly does not capture all notions of preservation (e.g., preserving rhyme schemes or puns).
We want the revision to preserve the original intent while avoiding superfluous edits. To reflect this, we finally combine the two metrics as
Prescomb(x, y) = Presintent(x, y) · PresLev(x, y). (4) which is 0.0 if the revision changes the intent and equal to PresLev(x, y) otherwise. Since Presintent requires human annotation, we use PresLev as an automated metric for model development.

heading: 2.3 Discussion
text: Optimizing for attribution alone cannot ensure a good revision: for example, an adversarial editor could ensure 100% attribution by simply replacing the input x with the text of any arbitrary retrieved document, which is trivially attributable to itself. Ideally, we want to maximize both attribution and preservation, while navigating any tradoffs between the two. In our experiments, we report both metrics, as well as their harmonic mean (F1AP, analogous to how recall and precision are combined in F1).
We emphasize that this evaluation scheme does not require any “gold” or “reference” edits (unlike many prior evaluations of text revision models), which are often only available for specialized domains. This enables us to broaden the scope to a much wider range of generation tasks.

heading: 3 Approach
text: We now present Retrofit Attribution using Research and Revision (RARR), a simple method for solving the Editing for Attribution task. As illustrated in Figure 2, given an input passage x, the research stage first generates a set of queries {q1, ..., qN}, each investigating one aspect of x that potentially
requires attribution. For each query qi, it retrieves web documents and selects the best evidence snippets {ei1, ei2, . . . }. The revision stage then revises the original text x using the retrieval results {(q1, e11), . . . }, yielding a revised text y.
Most components for RARR are implemented using few-shot prompting (Brown et al., 2020). We use PaLM (Chowdhery et al., 2022) as our language model. Figure 3 shows some few-shot examples we use, while Appendix D lists the full prompts.

heading: 3.1 Research stage
text: Query generation We perform comprehensive question generation (CQGen) which produces a sequence of questions covering all aspects of the passage x that need to be verified and attributed. A similar strategy has been employed to train textplanning models (Narayan et al., 2022). A prompt with six human demonstrations was sufficient for PaLM to adequately learn the task. To increase diversity and coverage, we sample from our CQGen model three times and take the union of the resulting queries.
Evidence retrieval For each query from CQGen, we use Google Search to retrieve K = 5 web pages. We extract candidate evidence snippets from each web page by running a sliding window of four sentences across the page, breaking at document headings. The evidence snippets for each query
are then ranked based on their relevance to the query. For this, we use an existing query-document relevance model trained following Ni et al. (2021), which computes a relevance score Srelevance(q, e) between a query q and an evidence snippet e. We then keep the top J = 1 evidence for each query. The final retrieval result is [(q1, e11), . . . , (q1, e1J), . . . , (qN , eN1), . . . , (qN , eNJ)], where eij denotes the jth evidence for the ith query, and N denotes the total number of queries from CQGen (which can be different for each input x).

heading: 3.2 Revision stage
text: After retrieving evidence, certain parts of x may now be properly attributed, but other parts remain unattributed and should be revised. As illustrated in Figure 2, the revision stage initializes the output y = x. Then for each retrieved (q, e) = (qi, eij), the agreement model checks if the evidence e disagrees with the current output y regarding the issue in query q. If a disagreement is detected, the edit model edits y to agree with e; otherwise, it does nothing. The process continues until all retrievals are processed.
Agreement model The agreement model takes the partially edited passage y, a query q, and the evidence e as input. It then decides whether both y and e imply the same answer to the question in q. This form of question-guided agreement was previously explored by Honovich et al. (2021). We implement this by few-shot prompting PaLM using a chain-of-thought style prompt (Wei et al., 2022), where we ask the model to explicitly state the implied answers for both y and e before producing its judgment about their agreement.
Edit model The edit model is run only if a disagreement is detected. The model takes y, q and e as input, and outputs a new version of y that aims to agree with e while otherwise minimally altering y. We again use few-shot prompting and chain-ofthought, where we ask the model to first identify a particular span in y that needs to be edited before generating the revised y. This helps reduce the editor’s deviation from the current y.3
3The editor occasionally produces large edits that bring the new revision close to e but far from the current y. Since this is rarely desirable, we reject edits with edit distance above 50 characters or 0.5 times the original text length.

heading: 3.3 Attribution report
text: Finally, we select at most M = 5 evidence snippets to form an attribution report A. Note that during evidence retrieval and revision, we may have encountered and used more than M snippets. Our goal is to find a subset of snippets that maximizes coverage over the potentially attributable points in the passage, as represented by the queries q1, . . . , qN . We use the relevance model from Section 3.1 as a proxy for measuring how much an evidence e covers the point raised by a query q. Then, we exhaustively search for A ⊆ {e11, . . . , eNJ} of size at most M that maximizes
Cover(A, q1:N ) := N∑
i=1
max e∈A Srelevance(qi, e). (5)

heading: 4 Related work
text: Fact-checking Our research builds upon works to identify whether a claim is supported or refuted by the given evidence (Thorne et al., 2018; Wang, 2017; Karadzhov et al., 2017; Augenstein et al., 2019; Wadden et al., 2020). In real-world scenarios such as the one which RARR operates in, relevant evidence may not be provided, necessitating retrieval (Fan et al., 2020; Piktus et al., 2021).
Post-hoc editing for factuality Recent work has gone beyond checking the validity of a claim to correcting a piece of text to be factually consistent with a set of evidence via post-hoc editing (Shah et al., 2020; Thorne and Vlachos, 2021; Schuster et al., 2021; Balachandran et al., 2022; Cao et al., 2020; Iso et al., 2020). FRUIT (Logan IV et al., 2022) and PEER (Schick et al., 2022) both implement an editor that is fine-tuned on Wikipedia edit history with the goal of updating outdated information and collaborative writing respectively. Evidence-based Factual Error Correction (EFEC; Thorne and Vlachos, 2021) also implements a full research-and-revise workflow trained on Wikipedia passages (Thorne et al., 2018). A key differentiator of RARR is its ability to edit the output of any generation model without being restricted by the domain, task, or the need for training data.
Measuring attribution A key part of improving attribution is being able to quantify it. Apart from human evaluation (Rashkin et al., 2021), several automated evaluation methods have been proposed. Our work uses an entailment-based metric, which measures whether the referenced evidence entails
PaLM outputs on NQ (factoid statements) Millie Inbetween is a British comedy television series. It premiered on 24 February 2014 on BBC One. The first series was produced by John Yorke and Phil Clymer.
PaLM outputs on SQA (reasoning chains) The highest point of Mount Wycheproof is 70 metres. Edmund Hillary climbed Mount Everest, which is 8,848 metres. So Mount Wycheproof would be a breeze for Edmund Hillary.
LaMDA outputs on QReCC (knowledge-intensive dialogs) When was Welsh social reformer Robert Owen born?   
contextRobert Owen was born on 14 May 1771. . . Did he have another job? In 1810 he moved to Manchester and established a draper’s shop.
Figure 4: Examples of input passages. For QReCC, prior dialog turns are also given as the context.
the output text (Bohnet et al., 2022; Kryscinski et al., 2020; Goyal and Durrett, 2021). A common alternative is to evaluate whether the output text contains the same factual information as the evidence; e.g., by checking if both yield the same answer to the same question (Wang et al., 2020). We use this notion of attribution in RARR’s agreement model rather than for evaluation.
Retrieval-augmented models Models with a retrieval component have seen successes in question answering (Chen et al., 2017; Lee et al., 2019; Nakano et al., 2021), machine translation (Zhang et al., 2018), code generation (Hayati et al., 2018), language modeling (Khandelwal et al., 2020), and other knowledge-intensive tasks (Lewis et al., 2020). Their retrievals are not necessarily attributions (Dziri et al., 2022; Longpre et al., 2021) and typically are not used to revise an existing output. An exception is LaMDA (Thoppilan et al., 2022), a language model for dialog that performs revision by training on human annotations.

heading: 5 Experiments

heading: 5.1 Evaluation setups
text: RARR aspires to be a general-purpose method for improving the attribution of any text generation model in any text domain. We thus construct evaluation benchmarks by taking the task input from three diverse datasets, and prompting different generation models to produce long-form outputs which may contain “hallucinations,” as demonstrated in Figure 4. These long-form outputs serve as input text passages to RARR. We generate 150 development and 150 test passages for each combination of generation model and source dataset.
Factoid statements We prompt PaLM 540B and GPT-3 text-davinci-002 to generate long-form answers to questions from the Natural Questions dev set (NQ; Kwiatkowski et al., 2019). The resulting passages are mostly coherent but often contain factual errors. This setup examines the ability to attribute a diverse range of factoid knowledge.
Reasoning chains Language models can generate reasoning chains to answer complex questions (Wei et al., 2022). We use PaLM and GPT-3 to generate reasoning chains for the StrategyQA train set (SQA; Geva et al., 2021). This setup tests whether the revision model can provide better attribution for intermediate steps of reasoning, while preserving the overall reasoning process.
Knowledge-intensive dialogs We consider the conversational QA task from the QReCC dev set (Anantha et al., 2021). Given the previous dialog turns, which are rounds of questions and answers (Q1, A1, Q2, A2, . . . , Qk), we use LaMDA and GPT-3 to answer to the final question Qk conditioned on the dialog history. The answer tends to be context-dependent, featuring pronouns and implicit references. All dialog turns are given alongside the answer as inputs to the revision model.

heading: 5.2 Models
text: We compare RARR to several systems that have a research-and-revise workflow.
EFEC We consider EFEC (Thorne and Vlachos, 2021) as a representative fine-tuned editor. EFEC fine-tunes a T5-based model to revise text conditioned on multiple evidence snippets using both semi-supervised and fully-supervised approaches. We compare against their fully-supervised approach, which performed best in their experiments. EFEC uses a neural retrieval model (Karpukhin et al., 2020) to retrieve from Wikipedia; however, not all passages in our experiments are supported by Wikipedia articles. To more fairly compare the editing capabilities of EFEC, we instead use the evidence retrieved by our research stages (CQGen and web search). Note that the EFEC editor conditions on multiple pieces of evidence at once, while our editor iteratively conditions on one at a time.
LaMDA LaMDA (Thoppilan et al., 2022) generates responses in three steps: 1) generate a “base response”; 2) generate search queries from the base response; 3) generate a “revised response” conditioned on the base response and retrieved evidence.
Attribution Preservation
Model auto-AIS AIS intent Lev comb F1AP
PaLM outputs on NQ EFEC 45.6 → 64.3 35.4 → 48.3 16.0 39.1 10.4 17.1 LaMDA 39.5 → 49.9 18.3 → 30.4 26.0 39.6 21.1 24.9 RARR 45.6 → 54.9 35.4 → 43.4 90.0 89.6 83.1 57.0
PaLM outputs on SQA EFEC 37.8 → 58.6 24.5 → 51.7 6.0 31.0 3.8 7.1 LaMDA 32.7 → 43.2 15.8 → 27.0 40.0 46.4 33.7 30.0 RARR 37.6 → 45.1 24.5 → 31.5 92.6 89.9 84.6 45.9
LaMDA outputs on QReCC EFEC 19.1 → 47.4 13.2 → 48.7 39.7 39.4 23.7 31.9 LaMDA 16.4 → 36.2 16.0 → 27.1 21.3 24.8 12.0 16.6 RARR 18.8 → 29.4 13.2 → 28.3 95.6 80.2 78.1 41.5
RARR Our model uses few-shot prompting on PaLM 540B for query generation, the agreement model, and the edit model. We use the same prompts for all tasks except when the context comes from a dialog, where we slightly modify the prompts to use the dialog context (e.g., CQGen now maps dialog context + x to queries). The queryevidence relevance model Srelevance is a pretrained T5-large model (Raffel et al., 2020) fine-tuned following Ni et al. (2021) on MS MARCO (Nguyen et al., 2016). See Appendix D for the few-shot prompting strategies and more modeling details.

heading: 5.3 Results
text: For the main experiments, we report results on passages generated by PaLM and LaMDA. Results on GPT-3 passages show similar trends (Appendix A). Table 1 and Figure 5 show attribution and preservation results for each model and dataset. We also report F1AP, the harmonic mean of the two metrics, which is shown as level curves in Figure 5.
RARR significantly improves attribution while preserving most of the original text. In terms of F1AP, RARR is the only method that performs robustly across all three datasets, and significantly outperforms prior methods on NQ and SQA.
We found that RARR is the only method that preserves the original intent of x over 90% of the time — EFEC and LaMDA only manage to preserve the original intent 6–40% of the time. We also see that editing is crucial to improve attribution: if we only retrieve evidence to support the original response x without editing, attribution ranges from the low 10s to mid 30s. After editing, RARR can increase attribution by up to 13% absolute, while changing only 10–20% of the text.
As noted in Section 2, one can sacrifice preservation for higher attribution. EFEC is able to obtain strong F1AP on QReCC by making larger changes to the text in exchange for a higher attribution score. However, it occupies a very different point from RARR on the attribution-preservation tradeoff curve, as visualized in Figure 5.

heading: 6 Analysis

heading: 6.1 Qualitative analysis
text: Human oracle To understand the remaining headroom in our task, we ask: what is the minimal amount of editing needed to make a text passage fully attributed? The answer would depend on the quality of the LM that generated the text as well as the task difficulty. As an approximation, we manually edited 30 examples in our NQ benchmark until we judged them to be 100% attributable. We achieved a preservation score of 88%, which (when combined with 100% attribution) translates to 93.6 F1AP, indicating a significant headroom.
Analyzing the baselines As exemplified in Figure 6, EFEC frequently attempts to summarize the entire passage into one sentence, or drops later sentences. This is likely due to EFEC’s training data, which was limited to single sentences. This behavior generally increases the attribution score, because it is usually easier to make one sentence fully attributable than many sentences. However, in datasets where the claim contains multiple sentences (NQ and SQA), such a behavior yields low preservation scores, and also results in outputs that are less informative. We expect that EFEC could perform much better if its training data were augmented to include multiple sentences. LaMDA Research achieves similar attribution scores to RARR. But as mentioned in Section 5.2, the intent and linguistic style of the output tend to deviate from the input, resulting in lower preservation scores (Figure 6). We emphasize that this is not a purely apples-to-apples comparison since LaMDA was not optimized for preservation. Overall, these experiments are mainly meant to illustrate that prior models were simply not designed for the task of Editing for Attribution, rather than to mark RARR as the best method.
Analyzing RARR For the research stage, the question generation model had comprehensive coverage: a manual inspection of 40 examples shows > 80% with questions that fully cover all aspects of the input text. The retriever was strongest at re-
(a) Correctly revising an entity y: If She Knew What She Wants was written by Henry Roth. e: [en.wikipedia.org] “If She Knew What She Wants” is a song written by
American singer-songwriter Jules Shear and introduced on . . . y′: If She Knew What She Wants was written by Jules Shear.
(b) Correctly revising a number y: God Save the Queen became the British national anthem in 1745. . . . e: [www.britannica.com] The oldest national anthem is Great Britain’s “God
Save the Queen,” which was described as a national anthem in 1825, . . . y′: God Save the Queen became the British national anthem in 1825. . . .
(The year 1745 was when the song was first performed.)
(c) Performing a necessary larger revision y: “It’s My Party” is a song written and composed by American singer-
songwriter and producer Walter Gold. The song was published in 1962 and first recorded by American singer-songwriter, producer and arranger Lesley Gore. e: [secondhandsongs.com] Song: It’s My Party written by John Gluck, Wally Gold, Seymour Gottlieb, Herbert Weiner. . .
y′: “It’s My Party” is a song written and composed by John Gluck, Wally Gold, Seymour Gottlieb and Herb Weiner. The song was published in 1963 and first recorded by American singer Lesley Gore.
(d) Confused by misleading evidence y: In the Heat of the Night is a police procedural crime drama TV series produced
by MGM Television. It aired on NBC from September 8, 1988 to May 3, 1992. . . . e: [metv.com] Watch In the Heat of the Night on MeTV. y′: In the Heat of the Night is a police procedural crime drama TV series produced
by MGM Television. It aired on MeTV from September 8, 1988 to May 3, 1992. . . . (The show did actually air on NBC in the 1980s, but the model fails to realize that MeTV is a re-run much later in time.)
(e) Improved attribution, but reasoning doesn’t adapt to new info (Q: Does Homer Simpson need two hands worth of fingers to count to 5?)
y: Homer Simpson has 5 fingers and 2 hands. Therefore, he does not need two hands worth of fingers to count to 5. He only need one hand to count to 5. e: [refinery29.com] Other iconic animated characters, such as Homer Simpson and his brood, . . . only have four fingers, too.
y′: Homer Simpson has 4 fingers and 2 hands. Therefore, he does not need two hands worth of fingers to count to 5. He only need one hand to count to 5. (Having only 4 fingers per hand, he does need two hands to count to 5.)
Figure 7: Example revisions from RARR, both good and bad. y = partially edited passage; e = evidence; y′ = passage after editing with e.
searching content involving distinct entities (e.g., a movie, a major event, or a person). In contrast, we found significant headroom for better attribution of statements involving generic objects and more abstract claims (e.g. “Video games require electricity.”— since this is obvious to most humans, retrieved articles from the web tend to address related but different topics). We suspect that a significant amount of attribution headroom on our benchmarks would benefit from a better research stage.
For the revision stage, RARR was able to revise many unattributed claims, especially those involving entities and numbers (Figures 7a and 7b). It can also perform larger revisions when necessary (Figure 7c). Moreover, RARR abstains from editing when the claim is already well-attributed: on NQ, among the inputs with near-perfect attribution (preedit AttrAIS > 0.9), RARR does not make an edit in 90% of the cases. However, the system also has several shortcomings. Some erroneous edits arise from misleading irrelevant evidence (Figure 7d). We also observed an interesting challenge when revising reasoning chains, where the model suc-
cessfully revised an incorrect claim, but did not revise subsequent reasoning steps that depend on the earlier claim (Figure 7e). In this case, further editing to improve logical coherence could help.

heading: 6.2 Ablations
text: Ablating query generation RARR uses generated questions as search queries for evidence retrieval. We consider two natural alternatives: using the entire input passage as a single search query, or using each sentence as a search query. For the former, we retrieve J = 3 evidence snippets to make the amount a closer match to other methods.
The results are in Table 2. Using the entire input passage as the query gives poor results, as the retrieved evidence tends to not focus on potentially unattributed parts in the passage. Using sentences as queries gives results closer to the full CQGen, but a closer analysis reveals two caveats.
First, sentences-as-queries are more effective when such sentences “mimic” content on the Web, and are less effective otherwise. In Table 3, we test this by excluding all of Wikipedia from web search results (since many PaLM outputs for NQ have a Wikipedia style). The attribution performance of sentences-as-queries drops significantly, while CQGen is more robust.
Second, sentence-as-queries tends to retrieve passages that may encourage confirmation bias. Consider the example “Georgia is called the Peach State, but California actually produces the most peaches.” Retrieval using sentences-as-queries found an article echoing that California produces the most peaches, while CQGen generated the more impartial query “Which state produces the most peaches?” and found a newer article saying that South Carolina replaced California as the top peach producer. In this case, RARR using CQGen needs to sacrifice more preservation score to edit the text, leading to a lower F1AP score. This underscores that attribution alone cannot measure “correctness” since not all evidence is up-to-date or reliable.
Ablating agreement model We try removing the agreement model, which effectively forces the model to revise the passage based on every retrieved evidence. The results are shown in Table 2. As expected, more revision leads to less preservation score and spurious changes to the text passage, as demonstrated in Figure 8.
Impact on downstream task performance We have measured preservation using the metric de-
fined in Section 2.2. However, another measure of preservation is whether the revised text can still be used to perform the task that it was originally generated for. Following EFEC, we quantitatively evaluate this on short answer tasks NQ and SQA, and we summarize the result in Figure 9.
For NQ, each original text x is a long-form response to a factoid question. To determine whether the revised text y still serves this purpose, we feed the factoid question and y back into PaLM and prompt it to extract a short answer from y. We find
that RARR not only preserves the short answer accuracy but actually improves it by roughly 5%.
For SQA, each original text is a reasoning chain that helps to answer a yes/no question. We feed the SQA question and y back into PaLM and prompt it to output a yes/no answer, and evaluate answer accuracy. Here, we find that increasing attribution comes at a slight cost in downstream task performance: answer accuracy drops modestly for all revision models (up to 2.6%). We suspect that this may be due to noisy retrievals, which sometimes provide misleading evidence (exemplified in Figure 7d). Furthermore, even though revisions can address factoid errors in the passage (e.g., “Homer Simpson has 5 fingers” from Figure 7e), RARR currently does not try to modify subsequent reasoning steps which may no longer be logically entailed (e.g., “He only needs one hand to count to 5”).

heading: 7 Conclusion
text: Language models have developed increasingly good “procedural” knowledge of what should be discussed and how it should be presented, but often struggle to memorize “factoid” knowledge and produce unsubstantiated claims. We proposed RARR, a framework for revising such claims to make them attributable to the researched evidence. From experiments on text passages generated by different models on various domains, we showed that RARR can revise the passages to improve attribution while preserving other desirable properties such as writing style or structure. Furthermore, RARR sits on top of existing generation models without needing to re-design or re-train LMs.
Major headroom still remains, as discussed in Section 6 and the Limitations section. We hope our analysis of RARR would help with developing new approaches for integrating attribution to LMs.

heading: 8 Limitations
text: Limitations of our task definition Depending on the application, attribution and preservation may not deserve equal weight. For instance, if there are multiple acceptable options for the output, such as in a dialog system, we might trade-off preservation for attribution, similar to how LaMDA behaves in our experiments.
Our evaluation metrics also do not measure all aspects of attribution. For instance, some sentences are self-evident and do not require attribution (e.g., “I agree.”) but would be penalized in our evaluation. It is also necessary to note that linguistic assertions have varying scope: for example, there is a difference between “Frozen is a scary movie” and “I got scared watching Frozen” — while expressing a similar sentiment, the former makes a more general statement that many would disagree with, while the latter is scoped to the speaker’s own experience. In some applications, one could even argue that the latter case does not require attribution, since the speaker is their own source-of-truth. In addition to varying scope, utterances can also make assertions with varying levels of directness. For example, according to standard linguistics, “John ate some of the cookies” yields the implicature that John did not eat all of the cookies, even though it is not logically entailed. This raises the question of which implicatures or implied assertions should be detected and attributed, which should be explored in future work. For more nuances, we refer to Rashkin et al. (2021).
For preservation, we wish to explore other properties that should be preserved, such as discourse or logical coherence. Additionally, if the input text passage is completely misguided or flawed, it can be difficult to revise the text without significant changes, which would be heavily penalized by the current metrics.
Limitations of our model While we aspire to improve attribution for arbitrary text, it is clear that RARR is not yet fully general. For example, the current implementation of RARR would not be well-prepared to edit poetry (where preserving rhyme matters) or long documents, primarily because we do not provide examples of such inputs in our few-shot LLM prompts. However, we do believe that future developers may be able to quickly adapt RARR to such tasks by simply changing the prompts. Second, RARR tends to preserve rather
than delete claims that it cannot attribute. Some of these claims genuinely do not require attribution, but others are hallucination and should be removed. Judging whether a claim requires attribution can be subjective and challenging. Finally, our model is computationally costly, since it is based on prompting a large language model. One potential solution is to leverage recent synthetic data generation recipes to train a smaller model (Lee et al., 2021; Schick et al., 2022).

heading: 9 Ethical considerations
text: Partial attribution When RARR is not 100% successful in making text consistent with retrieved evidence, the revised text will be partially attributed. One could identify unattributed parts using either the automated attribution score (AttrAIS) or the relevance scores used to generate the attribution report (Section 3.3). Such information should be presented to avoid misleading readers into thinking that the entire revision is attributed.
Evidence trustworthiness RARR seeks to improve attribution for the output of any generative model. However, even if RARR can attribute content to a particular source, the user must still consider whether the source itself is trustworthy. Even for sources that are traditionally considered “authoritative” (such as an encyclopedia), there may still be factual inaccuracies or biases. This work does not address the question of whether a source is trustworthy, or the related topic of misinformation. While we do not provide a means for judging trustworthiness, the design of RARR does allow for the research stage to restrict its search over a userspecified corpus, based on what the user deems trustworthy.
Conflicting evidence There is also the possibility that some content may be simultaneously supported by certain sources, while contradicted by others. This can easily occur for content involving subjective or imprecise claims. The current implementation and evaluation for RARR does not explicitly address this issue — we adopted a “permissive” definition of attribution, where we consider content to be attributed if there exists any source that supports it. For some applications, a more restrictive definition that requires both existence of supporting sources and absence of contradicting sources would be needed.

heading: Acknowledgments
text: We wish to thank Raphael Hoffmann, Slav Petrov, Dipanjan Das, Michael Collins, Iftekhar Naim, Kristina Toutanova, William Cohen, Sundeep Tirumalareddy, Samer Hassan, Quoc Le and Heng-Tze Cheng for their research mentorship, feedback and support. We are grateful to Hao Zhou and Petr Pilar for helping us experiment with LaMDA and motivating our dialog experiments. We also wish to thank Tal Schuster for pointing us to relevant work in the fact checking literature, and helping us reproduce it. We thank Vitaly Nikolaev, David Reitter and Roee Aharoni for helping us use AIS and auto-AIS. We also wish to thank Jianmo Ni and Honglei Zhuang for developing the query-evidence relevance model we use, Daniel Andor for developing the sentence decontextualization model we use, and Ran Tian for the initial prototype of CQGen. Finally, we thank Kathy Meier-Hellstern, Philip Parham and Diane Korngiebel for their thoughtful feedback on ethical considerations.
Contributions
Luyu Gao: Designed RARR’s few-shot prompting strategies and implemented the first PaLM-based prototype. Analyzed results, and advised on the design of human and automatic evaluation.
Zhuyun Dai: Proposed the evaluation setup of editing long-form generations from PaLM/LaMDA on various QA datasets. Hosted and mentored Luyu Gao (student researcher) in prototyping RARR. Implemented the final models, designed overall experiments, and obtained main results and ablations (together with Ice Pasupat). Contributed many parts of the writing.
Ice Pasupat: Implemented the final models, designed overall experiments, and obtained main results and ablations (together with Zhuyun Dai). Automated experimental infrastructure, conducted error analyses, and oversaw many parts of the paper writing.
Anthony Chen: Developed the automatic evaluation for attribution and preservation and worked with Arun Chaganty to design human evaluation. Developed the open-source implementation (GPT3 RARR), made improvements to prompts, and helped with writing.
Arun Chaganty: Led and implemented all human evaluation. Proposed the two-dimensional attribution + preservation metric (together with
Kelvin Guu). Advised on model design and contributed many parts of the writing.
Yicheng Fan: Worked with Kelvin Guu to develop the first prototype of RARR. Proposed multiple retrieval strategies and implemented the EFEC baseline.
Vincent Zhao: Co-hosted and mentored Luyu Gao (student researcher) in prototyping RARR. Enabled bulk inference for PaLM. Proposed the downstream task evaluation.
Ni Lao: Research mentorship, advising and contributed many parts of the writing.
Hongrae Lee: Research mentorship and advising. Helped integrate RARR with Google Search and evaluate LaMDA.
Da-Cheng Juan: Research mentorship and early design discussions.
Kelvin Guu: Proposed the original research-andrevise concept, implemented the first prototype, initiated the project and involved all collaborators. Implemented baselines (together with Yicheng Fan). Research mentorship, oversaw project coordination and paper writing.

heading: A Additional experiments and analysis
text: Model variance The main experiments in Section 5 are based on a single run. We ran automated evaluation on 3 random runs of RARR, using PaLM outputs on NQ as input passages. The standard deviations of Attrauto, PresLev, and F1AP are 1.2, 0.5, and 1.0 respectively.
Impact of the retriever choice We tried using Microsoft Bing in place of Google Search, with near identical results (< 1% difference).
Impact of model scale Many components in RARR work by few-shot prompting PaLM, a large 540B parameter LM. To assess the benefit of LM scaling, we replaced PaLM 540B with a smaller 62B parameter PaLM. As shown in Table 4, we found that 540B outperforms 62B by a large margin, suggesting that RARR could potentially further improve with even more scaling. We also experimented with keeping the editor stage at 540B while shrinking the query generation stage to 64B — this yielded a relatively small performance drop, suggesting that model scaling is more important for the editor.
Impact of model type Few-shot prompting has proven to be effective for many recent large language models. We try replacing the query generation model, agreement model, and edit model with GPT-3 text-davinci-003. The few-shot prompts were slightly tuned to fit the GPT-3 model. Table 4 shows the results, which are slightly better than RARR implemented with PaLM 540B on all three
datasets. We will release this open-source version of RARR that uses GPT-3 as the backbone.
Results on GPT-3 passages Table 5 shows automated evaluation results on passages generated by GPT-3. The results follow the same trend as the results on PaLM and LaMDA passages.
Challenging domains We report results on tasks where attribution was particularly hard, and significant future work is needed.
We considered news article summaries produced by summarization models from SummEval (Fabbri et al., 2021) (e.g., “John Doe was left homeless when the storms hit Staten Island, New York . . . ”). Results are shown in Table 6. First, we note that the before-edit auto-AIS scores for all models are low. These news article summaries are often about less widely known people and events, which is challenging for retrievers, leading to low attribution. For example, our query generator may ask “where does John Doe live” but get results for a different John Doe. EFEC and LaMDA also face this issue, but instead trade preservation for attribution and rewrite the text to a different topic. This result suggests that using web search with standard question generation methods may fail to capture important context from the input, and is not sufficient for the attribution task.
We also considered long-form explanations generated by PaLM for the ELI5 dataset (Fan et al., 2019) (Table 6). ELI5 was collected from online forums, so many answers tend to have subjective opinions instead of specific entities and facts (e.g., “How do our brains interpret scary music? To me, scary music often sounds a little bit like a person . . . ”), and are thus difficult to attribute. Sometimes the whole output is based on a false premise and needs to be completely rewritten, in which case RARR cannot satisfactorily edit due to our revision threshold (Section 3.2).
Finally, we considered technical explanations to questions from the MMLU dataset (Hendrycks et al., 2021) which covers diverse subjects from social science, humanities, STEM, and others.4 An example input looks like “Every time you remove an edge from a complete graph, you divide it into two connected components. So, a complete graph with 13 vertices must have 12 connected components.” Results are shown in Table 7. RARR im-
4MMLU has questions from 57 subjects; we took 10 random question from each topic and generated answer explanations by prompting PALM 540B.
proves attribution of the explanations on all four categories of MMLU, although the increases are relatively small. We also found that RARR’s performance is low on examples with mathematical reasoning, as these are beyond the capability of the edit model with our current prompt.

heading: B Details on automated evaluation
text: Sentence splitting When computing the attribution score, we use spaCy en_core_web_sm v3.0.0a1 to segment the text passage into sentences.
(More recent models gave similar results.) While each sentence may contain multiple claims that could be attributed independently, there is currently no linguistic consensus on what constitutes a claim. Instead of depending on a particular definition of claims, we use sentences as claims for simplicity and reproducibility. The same segmentation is also used for human evaluation.
Decontextualization We decontextualize each sentence in the text passage before computing the attribution score. We use the model from Choi et al. (2021), which is a T5 model fine-tuned to map the input “[HEAD] [SEP] context and passage [start] sentence [end]” to the output “[OPCODE] decontextualized sentence”, where the OPCODE can be “done” (success), “un” (unnecessary), or “imp” (impossible). We feed the passage’s context (questions for NQ and SQA; dialog context for QRECC) along with the passage itself to the input. We use beam search with beam size 8 and discard any result whose number of tokens differ by more than 4.
NLI model We obtained a newer version of the end-to-end NLI model from the authors of Honovich et al. (2022), which was trained on MNLI, SNLI, FEVER, PAWS, SciTail and VitaminC (Williams et al., 2018; Bowman et al., 2015; Thorne et al., 2018; Zhang et al., 2019; Khot et al., 2018; Schuster et al., 2021). The model is a T5
model fine-tuned to map the input “premise: evidence hypothesis: claim sentence” to either “1” (entailed) or “0” (not entailed). As suggested by the authors, we use the probability of producing “1” as the entailment score.
Comparing human and automated evaluation We conducted correlation studies between human and automatic metrics and found strong Pearson correlation (attribution = 0.74; preservation = 0.62). We visualize the correlation between human and automated attribution scores on NQ and SQA in Figure 10. We found that the AIS scores from human correlate well with auto-AIS scores, with some bias for non-attributed sentences to be judged as attributed by auto-AIS.

heading: C Details on human evaluation
text: To end-goal of RARR is to improve the attribution of generation models through post-editing while preserving the original intent. Attribution and preservation are both subjective properties that may change with even small edits. In the main paper, we present two automatic metrics to conveniently gauge these properties, but rely on a human evaluation as the gold standard. In this section, we describe how we conducted the human evaluation and what instructions and examples annotators were provided.
Rater recruitment and training We engaged with a vendor supplier of full-time crowd workers to recruit human annotators for our task. Annotators were asked to review the instructions below and were provided direct feedback on their
responses during the pilot annotation runs. We had 3 annotators rate each example in the pilot phase to measure inter-annotator agreement, and had a single rater annotate each example afterwards.
C.1 Instructions: Overview In this task you will evaluate the quality of text generated by a system (the “passage”) based on how well it represents information from multiple pieces of “evidence”.
We will be using two categories to evaluate the quality of the passage: Attribution and Intent Similarity. You will evaluate these categories in succession. In some tasks, you will only evaluate Attribution. The task interface will guide you through the flow; you can also see the overall task flow in the diagram below.
Note: The passage may appear very fluent and well-formed, but still contain slight inaccuracies that are not easy to discern at first glance. Pay close attention to the text. Read it carefully as you would when proofreading.
C.2 Instructions: Attribution In this step, you will evaluate how much of the passage is attributable to one or more pieces of evidence (Figure 11).
In the interface, the passage of text and the context in which it was generated is shown on the left, and each piece of evidence is shown on the right. You will use all three (context, passage, evidence) to answer the following question for each sentence in the passage: Is all of the information provided by this sentence fully supported by at least one piece of evidence?
Determining the information provided by the sentence. Three points are key when determining information provided by the sentence:
1. The context and the other sentences of the passage are often critical in understanding the information provided by the sentence.
2. The context should only be used to understand the information provided by the sentence.
3. The evidence should be completely ignored for this step.
Consider the following example:
Context: who plays doug williams in days of our lives
sentences as being fully attributable or not fully attributable by clicking each sentence, and rating each piece of evidence as being useful or not in helping determine attribution of the passage. Annotators were also presented with the context of the generation.
Passage: In the American daytime drama Days of Our Lives, Doug Williams and Julie Williams are portrayed by Bill Hayes and Susan Seaforth Hayes.
In the above example, the meaning of the passage is clear even without seeing the query. But consider another example:
Context: who plays doug williams in days of our lives Passage: he is played by Bill Hayes Passage (interpreted): Doug Williams is played by Bill Hayes in days of our lives
In this case the pronoun “he” depends on the context, but it is clear that the intended meaning of the passage can be reasonably interpreted as “Doug Williams is played by Bill Hayes in days of our lives”. This interpretation is the “information provided by the passage”.
Pronouns such as he/she/it/they etc. are one case where context is needed to figure out the intended meaning of the system response. Here’s another example (given with paraphrases of the information highlighted below):
Context: when is the last time the us lost basketball at the olympics Passage: The last time they lost was in 2004, when Argentina defeated the US 89–79. Most recently, they won gold in 2016. Passage (interpreted): The last time the United States lost basketball at the Olympics was in 2004.
The context should only be used to determine the information provided by the passage; at times, the passage may be about a slightly different topic than the context, for example:
Context: the south west wind blows across nigeria between Passage: The Harmattan is a dry and dusty northeasterly trade wind that blows across West Africa from December to March. It is very dusty because it blows across the Sahara.
Here, the passage talks about a northeasterly wind, while the context asks about a south-west wind, but the passage can be fully understood.
In general, use your best judgment to determine the information provided by the passage. If the passage is hard to understand and you are unsure what the intended meaning of the passage is, mark the sentences as not attributed and enter a comment with an explanation. As one example, take the following:
Context: how many NBA championships did Michael Jordan win? Passage: it is the best team in the NBA
Determining if the information accurately represents the evidence. Two points are key when determining whether the information accurately represents the evidence: When interpreting a piece of evidence, use only the title and text of that specific evidence. Completely ignore the context, passage and all other evidence. Check all the information in a sentence. If only some information is supported by the evidence, mark the sentence as not fully attributable.
Consider the following example:
Context: when did reba mcentire record back to god Passage: Back to God was released by McEntire in 2017. Evidence: “Back to God” is a song performed by American singer, Reba McEntire. It was released as the second single from her 2017 album, Sing it Now: Songs of Faith & Hope, on January 20, 2017.
In the above example, it is reasonable to conclude that the evidence supports all the information in the passage, and we can mark the passage as being fully attributable. But consider another example:
Context: who won the womens 2017 ncaa basketball tournament Passage: South Carolina Gamecocks won the 2017 NCAA Women’s Division I Basketball Tournament. Evidence: The South Carolina Gamecocks defeated the Mississippi State Bulldogs, 67–55, to claim their first-ever national championship.
In this case, while the evidence also mentions the “South Carolina Gamecocks”, it isn’t clear that the national championship being mentioned is indeed the 2017 NCAA Women’s Division I Basketball
Tournament. The passage should be marked as not attributable.
Finally, when the passage contains multiple sentences, evaluate whether each sentence can be fully attributed to one or more pieces of evidence—it is possible for one sentence to be attributed while another is not. For example:
Context: who won the womens 2017 ncaa basketball tournament Passage: South Carolina Gamecocks won the 2017 NCAA Women’s Division I Basketball Tournament. The final score is 67-55. The championship game was held in Dallas, Texas. Evidence 1: The South Carolina Gamecocks defeated the Mississippi State Bulldogs, 67–55, to claim their first-ever national championship. Evidence 2: The 2017 NCAA Women’s Division I Basketball Tournament was played from Friday, March 17 to Sunday, April 2, 2017, with the Final Four played at the American Airlines Center in Dallas, Texas on March 31 and April 2.
The first two sentences cannot be attributed to either evidence for the same reason as the previous example, but the last sentence is fully supported by Evidence 2 and should be marked as attributed.
In general, you should use your best judgment in determining whether all of the information provided by the passage is “an accurate representation of information in at least one evidence”. See Table 8 for additional examples.
We give the following final notes of guidance:
• Marking evidence as useful. When reviewing each piece of evidence, mark it as useful if it helps you judge the attributability of any sentence, and mark it not useful if not. In the above example Evidence 1 is not useful because it didn’t contain enough context to actually help you assess if the passage was attributable, but Evidence 2 was useful.
• Contradicting evidence. Mark a sentence as being attributed if any piece of evidence supports it: if two pieces of evidence contradict each other, but one of them supports the passage, mark the sentence as fully attributable.
• More on the concept of “accurate representation”. We take as inspiration the journalist’s conception of “accurate representation”. For example, take this excerpt on Accuracy in the NPR Ethics Handbook: “When quoting or paraphrasing anyone . . . consider whether the source would agree with the interpretation...” In other words, if you had written the source document, consider whether you would view the system response as an accurate representation of information in that source document.
C.3 Instructions: Intent Similarity In this step, you will evaluate how much similar the passage is to another passage (Figure 12).
In the interface, the passage A and passage B are both text generated by a system—given the same context in which it was generated. You will use all three (context, passage A, passage B) to answer the following question: How similar is the intent expressed by Passage A and Passage B? Please ignore any differences in details.
Two points are key when determining whether the two passages convey the same intent:
1. Judge the similarity solely based on the similarity in the type and quantity of information provided by each passage.
2. Ignore any differences in factual details between the two passages.
Consider the following examples:
Context: who pays medical bills in great britain where does the money come from to pay these bills Passage A: Britain’s National Health Service (NHS) is paid for through general taxation and national insurance. In 2017/18, the NHS budget was £176.5bn. Passage B: Britain’s National Health Service (NHS) is paid for through general taxation. In 2017/18, the NHS budget was £118bn. Rating: Very similar. Passage A is about the same topic as Passage B, with a similar level of detail and style of presentation. They may differ in factual details.
The above example should be rated “very similar” because both passages include information about (1) how the NHS is paid for, and (2) what its budget
Context + Passage Evidences Notes
Context: who played morticia in the addams family tv show The Addams Family is an American animated sitcom TV series. It was first aired on NBC on September 24, 1973. Carolyn Jones played the role of Morticia. 1/ The Addams Family (1973 TV series): The Addams Family is an American animated sitcom adaptation of the Charles Addams comic. The series was produced in 1973 and was rebroadcast the following season. 2/ The Addams Family (TV Series 1964–1966): When The Addams Family went off the air in 1966, network executives in charge of children’s programming for NBC brought them back in 1973 for their own Saturday Morning cartoon show featuring the voices of Carolyn Jones from the original series. While the evidence supports the show being aired in 1973, it doesn’t specify the exact date (September 24, 1973). Similarly, while the evidence mentions Carolyn Jones as being a voice actor, it doesn’t say she played the role of Mortica.
Context: when will the la sagrada familia be finished The La Sagrada Familia is a large Roman Catholic church in Barcelona. It is designed by Antoni Gaudi. It started construction in 1882, and the construction is still going on. The estimated date to finish is 2026. 1/ Sagrada Família - Wikipedia: The Basílica i Temple Expiatori de la Sagrada Família is a church in the Eixample district of Barcelona, Catalonia, Spain, and is currently the largest unfinished Roman Catholic church. 2/ Find Out Sagrada Familia’s Expected Finish Date: Visiting the breathtaking Sagrada Familia today also means witnessing the slow progress towards the completion of the project. Sagrada Familia is now expected to be completed in 2026, the centenary of Gaudi’s death. It’s a reasonable inference that La Sagrada Familia is the same as Sagrada Familia, even though the names differ slightly. While Evidence 2 mentions Gaudi, it isn’t clear this is a reference to Antoni Gaudi and further doesn’t say that he designed the church.
Table 8: Additional examples for annotating attribution.
in 2017/18 was, though they differ in their actual answers to these questions.
Context: who is the owner of reading football club Passage A: Reading’s owner is Yongge Dai. Yongge Dai is also the president of Chinese company Dai Yongge Real Estate. Yongge’s son, Dai Xiu Li, is Reading’s vice-president. Passage B: Reading’s owner is Dai Yongge. Yongge’s brother and sister pair behind the Reading FC takeover—Dai Yongge and Dai Xiu Li—has made their fortune through a massive property empire. Mr Dai, has been the chairman of Renhe Commercial since 1999, which is an organisation owned by his sister behind a vast network of underground shopping centres in China. Rating: Somewhat similar. Passage A is about the same topic as Passage B, but differs substantially in level of detail or style of presentation. They may differ in factual details.
The above example should be rated “somewhat similar” because both passages are still about the same topic—Reading’s owner— but differ substantially in the information they discuss: Passage A includes information about (1a) who Reading’s owner is, (2a) which company they are the pres-
ident of and (3a) who their vice-president is. In contrast, while Passage B shares information about (1a), it also includes information about (2b) how the Reading owner made their fortune, (3b) their company position and how long they held it for and (4b) what the company also owns.
Context: what is the numbers of total elected member of indian parliment in present time Passage A: The total number of elected members of the Lok Sabha is 543. Passage B: The total number of elected members of the Rajya Sabha is 238. Rating: Not at all similar. Passage A is about a significantly different topic than Passage B.
Even though the passages look very similar, the above example should be rated “not at all similar” because the two passages are about significantly different topics: “the Lok Sabha” vs “the Rajya Sabha”.

heading: D Details on the model
text: Few-shot prompting with LLMs We implement many sub-tasks within RARR using fewshot prompting of LLMs (also known as in-context learning (Brown et al., 2020)) as follows:
1. For each sub-task, we manually author a small number of training examples:
(inputj , outputj) for j = 1, . . . , J , where J ranges between 5 and 10 and where both the input and output are strings.
2. We form the following prompt: input1 ⋄ output1⊕input2⋄output2⊕. . .⊕inputJ ⋄ outputJ ⊕ new_input, where ⋄ denotes a newline character and ⊕ denotes a double newline character.
3. To perform inference on a new input, we condition the LLM on the prompt and sample continuations of the prompt up until the next double newline character.
All of our prompts are included in Figures 13, 14, and 15. The contextual version used for QReCC are in Figures 16, 17, and 18.
Model statistics We implemented most parts of RARR with the PALM model which has 540B parameters. We prompted PALM without any training or finetuning. We used a TPU v2-128 to run inference with PALM.
We manually wrote our prompts by eye-balling quality on a dozen of examples from a separate validation set. We tune our hyperparameters on the validation set as well. We used sampling temperature 0.7 for all generation tasks. For each input text, we sample 3 question generations, and for each question we retrieve 5 results. For agreement gate and editing, we only sample 1 generation. We reject an editing if the edit distance is more than 50 characters or more than half of the original text length.

heading: E Details on the dataset
text: As explained in Section 5.1, we generated 150 development and 150 test passages for each of the 6 combinations of dataset and model: (NQ, PaLM), (SQA, PaLM), (QReCC, LaMDA), (NQ, GPT-3), (SQA, GPT-3), (QReCC, GPT-3). Figures 19, 20, 21, and 22 are the few-shot prompts used to generate the passages.
Following the corresponding datasets, all generated passages are in English. The authors have manually looked through most of the data and found no personal identifiers.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
8
3 A2. Did you discuss any potential risks of your work? 8,9
3 A3. Do the abstract and introduction summarize the paper’s main claims? 1

heading: 7 A4. Have you used AI writing assistants when working on this paper?
text: Left blank.
B 3 Did you use or create scientific artifacts? 2,3,5
3 B1. Did you cite the creators of artifacts you used? 2,3,5
3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? E
3 B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? 5
3 B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? E
3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? 5,E
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. 5
C 3 Did you run computational experiments? 5,6
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? 5,D
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? 5,D
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? 5
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? 5,B,D
D 3 Did you use human annotators (e.g., crowdworkers) or research with human participants? 5,C
3 D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? C
3 D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? C
3 D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? C
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? Not applicable. Left blank.
3 D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? C

heading: Future Work
text: Challenging domains We report results on tasks where attribution was particularly hard, and significant future work is needed. We considered news article summaries produced by summarization models from SummEval (Fabbri et al. , 2021) (e. g. , “John Doe was left homeless when the storms hit Staten Island, New York. ”). Results are shown in Table 6. First, we note that the before-edit auto-AIS scores for all models are low. These news article summaries are often about less widely known people and events, which is challenging for retrievers, leading to low attribution. For example, our query generator may ask “where does John Doe live” but get results for a different John Doe. EFEC and LaMDA also face this issue, but instead trade preservation for attribution and rewrite the text to a different topic. This result suggests that using web search with standard question generation methods may fail to capture important context from the input, and is not sufficient for the attribution task. We also considered long-form explanations generated by PaLM for the ELI5 dataset (Fan et al. , 2019) (Table 6). ELI5 was collected from online forums, so many answers tend to have subjective opinions instead of specific entities and facts (e. g. , “How do our brains interpret scary music? To me, scary music often sounds a little bit like a person. ”), and are thus difficult to attribute. Sometimes the whole output is based on a false premise and needs to be completely rewritten, in which case RARR cannot satisfactorily edit due to our revision threshold (Section 3. 2). Finally, we considered technical explanations to questions from the MMLU dataset (Hendrycks et al. , 2021) which covers diverse subjects from social science, humanities, STEM, and others. 4 An example input looks like “Every time you remove an edge from a complete graph, you divide it into two connected components. So, a complete graph with 13 vertices must have 12 connected components. ” Results are shown in Table 7. RARR im-
4MMLU has questions from 57 subjects; we took 10 random question from each topic and generated answer explanations by prompting PALM 540B. proves attribution of the explanations on all four categories of MMLU, although the increases are relatively small. We also found that RARR’s performance is low on examples with mathematical reasoning, as these are beyond the capability of the edit model with our current prompt