Analysis of Plan-based Retrieval for Grounded Text Generation (2024)

Abstract: In text generation, hallucinations refer to the generation of seemingly coherent text that contradicts established knowledge. One compelling hypothesis is that hallucinations occur when a language model is given a generation task outside its parametric knowledge (due to rarity, recency, domain, etc.). A common strategy to address this limitation is to infuse the language models with retrieval mechanisms, providing the model with relevant knowledge for the task. In this paper, we leverage the planning capabilities of instruction-tuned LLMs and analyze how planning can be used to guide retrieval to further reduce the frequency of hallucinations. We empirically evaluate several variations of our proposed approach on long-form text generation tasks. By improving the coverage of relevant facts, plan-guided retrieval and generation can produce more informative responses while providing a higher rate of attribution to source documents.

## introduction large, parametric language models (llms) provide highly fluent text for many applications such as summarization, dialogue, and translation @xcite @xcite @xcite . among the errors made by these models, producing generations with factual and/or grounding errors, often referred to as hallucinations, limit the broader applicability and capability of language models @xcite @xcite @xcite . hallucinations differ from other kinds of errors in that the generated text is syntactically correct and semantically plausible. these hallucinations are generations that, were they factually accurate, would be satisfactory model output. as an example of such hallucinations, we consider the following generation from the falcon 180b model demo foot_0 for the query write a bio about the author lorrie moore. while the model accurately generates the book titles, it hallucinates many facts (red, bold-faced) about moore such as the place of her birth (she was born in glens falls, new york), the university of her mfa (she received an mfa from cornell university). the model also confuses all of the awards given to the author (incorrect years, award names, missing awards). recent work on detecting and mitigating such hallucinations has included modeling uncertainty @xcite , constrained decoding @xcite , specially trained or auxiliary model-based decoding @xcite , question-answering planning @xcite , and retrieval-based verification of claims @xcite . these methods demonstrate success by leveraging the model's ability to accurately produce factual information via its parameters or through text provided in-context. however, understanding and analyzing where and why such hallucinations appear and why providing evidence in-context can successfully mitigate hallucinations has only begun to be studied recently @xcite @xcite . in this work, we study how planning can be used to guide retrieval to improve factual text generation. we hypothesize that language models hallucinate when they are required to generate certain facts based on the prompt but do not have the information either memorized or in their context. thus, we hypothesize that with specific, comprehensive facts in-context can mitigate hallucinations. our investigation considers the following questions: q1. how to effectively discover the comprehensive collection of facts needed to generate text about a particular subject? ( §3.2, §3.3, we demonstrate the generality of the approach in writing grounded text by evaluating model responses in two domains: writing biographies and writing event descriptions. in addition to entities and events that may have a significant internet presence, we specifically consider 2 settings that would challenge llms: (1) writing biographies for people (researchers) who do not have wikipedia profiles (long-tail, low-frequency entities) and (2) writing about current news events (outside of the parametric knowledge of the llms). ## related work the focus of our paper is to provide analysis as to how retrieval augmented language models can reduce hallucinations by conditioning on relevant facts in-context. this work abuts many related areas in text generation (factuality-focused and otherwise), in-context learning, retrieval, and more. refer to other related work in app a.3. verifying attribution. @xcite in-troduce formal definitions of what it means for a piece of text to be attributable to a given source, referred to as ais (attributable to identified sources). this definition has been used to design datasets and models to automatically predict whether a particular source document supports a claim. verification models have considered a variety of formulations such as question-answering based @xcite , lexical alignment based @xcite and nli-based @xcite . among these, nlibased models are a commonly used automated ais metrics to measure whether a particular output is grounded in provided sources @xcite . by incorporating web or corpus search, the ais models can be used to check whether a particular claim is true with respect to general world knowledge @xcite . factscore @xcite prompts an llm to break down claims into atomic facts to be verified. we use a simplified version of the factscore metric by compute sentence-level entailment scores. these search-enabled verification models have enabled the study of how language models can generate attributed text along with citations and study accuracy of such deployed approaches @xcite . retrieval for generating attributable text. it is well known that language models store a large amount of knowledge in their parameters @xcite . retrieving evidence documents allows llms to synthesize text about facts outside their parametric knowledge @xcite , including using up-to-date @xcite and proprietary @xcite information. past work has incorporated retrieval for post-hoc verification and editing @xcite . other approaches incorporate search results throughout the generation process either based on heuristics @xcite @xcite or based on trigger tokens generated by the model @xcite . planning of long-form generation. past work has demonstrated that planning (outlining the output) before generation can help to improve the factuality and quality of the model output. qa blueprint-based methods @xcite use question-answer pairs to change the llm distribution over output text to make summarization models more grounded. huot et al. @xcite extend this approach to cross-lingual initial generation prompt write a clear, fluent paragraph about lorrie moore web search: lorrie moore produce qa plan ...for each paragraph, write a descriptive set of instructions... paragraph 1: -where was lorrie moore born? -where did lorrie moore receive her advanced degrees? ... paragraph 1: describe lorrie moore's early life, birthplace, education. paragraph 2: describe lorrie moore's writing, style, and major themes paragraph 3: list the awards that lorrie moore has received. paragraph 2: -what books has lorrie moore written? -what are the themes of lorrie moore's novels? paragraph 3: -which of lorrie moore's writing has won awards? -what awards did lorrie moore receive? independently generate questions per paragraph web search + qa model for each question: e.g., where was lorrie moore born? produce generation from qa plan ...{ques. n} {ans. n} write a fluent, clear paragraph... the american author lorrie moore was born in glenn falls, ny in 1957. she received an mfa from cornell university following her undergraduate studies at st. lawrence college. moore has published numerous novels and short story collections including: who will run the frog hospital and birds of america. moore's …. summarization. akash et al. (2023) use a trained model to generate a keyword plan before generating a each new sentence and use the plan to inform search. our contribution: we study whether zeroshot llm generated plans can be used to retrieve relevant information beyond simple search and how the plans along with the new, diverse search results can guide the final generation to be more grounded. concurrently with this work, shao et al. (2024) also demonstrate the importance of pre-writing (outlining, iterative search and outline refinement) for long-form expository writing, further supporting the findings of our work. ## empirical study our goal is to understand the effect of providing more complete contextual information to language models on hallucination in text generation. we compare different strategies for performing retrieval and placing facts into context. we consider the task of generating descriptive text about an entity. this task effectively constitutes writing a bio about an individual (such as author lorrie moore in the previous example) or a summary about an event. this text will then be verified against web sources for attribution using automated methods @xcite . ## empirical analysis we will measure quantitatively the ability of models to provide attributed generations. we will analyze how producing attributed text is affected by the way in which evidence is retrieved and used. in particular, we aim to investigate the following: q1: does providing more comprehensive collections of facts yield more attributable generations? ## conclusion this paper explores the use of planning to improve retrieval for grounded long-form generation. we investigate how planning can guide retrieval allowing llms to generate more attributable responses and provide detailed analysis of the effect of the granularity of the plans and the content and quantity of retrieved information. we show empirically how the planning-based approaches outperform, in terms of automated groundedness metrics, standard retrieval-augmented generation approaches. our empirical investigations span topics in the topend and long-tail of the parametric knowledge of llms.