Towards Verifiable Text Generation with Evolving Memory and Self-Reflection (2024)

Abstract: Despite the remarkable ability of large language models (LLMs) in language comprehension and generation, they often suffer from producing factually incorrect information, also known as hallucination. A promising solution to this issue is verifiable text generation, which prompts LLMs to generate content with citations for accuracy verification. However, verifiable text generation is non-trivial due to the focus-shifting phenomenon, the intricate reasoning needed to align the claim with correct citations, and the dilemma between the precision and breadth of retrieved documents. In this paper, we present VTG, an innovative framework for Verifiable Text Generation with evolving memory and self-reflection. VTG introduces evolving long short-term memory to retain both valuable documents and recent documents. A two-tier verifier equipped with an evidence finder is proposed to rethink and reflect on the relationship between the claim and citations. Furthermore, active retrieval and diverse query generation are utilized to enhance both the precision and breadth of the retrieved documents. We conduct extensive experiments on five datasets across three knowledge-intensive tasks and the results reveal that VTG significantly outperforms baselines.

## introduction large language models (llms) @xcite @xcite have showcased remarkable performance across a spectrum of downstream tasks recently. despite their advancements, llms often generate responses that include hallucinated facts and inaccurate information @xcite @xcite , undermining their reliability. to enhance the reliability of llms, a new generation paradigm, verifiable text generation @xcite @xcite @xcite , @xcite drinking coffee: …coffee consumption contribute positively to overall health, off ering antioxidants a nd beneficial… [2] type 2 diabetes: …finds a consistent lin k between moderate coffee consumption and lower risk of type 2 diabetes… however, verifiable text generation is challenging for the following reasons. firstly, it often involves long text generation, where the focus of the content changes over time, characterized by focusshifting phenomenon @xcite . this dynamic poses challenges in consistently aligning claims with the appropriate evidential references. for instance, as depicted in figure 1 , the discussion evolves from the health benefits of moderate coffee consumption to the adverse effects of excessive consumption. such shifts demand a dynamic adaptation in the document pool to support the shifting focus of the generation. secondly, identifying the intricate relationship between a claim and its potential evidence requires more than just linguistic matching-it demands careful analysis. for example, in figure 1 , the third claim suggests excessive coffee consumption leads to various mental and physical discomforts. the corresponding supporting document, although not explicitly mentioning these discomforts, describes symptoms inherently related to them. this necessitates an in-depth examination to confirm that the evidence truly supports the specific claim. thirdly, striking a balance between the precision and breadth of retrieved documents presents a complex challenge for verifiable text generation. on the one hand, the task is susceptible to noisy documents during the claim-citation alignment process, emphasizing the need to selectively retain a few highly relevant documents. on the other hand, the intrinsic nature of verifiable text generation calls for a comprehensive collection of documents to enhance credibility. therefore, crafting strategies to balance precision and breadth in document retrieval is crucial for advancing verifiable text generation. when composing text with citations, individuals are capable of adaptively gathering the most relevant information regarding the claim being written, typically involving active information seeking and frequent verification. inspired by this process, we propose vtg, short for verifiable text generation, a novel framework that operates through iterative generation and verification, utilizing an evolving memory and a two-tier verifier. specifically, to address the challenge of focus-shifting, vtg employs an evolving long short-term memory system. this system effectively archives important documents in long-term memory and maintains recent ones in short-term memory, thereby providing support for the evolving focus of the generation. moreover, to identify the complex relationship between a claim and its potential evidence, vtg employs a generation verifier and a memory verifier, both using natural language inference (nli) model to assess the logical support of potential evidence for the claim. the generation verifier first checks if the cited documents logically support the claim. if there's a misalignment, the memory verifier reevaluates the claim against the documents stored in memory. a positive outcome suggests that the misalignment is due to the citation generation process, not because the information in the claim is wrong, leading to the adoption of a refined set of documents from memory for citation. conversely, a negative outcome indicates potential factual inaccuracies in the claim, triggering an evidence finder to gather external information, which facilitates the regeneration of a more accurate and verifiable claim. lastly, to balance between precision and breadth in document retrieval, vtg incorporates active retrieval and diverse query generation. retrieval is initiated only when the claim does not pass the memory verifier, indicating potential factual inaccuracies. this approach guarantees the necessity of retrieval, reducing noise from unnecessary retrieval, and thereby enhancing retrieval precision. by instructing llms to generate diverse queries, the breadth of retrieved documents is broadened, enabling the documents to offer comprehensive support for the claim. to summarize, our main contributions are: ## methodology in this section, we first present the overall framework of vtg. then we will go over each part of the model in detail. ## conclusion in this paper, we introduce vtg, a novel framework tailored to address the challenges of verifiable text generation. central to its design is an evolving long short-term memory, which adaptively keeps both valuable documents and up-to-date documents. the two-tier verifier coupled with an evidence finder facilitates a deeper analysis and reflection on the relationship between claims and citations. through the integration of active retrieval mechanisms and diverse query generation, vtg skillfully enhances both the precision and breadth of the document retrieval process. extensive experiments on five datasets across three knowledge-intensive tasks verify the effectiveness of our method.