Ask, Assess, and Refine: Rectifying Factual Consistency and Hallucination in LLM s with Metric-Guided Feedback Learning (2024)

Abstract: Recent advancements in Large Language Models (LLMs) have heralded unprecedented capabilities in information-seeking and text generation, as evidenced by applications like Bing Chat and perplexity.ai. Despite these strides, challenges on hallucination and factual inconsistency continue to impede their wider real-world adoption. Contemporary methods, including retrieval-augmented LLMs and feedback-based learning, serve as alternatives to mitigate these challenges. However, challenges remain, particularly regarding referencing erroneous evidence (citation errors) and generating information not present in the evidence (hallucination). In this paper, we introduce the \mathsf{A}^2\mathsf{R} framework: A sk, A ssess, and R efine. Our approach utilizes an explicit evaluation paradigm, incorporating metrics specifically tailored to assess citation errors and hallucination, aiming to address these prevalent challenges robustly. Capitalizing on these evaluations, we devise a strategy to formulate actionable natural language feedback, enabling iterative refinements that yield improved factual consistency and reduced hallucinations in responses. Our experiments on ASQA, ELI5, and QAMPARI datasets demonstrate our method’s superiority in enhancing correctness, fluency, and citation quality.

## introduction recent pioneering works on large language models (llms) have facilitated for information seeking and text generation, thereby showcasing the various real-world applications such as bing chat 1 and perplexity.ai 2 . however, despite of significant advancements with a combination of supervised fine-tuning and reinforcement learning, llms still tend to generate hallucination and contains the factually incorrect information in their output. since the lack of factual consistency constrains their extensive use in real-word applications, a substantial research has been dedicated to addressing this deficiency. additionally, users struggle with challenges verifying the faithfulness of generated responses from llms. for example, as illustrated in figure 1 , a user queries the system with, "who was the top scorer when arsenal last won the premier league?". in responding, the system first seeks out relevant evidence. then, it processes both the question and the gathered search results. finally, it formulates a response, referencing appropriate citations. however, the response has hallucinations and inaccurate citations. because robin van persie was not in during arsenal's last league-winning 2003-2004 season, and patrick vieira was cited from doc [2] instead of doc @xcite . to address the challenges of hallucination and factual inconsistency in llm outputs, contempo-rary research focuses on two-folds: 1) retrieval-augmented llms @xcite @xcite @xcite : this approach conditions llm generation on retrieved evidence from an external knowledge source. a notable contribution in this domain is by @xcite , which introduces a novel paradigm where llm outputs are appended with citations referencing one or more passages. this framework also emphasizes automatic evaluation across diverse aspects, such as fluency, correctness, and citation quality, facilitating a rigorous assessment of the initial output and directing further enhancements. in practical terms, platforms like bing chat and perplexity.ai furnish user queries with responses that cite current web pages, typically sourced from closed search engines. despite the advances in llms, contemporary research has highlighted several inherent challenges. primarily, while promising, retrieval-augmented llms face issues such as referencing erroneous evidence (citation errors), and generating information not present in the evidence (hallucination), as in figure 1 . to mitigate these issues, the prevalent use of intrinsic self-correction methods involves llms to rectify their initial responses based solely on inherent capabilities of llms by utilizing feedback-based learning. however, as highlighted by @xcite , employing llms' inherent capabilities for self-correction reveals a significant challenge, because llms are not adept at self-assessment, struggling to accurately evaluate their output's quality and identify factual inconsistencies. this stems from the llms' inadequate self-evaluative capabilities, which are crucial for the identification and rectification of hallucination independently. consequently, relying solely intrinsic methods could lead to degradation of the llms' ability for self-correction. in a bid to rectify these shortcomings, our paper introduces the framework a 2 r: ask, assess, and refine -an approach armed with augmenting llms through metric-based iterative feedback learning. distinct from self-refine @xcite , which relies on intrinsic evaluation of the initial output y, our study adopt an explicit evaluation encompassing multiple aspects, namely correctness, citation quality, as conceptualized in alce @xcite . leveraging llms, we formulate natural language feedback f for each dimension, predicated upon the respective evaluation outcomes. using the initial output y along with the feedback f, the model m iteratively refines its output, stopping when it determines that additional refinement is no longer needed. experimental results on benchmark datasets, including asqa, eli5, and qampari, validate the efficacy of our approach, manifesting in substantial enhancements across correctness, fluency, and citation quality metrics. our findings emphasize the pivotal role of explicit evaluations in the feedback generation process, emphasizing their potential in engendering more dependable and articulate outputs from llms. the contributions of our paper can be briefly summarized as follows: (1) we introduce the a 2 r framework, specifically designed to address the pressing challenges of hallucination and factual inconsistency in llms. (2) we pivot from traditional intrinsic evaluations to explicit evaluations, enabling granular assessments on aspects like correctness and citation quality. (3) utilizing the results from explicit evaluations, we propose a method to formulate natural language feedback that is both understandable and actionable. this facilitates iterative refinements to produce responses improved in factual consistency and reduced hallucinations. (4) our method demonstrates substantial enhancements in key metrics like correctness, fluency, and citation quality on renowned benchmark datasets. figure 2 : an overall architecture of a 2 r framework: ask access , refine . starting with an input sentence and evidence, the framework generates an initial response, assesses its quality through automatic metrics, provides metric-informed feedback, and iteratively refines the output. this process encapsulates phases from initial response generation to the eventual iterative refinement, ensuring enhanced correctness, fluency, and citation quality in the final output. ## related work the retrieval-augmented llms has witnessed significant advancements. this methodology tailors llm outputs based on evidence extracted from external knowledge sources @xcite @xcite @xcite . a pioneering work by @xcite sets itself apart by not only generating outputs from llms but also appending them with citations linked to one or multiple passages. this approach supports a comprehensive evaluation, touching upon aspects like fluency, accuracy, and citation quality. platforms such as bing chat and perplexity.ai have operationalized this by offering answers to user inquiries, referencing contemporary web pages, primarily harvested from closed search engines. drawing inspiration from human tendencies to refine their written content based on feedback, the feedback-based refinement approach offers a continuous enhancement process for textual content. reinforcement learning, in this context, has become a popular tool to uplift the standard of generated content @xcite @xcite @xcite @xcite . approaches that either employ reference-based feedback or operate freely without stringent reference dependence have been proposed @xcite ; @xcite ; @xcite ; @xcite ; @xcite . in this paper, we pivot towards the methodologies pioneered by @xcite . by explicitly utilizing metrics, that evaluate correctness, citation recall, and precision, we generate guided feedback. this structured feedback becomes instrumental in refining and enhancing our initial output, bridging the gap between humanlike refinement processes and computational excellence. ## conclusion in this work, we have shed light on the challenges presented by hallucinations and factual inconsistencies in large language models, issues evident even in advanced retrieval-augmented llms. we introduced the a 2 r framework, emphasizing the integration of explicit evaluations to provide an in-depth assessment of aspects such as correctness and citation quality. by adopting this approach, our system utilizes actionable natural language feedback to iteratively refine outputs, leading to considerable advancements in response quality. results across key datasets further substantiate the effectiveness of our method. as llms become increasingly pivotal in real-world applications, our findings demonstrate the necessity for rigorous, feedback-driven mechanisms to enhance their reliability and precision. future endeavors should expand upon this foundation, further bridging the gap between machinegenerated content and factual accuracy.