C a LM : Contrasting Large and Small Language Models to Verify Grounded Generation (2024)

Abstract: Grounded generation aims to equip language models (LMs) with the ability to produce more credible and accountable responses by accurately citing verifiable sources. However, existing methods, by either feeding LMs with raw or preprocessed materials, remain prone to errors. To address this, we introduce CaLM, a novel verification framework. CaLM leverages the insight that a robust grounded response should be consistent with information derived solely from its cited sources. Our framework empowers smaller LMs, which rely less on parametric memory and excel at processing relevant information given a query, to validate the output of larger LMs. Larger LM responses that closely align with the smaller LMs’ output, which relies exclusively on cited documents, are verified. Responses showing discrepancies are iteratively refined through a feedback loop. Experiments on three open-domain question-answering datasets demonstrate significant performance gains of 1.5% to 7% absolute average without any required model fine-tuning.

## introduction large language models (llms) are increasingly popular tools for information seeking. a recent trend emphasizes integrating citations from verifiable sources to boost llm credibility and enable user verification. this aims to reduce hallucinations and increase accountability @xcite @xcite . to achieve this, llms must not only identify relevant documents within vast retrieved collections but also accurately ground their responses in these achieving accurate responses and precise citations in a single attempt can be challenging due to the potential irrelevance of input documents. error and hallucination can propagate. sources and effectively generate their responses. this significantly increases the complexity of llm operations @xcite . a standard approach to achieve this grounded generation is by retrieval-augmented generation with instructions to guide llms to generate responses along with their corresponding sources in one single llm inference run (see fig. 1 (a) ). more recently, more sophisticated approaches utilize llms to first summarize relevant documents @xcite or use key information extraction and algorithms that explore different relevant document combinations by asking llms to enrich the original input query with additional information @xcite (see fig. 1 (b) ). however, both single-run (fig. 1 (a) ) and preprocessing (fig. 1 (b) ) strategies face challenges for accurate generation and citation. single-run approaches require llms to process the input query and a potential large volume of retrieved documents in one forward pass, which can strain their capabilities. preprocessing approaches, while more focused, risk error propagation or loss of information. additionally, both strategies limit the llm's ability to iterate, refine, and verify responses, impacting citation accuracy and answer correctness. in contrast to single-run and preprocessing strategies, we propose a novel post-verification approach that enables llms to fact-check and ground their responses. our design leverages the complementary strengths of larger and smaller lms. we observe that larger lms excel at identifying relevant information within a vast corpus but can rely excessively on internal parametric memory during generation. smaller lms, however, are adept at processing retrieved relevant information but less capable of identifying it from large collections (see § 3.3 for details). building on these observations, we propose calm (contrasting large and small language models to verify grounded generation). calm validates the large llm's response by crossreferencing it with output from a smaller lm. the smaller lm scrutinizes the cited documents to confirm the large llm's citation accuracy. if the responses align, the large llm's answer is verified. if not, calm extracts useful statements and evidence from the large llm's response and seeks additional supporting information to improve the query response. importantly, calm requires no model fine-tuning, allowing smaller lms to significantly enhance the grounded generation capabilities of large lms. fig. 3 illustrates this process. we conduct experiments on three open-domain question answering datasets (qampari, asqa, and eli5), which require consulting multiple sources for comprehensive answers. our method demonstrates significant improvements in both answer accuracy and citation quality, outperforming state-of-the-art methods by an average of 1.5% to 7%. crucially, our method remains robust even in challenging scenarios with less powerful retrieval systems, while other baselines often struggle. ## problem statement task setup. we cope with the problem of grounded generation @xcite . given a query q and a corpus of trustworthy text passages d, the model needs to generate an answer response a, which consists of n statements s 1 , s 2 , ...s n , based on the knowledge in d. each statement s i cites a list of passages c @xmath0, c 2 i , ...}, ∀c j @xmath1. the collective sets c i for @xmath2, 2, . . . , n constitute the grounded evidence g, from which a is derived. our goal is to jointly optimize the usefulness of a to q, the preciseness of c i for statement s i , and the integrity of g to adequately support a. ## automated verification for grounded generation although llms have demonstrated proficiency in a wide range of tasks, they remain susceptible to generating hallucinations @xcite . these hallucinations could occur in both answers and citations within grounded generations due to the high complexity of the entire working pipeline, which includes noise from the retrievers and the limited ability of llms to handle long contexts @xcite . this issue underscores the critical need for verification mechanisms to ensure the quality of the generated output and to leverage the interplay between verification and generation systems to improve the final output a. in this section, we first analyzes key factors to verify grounded generation ( § 3.1). subsequently, we introduce an automated and unsupervised verification method for grounded generation using a small lm as a verifier and contrasting results from large and small lms to verify large lms' response ( § 3.2 & 3.3). ## calm framework building on the automated verification design from § 3.2, we introduce calm, an inference framework that leverages the synergy between verification and generation systems to enhance the system's final grounded generation. fig. 3 depicts the iterative five-step algorithm of our calm framework. steps three and four correspond to the automated verification method detailed in § 3.2. the remaining steps involve the large lm making predictions. we now present a detailed breakdown of each step below. we differentiate between the "main lm," whose responses are verified, and the "verifier lm," an auxiliary lm assisting the verification process. as described in § 3.2, we recommend using a larger lm as the main lm and a smaller lm for verification to achieve optimal performance. ## experimental setup we consider three factoid question-answering (qa) datasets. for each dataset, we present an illustrative example in tab. 1 for better understanding. we prepare the trustworthy text passages d for each dataset accordingly following @xcite . each entry in d is a 100-word passage following previous works on open-domain qa @xcite @xcite . ## conclusion in this paper, we introduce calm, a novel verification approach for grounded generation. we observe that while larger lms excel at identifying relevant materials, they tend to rely excessively on internal parametric memory. conversely, smaller lms are adept at processing focused information. calm leverages these complementary strengths to offer a fresh perpective on robust and scalable solution for verification in grounded generation.