CaLM: Contrasting Large and Small Language Models to Verify Grounded Generation (2024)

Abstract: Grounded generation aims to equip language models (LMs) with the ability to produce more credible and accountable responses by accurately citing verifiable sources. However, existing methods, by either feeding LMs with raw or preprocessed materials, remain prone to errors. To address this, we introduce CaLM, a novel verification framework. CaLM leverages the insight that a robust grounded response should be consistent with information derived solely from its cited sources. Our framework empowers smaller LMs, which rely less on parametric memory and excel at processing relevant information given a query, to validate the output of larger LMs. Larger LM responses that closely align with the smaller LMs’ output, which relies exclusively on cited documents, are verified. Responses showing discrepancies are iteratively refined through a feedback loop. Experiments on three opendomain question-answering datasets demonstrate significant performance gains of 1.5% to 7% absolute average without any required model fine-tuning.

heading: 1 Introduction
text: Large Language Models (LLMs) are increasingly popular tools for information seeking. A recent trend emphasizes integrating citations from verifiable sources to boost LLM credibility and enable user verification. This aims to reduce hallucinations and increase accountability (Gao et al., 2023b; Huang and Chang, 2023; Liu et al., 2023b). To achieve this, LLMs must not only identify relevant documents within vast retrieved collections but also accurately ground their responses in these
∗ Work done while the author was a student researcher at Google Cloud AI Research. Correspondence to: I-Hung Hsu <ihunghsu@usc.edu>, Chen-Yu Lee <chenyulee@google.com>
sources and effectively generate their responses. This significantly increases the complexity of LLM operations (Gao et al., 2023b).
A standard approach to achieve this grounded generation is by retrieval-augmented generation with instructions to guide LLMs to generate responses along with their corresponding sources in one single LLM inference run (see Fig. 1 (a)). More recently, more sophisticated approaches utilize LLMs to first summarize relevant documents (Gao et al., 2023b) or use key information extraction and algorithms that explore different relevant document combinations by asking LLMs to enrich
12782
the original input query with additional information (Li et al., 2023) (see Fig. 1 (b)).
However, both single-run (Fig. 1 (a)) and preprocessing (Fig. 1 (b)) strategies face challenges for accurate generation and citation. Single-run approaches require LLMs to process the input query and a potential large volume of retrieved documents in one forward pass, which can strain their capabilities. Preprocessing approaches, while more focused, risk error propagation or loss of information. Additionally, both strategies limit the LLM’s ability to iterate, refine, and verify responses, impacting citation accuracy and answer correctness.
In contrast to single-run and preprocessing strategies, we propose a novel post-verification approach that enables LLMs to fact-check and ground their responses. Our design leverages the complementary strengths of larger and smaller LMs. We observe that larger LMs excel at identifying relevant information within a vast corpus but can rely excessively on internal parametric memory during generation. Smaller LMs, however, are adept at processing retrieved relevant information but less capable of identifying it from large collections (see § 3.3 for details).
Building on these observations, we propose CaLM (Contrasting Large and sMall language models to verify grounded generation). CaLM validates the large LLM’s response by crossreferencing it with output from a smaller LM. The smaller LM scrutinizes the cited documents to confirm the large LLM’s citation accuracy. If the responses align, the large LLM’s answer is verified. If not, CaLM extracts useful statements and evidence from the large LLM’s response and seeks additional supporting information to improve the query response. Importantly, CaLM requires no model fine-tuning, allowing smaller LMs to significantly enhance the grounded generation capabilities of large LMs. Fig. 3 illustrates this process.
We conduct experiments on three open-domain question answering datasets (QAMPARI, ASQA, and ELI5), which require consulting multiple sources for comprehensive answers. Our method demonstrates significant improvements in both answer accuracy and citation quality, outperforming state-of-the-art methods by an average of 1.5% to 7%. Crucially, our method remains robust even in challenging scenarios with less powerful retrieval systems, while other baselines often struggle.

heading: 2 Problem Statement
text: Task Setup. We cope with the problem of grounded generation (Gao et al., 2023b). Given a query q and a corpus of trustworthy text passages D, the model needs to generate an answer response A, which consists of n statements s1, s2, ...sn, based on the knowledge in D. Each statement si cites a list of passages Ci = {c1i , c2i , ...}, ∀cji ∈ D. The collective sets Ci for i = 1, 2, . . . , n constitute the grounded evidence G, from which A is derived. Our goal is to jointly optimize the usefulness of A to q, the preciseness of Ci for statement si, and the integrity of G to adequately support A. Evaluation of Response. The task involves measuring three dimensions of system responses, following the setup from Gao et al. (2023b). • Fluency: Determining whether the model’s gen-
erated text A is fluent and coherent. • Correctness: Assessing if A is accurate and cov-
ers all relevant aspects of query q. • Citation Quality: Evaluating whether cited pas-
sages directly support the answer and avoids irrelevant citations. This is achieved by evaluating both citation recall and citation precision. Gao et al. (2023b) propose measuring citation quality by averaging scores for each statement si. Citation recall ensures there is at least one supporting citation cji for si. Citation precision measures whether all the citations are “relevant”. Specifically, a citation cji is considered “irrelevant” to statement si if c j i cannot support si, and removing cji from Ci would not impact the overall support for si from the remaining citations.

heading: 3 Automated Verification for Grounded Generation
text: Although LLMs have demonstrated proficiency in a wide range of tasks, they remain susceptible to generating hallucinations (Huang et al., 2023; Zhang et al., 2023). These hallucinations could occur in both answers and citations within grounded generations due to the high complexity of the entire working pipeline, which includes noise from the retrievers and the limited ability of LLMs to handle long contexts (Liu et al., 2023a). This issue underscores the critical need for verification mechanisms to ensure the quality of the generated output and to leverage the interplay between verification and generation systems to improve the final output A.
In this section, we first analyzes key factors to
verify grounded generation (§ 3.1). Subsequently, we introduce an automated and unsupervised verification method for grounded generation using a small LM as a verifier and contrasting results from large and small LMs to verify large LMs’ response (§ 3.2 & 3.3).

heading: 3.1 Key Factors for Automated Verification
text: Automated verification, unlike the task evaluation in § 2, operates without a ground-truth reference and should be efficient for real-time system feedback. Here, our focus lies on assessing answer correctness and citation quality. To evaluate the correctness of a generated grounded response, we must ensure that generated responses (A) faithfully leverage information from the knowledge base D, avoiding hallucinations or model biases. Additionally, correct reasoning in deriving the answer is also crucial. For the automatic evaluation of citation quality, a trained Natural Language Inference (NLI) model can assess each citation and statement pair iteratively to measure the citation’s fidelity (Gao et al., 2023b). However, this process can be computationally expensive for lengthy generated answers with numerous citations. Efficient automated verification for grounded generation must consider these factors.

heading: 3.2 Contrasting Large and Small LMs for
text: Automated Verification
We propose a verification method using a smaller LM to assess the quality of a larger LM’s grounded generation. The small LM receives only the large LM’s cited documents (G) as input to answer the same query q. Consistency between their responses indicates the quality of G and the grounded generation from the large LM.
Our design exploit the inherent characteristics of smaller LMs. We posit that a robust G should enable even small LMs to deduce the correct answer. Notably, small LMs, having fewer parameters, are demonstrably more receptive to integrating external knowledge (Xie et al., 2023). Reaching consistent results from both LMs indicate high answer fidelity. 1 Leveraging different LMs as support also reduce the reasoning error risks as the different LMs exhibit diverse strengths and reasoning mechanisms (Jiang et al., 2023). These characteristics of
1In this paper, we differentiate LMs by size, labeling them as large or small. However, a more accurate categorization would be strong versus weak LMs, reflecting their varying performance levels across different LM families.
small LMs makes our design effective for verifying the answer correctness.
Furthermore, as will detailed in § 3.3, smaller LMs are more sensitive to the relevance of input evidence. Irrelevant documents in the evidence set G can easily mislead small LMs, while missing crucial citations hinder their ability to reach the correct answer independently, due to their limited parametric knowledge. This sensitivity allows us to utilize small LMs for assessing the quality of G.

heading: 3.3 Analyzing Model Size Impact on LMs’ Sensitivity to Input Document Relevance
text: Our automated verification method exploits the high sensitivity of small LMs. This section empirically demonstrates that, within the same model family, smaller LMs exhibit greater sensitivity to the relevance of input documents to a given query q compared to their larger counterparts.
We investigate the sensitivity of LMs to the relevance of input documents by modeling the performance of an LM as a function of the input document’s relevance. To mitigate the effects of the divergent abilities for different LMs to follow instructions, we examine the relative performance improvement of the model response as the relevance of the input document increases. The larger this value is, the more sensitive the LM is.
Yet, the complexity of this function is beyond simple linearity. As depicted in Fig. 2(a), this function typically exhibits a monotonic increase and hence we intend to additionally study second-order improvements gain analysis to further study the curvature of the function. This analysis use three anchoring points xlow, xmed, xhigh, and the corresponding performance P (xlow), P (xmed), P (xhigh) to study the much incremental performance gain the LM can get when the input document’s relevance keep increasing. Specifically, we use the ratio P (xhigh)−P (xmed)P (xmed)−P (xlow) to represent the incremental performance gain. A higher ratio indicates the LM is more sensitive to input document’s relevance, as illustrated by the orange line in Fig. 2(a).
We conduct empirical studies using a retrievalaugmented generation setting on the ASQA dataset (Stelmakh et al., 2022). Each instance in the dataset contains multiple answers and requires reading multiple documents to make correct prediction. In this experiment, a LM utilizes five input documents d to answer queries q, with model per-
formance evaluated based on answer accuracy, and the relevance of the input documents d is assessed by their average recall rate in containing answers. We prepare three different set of d with approximately 27%, 56%, and 78% recall rates, respectively, to represents xlow, xmed, xhigh in Fig. 2(a).
Fig. 2(b) presents the results of relative performance improvement, while Fig. 2(c) shows the second-order analysis. Our study encompasses LM families such as Yi-LM, Vicuna (Zheng et al., 2023), WizardLM (Xu et al., 2023), and Tulu-2dpo (Ivison et al., 2023). The findings indicate that within the same LM family, compared to the largest model, smaller LMs usually achieve higher relative improvement and receive a greater incremental performance gain, suggesting their higher sensitivity to the relevance of input documents to the query q. On the other hand, smaller LMs are more prone to erroneous results and suboptimal performance with irrelevant documents, which motivates our automated verification method design.

heading: 4 CaLM Framework
text: Building on the automated verification design from § 3.2, we introduce CaLM, an inference framework that leverages the synergy between verification and generation systems to enhance the system’s final
grounded generation. Fig. 3 depicts the iterative five-step algorithm
of our CaLM framework. Steps three and four correspond to the automated verification method detailed in § 3.2. The remaining steps involve the large LM making predictions.
We now present a detailed breakdown of each step below. We differentiate between the “main LM,” whose responses are verified, and the “verifier LM,” an auxiliary LM assisting the verification process. As described in § 3.2, we recommend using a larger LM as the main LM and a smaller LM for verification to achieve optimal performance.
Step (1) Context Retrieval. CaLM starts by selecting a ranked pool of trustworthy passages p for a given query q using a retriever.
Step (2) Main LM Generation. We select the top-k documents from the reference pool p and feed them into the main LM. This value of k is a hyperparameter constrained by the maximum input capacity of the main LM. The LM then analyzes these k passages to generate an answer candidate A for the query q. Our findings in § 3.3 suggest employing a large-scale LM at this stage. This is because larger LMs exhibit greater robustness in accurately identifying useful information and filtering out noise within the retrieved passages.
Step (3) Verifier LM Generation. Building upon the automated verification design outlined in § 3.2, this step leverages a grounded evidence set G extracted from the main LM’s answer candidate A. A smaller, dedicated verification LM then re-attempt the grounded generation process and obtain verifier output A′ for query q by solely providing the model with the evidence set, G, rather than the full top-k documents.
Step (4) Contrasting Answer Candidate and Verifier Output: A strong evidence G should enable small LMs to deduce correct answers reliably. Our goal in this step is to verify consistency (Wang et al., 2023) of A′ against the answer candidate A.
If the comparison shows enough consistency, we accept the answer A and stop the iterative process. Otherwise, we dismiss the inconsistent segments from the answer and citations, and continue with the next iteration. More technically, we extract Ā = A∩A′ and the corresponding Ḡ within Ā for Step (5) usage.
The realization of consistency measurement is done via calculating ROUGE-2 score between the
A′ and A. If the ROUGE-2 score exceeds a threshold θ, the answer candidate is considered acceptable. This threshold can be tuned using a small development set. Empirically, by our observation, setting θ = 0.2 to 0.5 yields satisfactory results by small set of qualitative examination.
Step (5) Input Preparation for Next Iteration: In this final step, we prepare input for the next iteration, including input reference document lists and the draft for correction. The new input reference document lists is initialized by Ḡ and is supplemented more passages from the next batch of passages from pool p until complete the budget k. This process boosts the likelihood of finding relevant documents while maintaining useful documents that have been verified. Then, we create a new prompt for large LM focusing on leveraging the new input reference document lists to correct the incomplete answer response Ā.
The full prompt we use for each step are detailed in Appx. §B. In practice, we should set a maximum iteration T to halt the whole process to prevent the verification condition cannot be satisfied. If this
maximum iteration is reached, we will output the last answer candidate we get from the main LM.

heading: 5 Experimental Setup
text: We consider three factoid question-answering (QA) datasets. For each dataset, we present an illustrative example in Tab. 1 for better understanding.
We prepare the trustworthy text passages D for each dataset accordingly following (Gao et al., 2023b). Each entry in D is a 100-word passage following previous works on open-domain QA (Karpukhin et al., 2020; Petroni et al., 2021; Liu et al., 2023a).

heading: 5.1 The ASQA dataset
text: Basic Introduction. ASQA (Stelmakh et al., 2022) is a long-form generation QA dataset derived from the AmbigQA dataset (Min et al., 2020). It comprises questions characterized by their ambiguity, necessitating multiple short answers to address various aspects. Each entry in the dataset is accompanied by a comprehensive long-form answer that covers all the corresponding short answers. 948 samples are tested for our experiment.
Experimental Setting Since most questions can be answered by Wikipedia, prior works usually use 2018-12-20 Wikipedia snapshot as D.
For retriever utilization, we examine the application of both DPR (Karpukhin et al., 2020) and GTRlarge (Ni et al., 2022). DPR introduces marginally more complex challenges for the LLM, achieving a recall rate of 51.5%, whereas GTR achieves 56.8% when considering the top-5 retrieved documents.
Evaluation • Fluency: We use MAUVE score (Pillutla et al.,
2021) to evaluate the corpus-wise similarity of
the machine generated text and the long answers generated by systems. • Correctness: We follow (Stelmakh et al., 2022) to calculate the exact matching recall (EM recall) of the presents of correct short answers. • Citation quality: As mentioned in § 2, we calculate citation recall and citation precision using the scripts provided by (Gao et al., 2023b).

heading: 5.2 The QAMPARI dataset
text: Basic Introduction. QAMPARI (Amouyal et al., 2022) is created from Wikipedia, pairing questions with multiple answers derived from its knowledge graph and tables. These answers, comprised of entities, describe simple relationships to the entities in the query q. As a result, this dataset focuses on testing systems’ abilities on entity identification within questions and accurately pinpointing the relevant entities. We use the same 1000 testing samples used in (Gao et al., 2023b) for experiments.
Experimental Setting Similar to the ASQA case, we employ the Wikipedia snapshot from 2018-12- 20, as our D. For retrieval, we again use both DPR and GTR-large, achieving recall rates of approximately 17.6% and 31.6%, respectively, for the top five retrieved documents for each query.
Evaluation Metrics In QAMPARI, we only considering the correctness and the citation quality since the output of the dataset is a list of entities. • Correctness: We follow (Stelmakh et al., 2022)
to to calculate the entity precision and recall of the model prediction using exact string match. When calculate recall, the evaluation considers recall to be 100% if the prediction includes at least 5 correct answer, denoted as recall-5. • Citation quality: We use the same way as the
ASQA dataset to evaluate the citation quality.

heading: 5.3 The ELI5 dataset
text: Basic Introduction. The ELI5 dataset, introduced by (Fan et al.), primarily features “How,” “Why,” and ‘What” questions. It tests a system’s ability to summarize complex information into clear and insightful answers. We use the 1000 samples used in (Gao et al., 2023b) for our experiment.
Experimental Setting Unlike ASQA and QAMPARI, the ELI5 dataset covers diverse topics and hence, documents in Sphere corpus are treated as D (Piktus et al., 2021). Given the large size of the Sphere corpus, BM25 is used for efficient retrieval.
Evaluation Metrics • Correctness: ELI5 dataset does not provide
short entity answers. We follow (Gao et al., 2023b) to calculate claim recall for correctness. For each reference answer in the dataset, three “sub-claims” are first extracted, and we will test whether the machine’s answer A can entail these sub-claims using a TRUE NLI model (Honovich et al., 2022) • Fluency & Citation quality: We use the metrics as the ASQA dataset for evaluation.

heading: 5.4 Compared Methods
text: We compare the following methods, all of which we have independently rerun, except Self-RAG. For our own rerun results, the reported results are the average of three random runs. 1. In-Context Learning (ICLCite): LLMs are in-
voked once for grounded generation through instruction-based in-context learning. Following (Gao et al., 2023b), we provide five documents to the LLM. They suggest that increasing the number of input document lists does not significantly improve performance when using GPT-3.5-turbo.

heading: 2. Summary then In-Context Learning (Summ
text: + ICLCite) (Gao et al., 2023b): This method follows the preprocessing paradigm shown in Fig. 1. Initially, LLMs generate summaries for each document based on the query q. Then, these summaries are fed into the LLM to execute ICLCite. In our experiments, we generate summaries for the top-9 documents retrieved for each instance.

heading: 3. Snippet then In-Context Learning (Snippet +
text: ICLCite) (Gao et al., 2023b): Similar to Summ
+ ICLCite, but LLM are guided to generate extractive summaries during preprocessing steps.

heading: 4. In-Context Learning with Self-Consistency
text: (ICLCite + USC): This post-processing method that employs ICLCite to initially generate various output samples. Then, it applies universal self-consistency Chen et al. (2023) to obtain the results. For a fair comparison with other baselines, like Summ+ICLCite, we first generate 9 samples and then use a LLM to determine the most consistent result among them. 5. Self-RAG (Asai et al., 2023): This method finetunes LMs to generate special tokens to trigger additional fact checks and retrieval. Since this method requires model finetuning, we only report results on ASQA dataset only. 6. CaLM: We use verification and an iterative refinement design to ensure the output quality. We follow the setting of ICLCite to set the our main LM’s reading budget k = 5. We set the consistency threshold θ = 0.25 for the ELI5 dataset and θ = 0.5 for the ASQA dataset. This θ is decided by manual qualitative examination on a small set of development data. Besides, we set the maximum iteration to be 4 for budget concern, and use the 13B version of tulu-2-dpo (Ivison et al., 2023) as the verifier LM.

heading: 6 Experimental Results

heading: 6.1 Main Results
text: For the main experiment, we consider two different large LMs as the backbone: GPT-3.5-Turbo1106 (Ouyang et al., 2022) and the PaLM-based LLMs (Anil et al., 2023), text-unicorn 2.
Tab. 2, Tab. 3, and Tab. 4 present the results on ASQA, QAMPARI, and ELI5, respectively. We have three discovery across the three datasets: 1. Snippet+ICLCite performs as the strongest base-
line. We hypothesize that for tasks involving grounded generation, preserving original evidence within documents is crucial for citation quality. Abstractive summarization can result in the loss of significant evidence crucial for solving the task. Additionally, accurately extracting consistent answers from multiple samples enhances correctness, yet determining corresponding consistent citations poses a challenge.
2. Despite Snippet+ICLCite being the strongest baseline, it does not always outperform ICLCite. 2https://cloud.google.com/vertex-ai/docs/generativeai/model-reference/text
We observe that with a weaker retriever, Snippet+ICLCite often fails to enhance performance. We conjecture that this is attributed to increased

heading: 6.2 Analysis
text: We conduct analysis of CaLM on the QAMPARI dataset. All the studies are conducted using GTR as the retriever and text-unicorn as the main LM, except where noted in the table.
What if we use larger LM as the verifier LM? Tab. 5 shows how the size of the verifier impacts
Choice of Verifier LM. Correctness Citation Average Prec. Rec.-5 Prec. Rec.
GPT-3.5-Turbo-1106 as Main LM
No verification 19.23 17.32 21.75 20.77 19.77
Tulu-2-dpo-7b 25.36 17.95 24.29 23.64 22.81 (+3.04) Tulu-2-dpo-13b (our choice) 26.71 18.55 25.16 24.39 23.70 (+3.93) Tulu-2-dpo-70b 25.61 18.69 23.64 23.57 22.88 (+3.11)
GPT-3.5-Turbo-1106 24.39 18.36 22.94 22.30 22.00 (+2.23)
text-unicorn as Main LM
No verification 26.27 21.42 25.52 25.21 24.61
Tulu-2-dpo-7b 29.71 21.80 27.67 27.45 26.65 (+2.04) Tulu-2-dpo-13b (our choice) 30.28 22.79 28.53 28.34 27.49 (+2.88) Tulu-2-dpo-70b 28.94 22.61 27.06 26.88 26.37 (+1.76)
text-unicorn 29.73 23.44 28.01 27.54 27.18 (+2.57)
Verifier LM’s input document Correctness Citation AveragePrec. Rec.-5 Prec. Rec.
Only cited documents (Ours.) 30.28 22.79 28.53 28.34 27.49 Same as the main LM 28.19 23.26 25.22 25.65 25.58
Table 6: Ablating our design of using only cited documents for verification. We can observe a significant performance drop if we remove the design.
the task performance. We can see that all verifiers improve the performance, demonstrating the robustness of our framework. Additionally, the smaller LM variants outperforms the largest LM (tulu-270b) in citation quality, and the medium-sized LM achieves the best performance. A medium-sized LM combines external knowledge integration and information discernment, resulting in a better verifier performance. Comparing with using the large LM itself as verifier, our choice of a smaller LM as the verifier performs better.
What if the verifier LM could access more than just the cited documents? In our automated verification method, the verifier LM is limited to accessing only the cited documents. Removing this constraint simplifies our verification algorithm to re-sampling with a different LM. The results, shown in Tab. 6, demonstrate a notable decline in performance when such verification design is removed, especially in citation quality. This highlights the essential role and effectiveness of our design of automated verification.
Further Analysis Appendix A details additional studies on model performance and prediction

heading: 7 Related Work
text: Retrieval-based Methods He et al. (2022); Gao et al. (2023a) used post-editing to ensure text consistency by retrieving relevant documents. Gao et al. (2023b) explored methods like document summarization and LLM-enabled searches for citation improvement, yet lacked verification for answer validation. Li et al. (2023) utilized an LLM as a verifier for document relevance but didn’t employ the answer to verify the correct grounding.
Self-reflection Prompting LLMs to self-reflect on their answers has been shown to improve factuality (Ji et al., 2023). Asai et al. (2023) employed this concept, enhancing LM quality and factuality via retrieval and self-reflection by training special tokens. CaLM outperforms this method without the need of training.

heading: 8 Conclusion
text: In this paper, we introduce CaLM, a novel verification approach for grounded generation. We observe that while larger LMs excel at identifying relevant materials, they tend to rely excessively on internal parametric memory. Conversely, smaller LMs are adept at processing focused information. CaLM leverages these complementary strengths to offer a fresh perpective on robust and scalable solution for verification in grounded generation.
Limitation
We acknowledge the limitations of CaLM from the following aspects to inspire future research opportunities in the field of grounded generation.
Firstly, as a postprocessing technique, our method introduces additional latency in generating the final answer. As a remedy, we can set the maximum iteration T smaller. From Fig. 4 in the appendix, we have shown that only even a single iteration of our correction process significantly enhances performance, yet latency remains an unavoidable factor.
Moreover, unlike preprocessing approaches depicted in Fig. 1(b), which can reduce input token consumption for the final LLM, our method necessitates that the LLM initially processes all documents, leading to a higher cost for token usage.
Lastly, despite the considerable advancements CaLM has made across datasets, the instances that pass our verification process are still not flawless. Given that both the answer candidate and the verifier output are outcomes from LMs, there is an inevitable risk of both models producing hallucinations simultaneously.
We hope future works can leverage the idea and insights from CaLM to advance the development of more robust grounded generation with low latency and reduced token costs.
Broader Considerations
As a method that directly apply LLMs, CaLM inherits all potential risks associated with LLMs, including but not limited to unethical outputs, toxicity, and biases (Bender et al., 2021; Yuan et al., 2024; Gallegos et al., 2023). Our qualitative assessment of CaLM, conducted across several samples from three datasets, indicates that LLMs generally adhere to instructions and generate responses relevant to the content of provided documents. However, we strongly advise conducting a comprehensive evaluation of these potential issues before deploying CaLM in practical settings.

heading: A Further Analysis
text: In this section, we focus on analysis when iteration of CaLM proceed.
How does the model’s performance improve as iterations proceed? Fig. 4 illustrates the outcomes of terminating the process from iteration 0 through 6. As demonstrated in the figure, the precision for correctness consistently improve with each iteration of CaLM as our framework only allows high quality final output. On the other hand, our design of updating the input document list contributes to a consistent rise in correctness recall.
However, we observe that performance tends to plateau after the third iteration, with subsequent iterations yielding diminishing returns. Extending the maximum iterations to six produced only a marginal average improvement of 0.4 compared to the third iteration, while significantly increasing the computational cost in terms of API calls.
We believe CaLM’s iterative nature is a key strength, allowing for continuous improvement. However, our findings suggest that two to three iterations offer substantial gains with minimal computational overhead. This demonstrates the framework’s efficiency and practicality for real-world applications.
Case study on CaLM’s correction. We present two case studies to demonstrate that by looping through our verification design, more accurate evidence can be found and more accurate responses can be generated. Fig. 5 is based on the ASQA dataset, and Fig. 6 utilizes the QAMPARI dataset.
From the example of Fig. 5, we can see that the small LM, serving as a verifier, when given access to cleaner input document sets, is capable of identifying overlooked information by the main LM. This detection initiates iterative correction processes in subsequent rounds.
The example in Fig. 6 demonstrates that (1) CaLM finds more convincing evidence documents in later rounds, and (2) CaLM catches citation errors through verification.

heading: B Used prompts
text: In this section, we list the prompts we use for our experiment.
B.1 Prompts for ASQA
Two different prompt sets are used for the ASQA dataset. Fig. 7 shows the prompts we used for the
LM to conduct grounded generation, which mainly follow the prompt used in (Gao et al., 2023b) with two shot examples. We design our own prompt for the main model to perform correction. The prompt is detailed in Fig. 8, which use 1-shot example.
B.2 Prompts for QAMPARI Two different prompt sets are used for the QAMPRI dataset. Fig. 9 shows the prompts we used for the LM to conduct grounded generation, which mainly follow the prompt used in (Gao et al., 2023b) with two shot examples. We design our own prompt for the main model to perform correction. The prompt is detailed in Fig. 10, which use 1-shot example.
B.3 Prompts for ELI5 Two different prompt sets are used for the ELI5 dataset. Fig. 11 shows the prompts we used for the LM to conduct grounded generation, which mainly follow the prompt used in (Gao et al., 2023b) with two shot examples. We design our own prompt for the main model to perform correction. The prompt is detailed in Fig. 12, which use 1-shot example.
C Implementation Details
For all experiments with public available models, we use vLLM framework for inference (Kwon et al., 2023). We operate vLLM on our machine with 16 NVIDIA-A100-40GB GPU. For experiments with GPT-3.5-Turbo-1106, we use the official API 3. For experiment with text-unicorn, we use Google-Cloud vertex API 4.

heading: D Dataset and Evaluation Tool
text: We use the artifacts provided by Gao et al. (2023b). The dataset and corresponding evaluation code is under MIT licence 5. We do not change any of the provided data and maintain consistent with their intended use.
3https://platform.openai.com/ 4https://cloud.google.com/vertex-ai/docs/reference/rest 5https://github.com/princeton-nlp/ALCE/tree/main

heading: Future Work
text: Limitation
We acknowledge the limitations of CaLM from the following aspects to inspire future research opportunities in the field of grounded generation. Firstly, as a postprocessing technique, our method introduces additional latency in generating the final answer. As a remedy, we can set the maximum iteration T smaller. From Fig. 4 in the appendix, we have shown that only even a single iteration of our correction process significantly enhances performance, yet latency remains an unavoidable factor. Moreover, unlike preprocessing approaches depicted in Fig. 1(b), which can reduce input token consumption for the final LLM, our method necessitates that the LLM initially processes all documents, leading to a higher cost for token usage. Lastly, despite the considerable advancements CaLM has made across datasets, the instances that pass our verification process are still not flawless. Given that both the answer candidate and the verifier output are outcomes from LMs, there is an inevitable risk of both models producing hallucinations simultaneously. We hope future works can leverage the idea and insights from CaLM to advance the development of more robust grounded generation with low latency and reduced token costs. Broader Considerations
As a method that directly apply LLMs, CaLM inherits all potential risks associated with LLMs, including but not limited to unethical outputs, toxicity, and biases (Bender et al. , 2021; Yuan et al. , 2024; Gallegos et al. , 2023). Our qualitative assessment of CaLM, conducted across several samples from three datasets, indicates that LLMs generally adhere to instructions and generate responses relevant to the content of provided documents. However, we strongly advise conducting a comprehensive evaluation of these potential issues before deploying CaLM in practical settings