Citation-Enhanced Generation for LLM-based Chatbots (2024)

Abstract: Large language models (LLMs) exhibit powerful general intelligence across diverse scenarios, including their integration into chatbots. However, a vital challenge of LLMbased chatbots is that they may produce hallucinated content in responses, which significantly limits their applicability. Various efforts have been made to alleviate hallucination, such as retrieval augmented generation and reinforcement learning with human feedback, but most of them require additional training and data annotation. In this paper, we propose a novel post-hoc Citation-Enhanced Generation (CEG) approach combined with retrieval argumentation. Unlike previous studies that focus on preventing hallucinations during generation, our method addresses this issue in a post-hoc way. It incorporates a retrieval module to search for supporting documents relevant to the generated content, and employs a natural language inference-based citation generation module. Once the statements in the generated content lack of reference, our model can regenerate responses until all statements are supported by citations. Note that our method is a training-free plug-and-play plugin that is capable of various LLMs. Experiments on various hallucination-related datasets show our framework outperforms state-of-theart methods in both hallucination detection and response regeneration on three benchmarks. Our code and datasets can be found at https: //github.com/Tsinghua-dhy/CEG.

text: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1451–1466 August 11-16, 2024 ©2024 Association for Computational Linguistics
Large language models (LLMs) exhibit powerful general intelligence across diverse scenarios, including their integration into chatbots. However, a vital challenge of LLMbased chatbots is that they may produce hallucinated content in responses, which significantly limits their applicability. Various efforts have been made to alleviate hallucination, such as retrieval augmented generation and reinforcement learning with human feedback, but most of them require additional training and data annotation. In this paper, we propose a novel post-hoc Citation-Enhanced Generation (CEG) approach combined with retrieval argumentation. Unlike previous studies that focus on preventing hallucinations during generation, our method addresses this issue in a post-hoc way. It incorporates a retrieval module to search for supporting documents relevant to the generated content, and employs a natural language inference-based citation generation module. Once the statements in the generated content lack of reference, our model can regenerate responses until all statements are supported by citations. Note that our method is a training-free plug-and-play plugin that is capable of various LLMs. Experiments on various hallucination-related datasets show our framework outperforms state-of-theart methods in both hallucination detection and response regeneration on three benchmarks. Our code and datasets can be found at https: //github.com/Tsinghua-dhy/CEG.

heading: 1 Introduction
text: Large Language Models (LLMs) have experienced rapid development in recent years, which show powerful general intelligence in various scenarios (Yue et al., 2023; Singhal et al., 2023). Current LLM-based chatbots, epitomized by ChatGPT and
† Weizhi Ma (mawz@tsinghua.edu.cn) and Yang Liu (liuyang2011@tsinghua.edu.cn) are corresponding authors.
GPT-4, demonstrate impressive capabilities across distinct domains in communicating with humans. There is a growing consensus that LLM-based chatbots can be the next generation of information acquisition methodology.
However, a critical and unsolved challenge of LLM-based chatbots is the hallucination problem (Ji et al., 2023), which indicates these chatbots may generate hallucinated content in responses randomly. As the underlying mechanisms of hallucinations remain unclear, this problem has substantial constraints on the deployment of LLM-based chatbots in various sensitive scenarios, such as healthcare and education, where reliability is paramount.
Previous approaches have attempted to mitigate this issue through retrieval augmentation (Borgeaud et al., 2022; Izacard et al., 2022) and
1451
value alignment (RLHF) (Ouyang et al., 2022; Touvron et al., 2023) in response generation, but these often require additional training and extensive data annotation. For example, InstructGPT (Ouyang et al., 2022) utilize RLHF to alleviate hallucinations in model output, but needs extra training. Gao et al. (2023a) attempt to reduce hallucination through adding retrieved related documents and citations before generation, while the pre-hoc way of incorporating citations may potentially harm the model performance, resulting in poor response results with hallucinations.
In this work, we propose a novel method to alleviate hallucination in LLMs, which leverages retrieval augmentation and Natural Language Inference (NLI) technologies to implement CitationEnhanced Generation (CEG) in a post-hoc way. Figure 1 is an illustration. Differing from previous studies, the retrieval augmentation module of the CEG framework works after generation (post-hoc), and CEG prompts the model to regenerate the answer when necessary. This approach is effective and easy to use, which can reduce the hallucination in the model’s output for various LLMs. We conduct experiments on distinct hallucination-related benchmarks, including detection and response regeneration, where our method achieved state-ofthe-art performance. Further analyses demonstrate the usefulness of each module on CEG.
In summary, the main contributions of our work can be summarized as follows:
• We are the first to propose the use of citation to alleviate hallucination in a post-hoc way with regeneration.
• We design a novel post-hoc citation-enhanced generation framework combined with retrieval augmentation and NLI to avoid hallucinations, which is flexible for existing LLMs.
• Experimental results show that our CEG framework achieves the best performance on three hallucination-related benchmarks.

heading: 2 Related Work

heading: 2.1 Hallucination Control in LLMs
text: Generative AI has achieved significant advancements, while still facing the hallucination problem. Existing strategies can be categorized into major two types: mitigation during training and mitigation during inference. For the first type, LLMs,
such as LLaMA 2 (Touvron et al., 2023), undergo extensive training cycles with high-fidelity data sources like Wikipedia to bolster factual consistency in pre-training. Zhou et al. (2023) alleviate hallucination during instruction fine-tuning, which adopts high quality manually annotated content to regulate hallucination. Some studies (Ouyang et al., 2022; Touvron et al., 2023) also introduce penalties for nonfactual responses to alleviate hallucination in RLHF. However, all these methods need extra training and annotations.
On the other hand, researchers try to deal with the hallucination challenge during inference. Inference-Time-Intervention (Li et al., 2023b) mitigates hallucination by shifting model activations along these factuality-related directions during inference. Retrieval-Augmented Generation (RAG) (Lewis et al., 2020) has become a prevalent technique in alleviating hallucination by retrieving reliable documents before generation (Yu, 2022). While these methods still generate hallucinations due to the lack of post-hoc verification and they are unable to provide citations for verification.

heading: 2.2 Citation Augmented LLMs
text: In the realm of LLMs, retrieval technology has become a crucial component (Zhang et al., 2023b; Gao et al., 2023b), as it provides related knowledge in generating more reliable results (also mitigates the occurrence of hallucinations). Previous studies point out that citation, generated by retrieval models, is key to building responsible and accountable LLMs (Huang and Chang, 2023).
Existing citation augmented strategies can be divided into two types: parametric and nonparametric. Parametric methods (Taylor et al., 2022) refer to information internalized from the training data, often leading to inaccurate annotated documents, as the annotation process itself can give rise to hallucinations. Non-parametric methods (Gao et al., 2023a; Menick et al., 2022; Izacard and Grave, 2021) involve querying relevant information and seamlessly integrating the retrieved content from outside corpus, which provides more reliable citations. Thus, most previous studies are non-parametric, but they are pre-hoc based. For example, Gao et al. (2023a) adopt retrieval processes to facilitate the annotation of documents within model-generated outputs. Nevertheless, their prehoc annotation strategy inadvertently escalates the complexity of a QA task by converting it into a dual
challenge of generating a response coupled with simultaneous annotation. Different from existing citation augmented studies, we propose a different strategy to utilize retrieval models to generate citations in a post-hoc way.

heading: 3 Approach

heading: 3.1 Overview
text: Firstly, we give an overview of our proposed CEG framework. Illustrated in Figure 2, CEG has several critical modules: 1) Retrieval augmentation module, designed to search for documents Dj relevant to the original response R. In cases where responses are excessively lengthy, they can be broken down into sub-claims R = {R1, R2, ..., Rn}. 2) Citation generation module, which assesses if the retrieved documents Dj substantiate the {Ri} in response or not. 3) Regeneration module, tasked with creating a new prompt that integrates the original user query and key retrieved information for the LLM M to get a more reliable response R′.
It is important to note that our method is a posthoc framework and is highly adaptable across different LLMs, as it does not require any additional training or fine-tuning. Consequently, we do not specify a particular LLM here.

heading: 3.2 Retrieval Augmentation Module
text: Retrieval augmentation has been shown to have powerful abilities in previous hallucination-related studies (Gao et al., 2023a; Zhao et al., 2023). Different from these studies that aim to retrieve documents as evidence before response generation (questions are queries), we propose to conduct retrieval augmentation in a post-hoc way to verify the correctness of the generated claim Ri (claims are queries). As there are various existing studies on how to retrieve the most related document, we use a simple but effective dense retrieval strategy to verify the performance of our CEG framework, and we believe stronger retrieval will bring further improvements.
Query: For the response R, it will be segmented into several claims if necessary, resulting in R = {R1, R2, ..., Rn}. Here, we adhere to previous work (Chen et al., 2023) and employ a heuristic algorithm for segmentation using the NLTK (Bird et al., 2009) sentence tokenizer. The NLTK sentence tokenizer is a well-performing and widely used (Chen et al., 2023; Bird et al., 2009; Liu et al., 2023) sentence tokenizer and generally segments
text correctly in most cases. We split R to obtain reasonable results that align with user reading habits to get the claims Ri. Then, Ri is adopted as the query one by one.
Corpus (Candidate Documents): The choice of Corpus decides the scope of applications, and there are multiple candidates. In this study, our focus lies predominantly in the domain of knowledge-based question answering, necessitating the employment of a curated corpus. To this end, we leverage a processed snapshot of Wikipedia from October 20, 20231, segmented into approximately 100-word candidate documents, each demarcated by a period or newline character. Note that you can replace it with any other corpus, and we use it as most hallucination benchmarks are based on Wikipedia.
Retriever: Dense vector based retrieval technologies have demonstrated powerful performances in recent years, which are also widely used in existing RAG models. Here, we adopt the SimCSE BERT2 (Gao et al., 2021; Wang et al., 2023d) as the query and document encoder for its promising efficiency in previous studies (Wang et al., 2023c,d). Then, candidate documents are ranked based on cosine similarity scores calculated by the following equation:
Sim(Ri, dj) = e(Ri) · e(dj)
∥e(Ri)∥ · ∥e(dj)∥ ,
Where e() is the SimCSE BERT encoder, dj is a candidate document in the corpus. As more documents need more calculation in further modules, the top-k retrieved documents with higher similarity are selected to form the reference document set Di. We add an extra threshold t to filter out the retrieved documents that have low cosine similarity. Apart from the top-1 document, if the Sim(Ri, dj) < t, dj will not be included in Di. These documents are subsequently concatenated to construct the final retrieved content Di for further calculation.

heading: 3.3 Citation Generation Module
text: After getting the reference document Di for each response segment Ri, the next step involves generating labels and citations to verify the correctness of Ri. We propose to adopt an NLI method to determine the relationship between each claimdocument pair (Ri, Di). In general, the relationship
1https://dumps.wikimedia.org/enwiki/ 2https://huggingface.co/princeton-nlp/sup-simcse-bert-
base-uncased/tree/main
can be categorized into three types: support, independence, and contradiction. But in hallucinationrelated scenarios, to adhere to previous studies, we only utilize two categories: (1) Factual, where Di serves as a reference for Ri, thereby substantiating the claim. (2) Nonfactual, which means Di presents a opposite claim to Ri.
Although numerous models (Honovich et al., 2022; Raffel et al., 2020) are capable of the NLI method, our CEG framework seeks to fully leverage the language comprehension capabilities of LLMs. Therefore, we prefer to utilize an LLM with predefined prompts to serve as the NLI method. An illustrative prompt is provided below:
Instruction: I will show you a question, a response segment of this question, and a reference document. Your task is to assess whether the given response segment contains factual errors or
not with the help of the reference document. ...
When the LLM output indicates “factual”, the document Di is identified as a valid reference for the claim Ri. Consequently, this citation can be added to the original response. If none of the retrieved top-k documents substantiate the claim Ri or if there are documents opposing the claim, we will label this claim as nonfactual (potential hallucination) to remind users to keep carefully reading. Based on the introduced two modules, we can detect whether there are hallucinations in responses.

heading: 3.4 Response Regeneration Module
text: In previous modules, our framework offers a posthoc method to conduct citation-enhanced verification for responses, where reliable responses are incorporated with citations. However, a new challenge is how to deal with potential hallucinations.
So we propose a response regeneration module. Assuming LLM M generates the original response R, our framework will provide a new prompt for regeneration. The prompt not only contains the original query, but is also incorporated with retrieved documents and the annotated nonfactual segments. Here we provide a brief illustration of the prompt (a full prompt is shown in appendix):
User: Question; Chatbot: Response; User: [There were some factual errors in your output: (Nonfactual Claims). Please generate a new response based on relevant docs: (Relevant Docs).]
Upon receipt of the regenerated response, we can initiate a new citation-enhanced generation process. If the response is adjudged to be free of factual errors, it becomes the final response and will be shown to users. However, if the new response still contains hallucinations, the regeneration cycle will be repeated. To conserve API resources and reduce the waiting time, a predefined parameter T can be set to constrain the max regeneration attempts.

heading: 4 Experimental Settings

heading: 4.1 Overview
text: To verify the effectiveness of our framework, we adopt four hallucination-related datasets in our experiments: WikiBio GPT-3 (Manakul et al., 2023), FELM (Chen et al., 2023), HaluEval (Li et al., 2023a), and WikiRetr. WikiBio GPT-3 and FELM are hallucination detection benchmarks. HaluEval is a hallucination generation benchmark. Besides, we construct a new dataset named WikiRetr, which is to evaluate the retrieval and citation annotation performance. Due to the tasks and baselines are distinct in various datasets, we will introduce each dataset and corresponding settings one by one.
We use GPT models as the LLM backbones, and the version involved in different datasets is distinct for fair comparison with existing baselines. Unless otherwise specified, “ChatGPT” refers to GPT-3.5Turbo-1106, and “GPT-4” refers to GPT-4-0613. We set the decoding temperature as 0 to maintain the reproducibility of the responses generated by LLMs. All prompts are listed in Appendix A, and more dataset information is shown in Appendix B.

heading: 4.2 WikiBio GPT-3 Dataset
text: WikiBio GPT-3 dataset is constructed to evaluate the hallucination of LLMs. Researchers ran-
domly select 238 biographical articles from WikiBio dataset (Lebret et al., 2016), and utilize the text-davinci-003 to generate new passages. The passages are split into 1,908 sentences, and then manually annotated into three categories: Major Inaccurate, Minor Inaccurate, and Accurate. Following previous studies, Major Inaccurate and Minor Inaccurate are categorized as Nonfactual (potentially with hallucinations, 1,392 segments), and Accurate is treated as Factual (516 segments).
Baselines: 1) HalluDetector (Wang et al., 2023b) utilizes external knowledge sources, a specific classification model and a Naive Bayes classifier to detect hallucination. 2) Focus (Zhang et al., 2023a) adopts a multi-stage decision-making process, where both pre-retrieval and task specific classifiers are adopted. 3) SelfCheckGPT 3, three variants of which are utilized: w/BERTScore, w/Prompt, and w/NLI (Manakul et al., 2023). SelfCheckGPT w/BERTScore is based on the inherent uncertainty of LLM, while SelfCheckGPT w/Prompt and w/NLI draw upon external knowledge sources. The Area Under the PrecisionRecall Curve (AUC-PR) and Balanced_Accuracy are adopted as evaluation metrics.

heading: 4.3 FELM Dataset
text: FELM dataset is designed to evaluate hallucination detection ability. Researchers assemble prompts from diverse scenarios, and use them to instruct GPT-3.5-Turbo-0301 to generate responses. These responses are manually annotated as nonfactual and factual, along with supporting documents. Our experiments are conducted on the WorldKnowledge subset of FELM as it is based on Wikipedia corpus.
Baselines: Following settings in FELM, we adopt four strategies with ChatGPT, GPT-4, and Vicuna-33B as the backbone LLM (Zheng et al., 2023): 1) Vanilla prompts. 2) Prompts augmented with Chain-of-Thought (CoT) reasoning (Kojima et al., 2022). 3) Prompts augmented with hyperlinks to reference documents and 4) Prompts augmented with human-annotated reference documents (Chen et al., 2023). Experiments are conducted at the individual response level. Following the previous work (Chen et al., 2023), we chose accuracy of nonfactual, factual, and balanced as final metrics to to facilitate comparison with previous works.
3The latest version in https://arxiv.org/pdf/2303.08896.pdf.

heading: 4.4 HaluEval Dataset
text: HaluEval dataset is a benchmark for assessing the ability of LLMs to discern hallucinations. Each instance comprises a question, a correct answer, and a hallucinated answer (multiple answers are automatically generated, and the most confusing one is selected by ChatGPT). The QA subset of HaluEval is adopted as it is constructed by Wikipedia corpus, and 2,000 samples of which are randomly sampled.
Baselines: We adopt several models building upon the updated version of ChatGPT as previous studies (Li et al., 2023a): 1) Vanilla prompts. 2) Prompts augmented with CoT reasoning. 3) Prompts with Pre-RAG, where a strong and finetuned retriever, All-mpnet-base-v24, is used. Accuracy is chosen as the evaluation metric.

heading: 4.5 WikiRetr Datasets
text: WikiRetr datasets are designed to conduct further analyses on our CEG framework, which is created based on the October 20, 2023 snapshot of Wikipedia. We randomly select 1,000 passages, and apply text-davinci-003 and GPT-4 to rewrite them as new claims, separately. So that each rewritten claim is accompanied by an original passage. The constructed datasets are named WikiRetrGPT3 and WikiRetr-GPT4, respectively. Discussion about the reliability of WikiRetr datasets is provided in Appendix D.
To analyze the retrieval module, we utilize various retrievers, including: 1) SimCSE BERT, which is employed in our CEG framework; 2) Sentence BERT (Reimers and Gurevych, 2019), a retriever trained with siamese networks; and 3) All-mpnet-base-v2. Recall@k is the metric to ver-
4https://huggingface.co/sentence-transformers/all-mpnetbase-v2.
ify if the original document is retrieved in top-k. Precision@k is the metric to verify if the claim is supported by a doc in top-k docs with NLI method.
For NLI methods in the citation generation module, we randomly select 100 instances from each dataset for evaluation. We conduct manual annotation to assess whether the original passages can support the rewritten claims by three annotators. Labeling results show that 90% and 94% generated claims are supported by original documents, which is the ground truth of NLI models. Then, we use 1) True-9B5 (Honovich et al., 2022) model and 2) GPT models as NLI methods in analyses. We choose the consistency between manual and model annotations as the evaluation metric.

heading: 5 Experimental Results and Analyses

heading: 5.1 Performance on Hallucination Detection
text: To verify the effectiveness of our method, we utilize our retrieval augmentation and citation generation modules for hallucination detection on WikiBio GPT-3 and FELM datasets.
Overall performances in WikiBio GPT-3 dataset are shown in Table 1, and we have the following observations: 1) Our CEG framework outperforms all baseline methods in Balanced_ACC and AUC-PR of Factual segments, achieving the second performance in AUC-PR of nonfactual segments. These results indicate the strength of our method. 2) Previous pre-retrieval augmented generation models, SelfCheckGPT w/NLI and w/prompt, also get good performances in AUC-PR. While suffering from the lengthy text, they cannot achieve comparable performance of our model in all metrics. 3) Our model performs slightly worse than w/NLI in the
5https://huggingface.co/google/t5_xxl_true_nli_mixture.
AUC-PR of nonfactual segments, the reason can be the NLI module of SeftCheckGPT undergoes additional training on detecting nonfactual segments (but achieve poor results in factual).
Experimental results in FELM dataset are summarized in Table 2. Firstly, our model achieves the best result in balanced accuracy with GPT-4, indicating its effectiveness. Most baseline models are biased in classifying a single type. Then, CEG with ChatGPT beats other ChatGPT baselines, showing the flexibility of our model. Thirdly, CEG outperforms all pre-retrieval baselines, which shows the strength of the proposed post-hoc segment-level retrieval module in hallucination detection. Finally, for Vicuna-33B, all methods exhibit some degree of decline compared to the Vallina method, indicating its limitations in general ability and using retrieved documents. However, our method shows the smallest decline, especially compared to the manually labeled Doc baseline, our method outperforms by 2.2 points, proving the effectiveness of our finer-grained document utilization.
To summarize, CEG outperforms various SOTA baselines in two benchmarks with distinct LLM backbones, indicating that post-hoc retrieval with NLI is powerful for hallucination detection.

heading: 5.2 Results on Hallucination Regeneration
text: On the HaluEval dataset, we adopt the full framework of CEG to further evaluate the regeneration module. If the doc is helpful in solving the problem and any of the response segments are classified into nonfactual, our method will generate a new prompt for regeneration as introduced in Section 3.4. Besides, a 2018 Wikipedia snapshot is adopted as the corpus (Gao et al., 2023a) in this experiment due to the inconsistency between this dataset and the previous corpus.
Experimental results are presented in Table 3. Firstly, our CEG framework with GPT-3.5-TurboInstruct achieves the best performance (69.45% in accuracy), which achieves 8.10% improvements compared to the pre-hoc retrieval strategy. CEG with ChatGPT also outperforms the pre-retrieval strategy, so these results demonstrate our post-hoc method is robust. Secondly, pre-hoc retrieval strategy even performs worse than the baseline with CoT (Li et al., 2023a), which indicates the retrieved documents are not always helpful. Thirdly, consistent with our experiments related to NLI models in Table 5, when using GPT-3.5-Turbo-Instruct as the NLI model, the results are superior to ChatGPT. We also conduct case studies to show our regeneration results, and some cases are shown in Appendix C.

heading: 5.3.1 Ablation Study

heading: 5.3 Further Analyses
text: We conduct ablation experiments on the FELM Worldknowledge dataset, where the retrieval augmentation (k = 4) and the document selection threshold are involved (threshold = 0.5). As shown in Table 4, the retrieval augmentation module plays an important role in providing better results on different backbone LLMs (ChatGPT and GPT-4). Furthermore, the threshold of retrieved documents is necessary, which can filter out irrelevant documents in citation generation. Thus, all designed modules contribute to improvements in the CEG framework.

heading: 5.3.2 Retrieval Models
text: The choice of retrieval model significantly impacts the performance of our retrieval augmentation module, as illustrated in Figure 3. Experimental results of three different retrieval models show that SimCSE BERT has better performances in post-hoc retrieval tasks (76.8% when using top 10 documents), where 64 million documents are used as candidates. Besides, although a larger value of k can improve recall, it also requires more resources for further computation. For a balance between efficiency and effectiveness, the value of k is set between 4 and 6 in our experiments.

heading: 5.3.3 NLI Models in Citation Generation
text: The performance of different NLI models in the citation generation module is illustrated in Table 5,
and there are two main observations we can make: 1) Despite being a state-of-the-art task-specific NLI approach, True-9B performs worse than best LLMs in this scenario. LLMs, such as GPT-3.5-TurboInstruct and GPT-4, are capable of playing the NLI model in our citation generation module, as they achieve high agreement rates with human annotators. 2) LLMs show better performance on data they generate, which is consistent with previous studies (Wang et al., 2023a; Zheng et al., 2023).
Table 6 shows experimental results of the citation generation module with distinct NLI models when the retriever is Simcse BERT, which indicate: 1) Even on the corpus with over 64 million candidates, our citation generation module exhibits outstanding performance, achieving 78.2 and 75.7 on the precision of the two datasets, respectively. 2) Compared to WikiRetr-GPT3, WikiRetr-GPT4 constitutes a more challenging and higher-quality dataset. The reason is that WikiRetr-GPT4 demonstrates lower recall, suggesting a lower semantic similarity between the original text and the generated claim. While its precision surpasses recall, indicating the generated claims are high quality.

heading: 5.3.4 Hyper-parameter Analysis
text: Due to the space limit, we only show the hyperparameter analysis of k on FELM datasets in Figures 4. We have the following observations: 1) Experimental results on both FELM and WikiBio
GPT-3 datasets demonstrate that more documents (larger value of k) do not always provide better performance. The reason can be more documents lead to longer input, attention becomes dispersed, and simultaneously, the relevance of the retrieved documents decreases. 2) Less or no retrieved documents also contribute to worse performance, which indicates the usefulness of the retrieval augmentation module in CEG. 3) The best performance achieved when k near 5 (Top-4 for FELM dataset and Top-6 for WikiBio GPT-3 dataset).

heading: 6 Conclusion
text: In this study, we propose a novel post-hoc citationenhanced generation framework to reduce hallucinations in LLMs, which involves retrieval augmentation and natural language inference technologies. Different from previous hallucination studies, our framework is post-hoc and flexible, which can be applied to distinct LLMs without additional training or annotations, making it hold significant practical implications. Experiments on three hallucination-related benchmarks and our datasets demonstrate that our CEG framework achieves state-of-the-art performance in hallucination detection and regeneration. Further analyses show the effectiveness of our proposed modules and adopted models. In the future, we plan to further expand the corpus to support more scenarios.
Limitations
Our study has several limitations: 1) Restricted retriever and corpus: In our experiments, we do not employ a fine-tuned specific retriever for posthoc methods, and our method utilizes only the Wikipedia corpus, limiting the applicability of our framework to general knowledge-based questionanswering scenarios but only demonstrate the effectiveness of our model. 2) Our experiments are conducted on existing benchmarks and manual annotations, where we do not propose new QA datasets for the verification of regeneration performance. 3) The adopted NLI method in the citation generation module inherently relies on the LLM’s world knowledge. More powerful NLI methods can contribute to better performance. 4) Prompting to regenerate and Using NLI technology to generate citations both incur API cost. API costs incurred for conducting our method and creating the WikiRetr datasets are shown in Table 11.

heading: Acknowledgement
text: This work is supported by the National Natural Science Foundation of China (No. 62276152, 61925601, 62372260). We appreciate all the reviewers for their insightful suggestions.

heading: A Prompts Used in Our Experiments
text: The prompt used in the evaluation of CEG and w/o threshold variants of CEG on the FELM dataset, as well as in the citation generation experiments on the HaluEval dataset, is presented in Table 12. The prompt for w/o retrieval augmentation variants of CEG in the FELM dataset evaluation is presented in Table 13.
The prompts for the baseline Vanilla, CoT, and Pre-Retrieval methods used in the HaluEval dataset are shown in Table 14, Table 15, and Table 16. The prompt to regenerate a new response on the HaluEval dataset is presented in Table 17.
Context: { Retrieved passages } Sentence: { Sentence to be verified } Is the sentence supported by the context above? Answer Yes or No:
The NLI prompt in experiments on WikiBio dataset is presented above.
premise: { Passage retrieved } hypothesis: { Hypothesis to be verified }
The NLI prompt employed by True-9B in experiments on WikiRetr datasets is presented above. We utilize the same prompt6 as in Gao et al., 2023a. The agreement rates with human annotaters of True9B in Table 5 and the precision of True-9B in Table 6 are based on this prompt.
Premise: { Passage retrieved } Hypothesis: { Hypothesis to be verified } Task: Determine the logical relationship between premise and hypothesis. Output format: If you believe the relationship is Entailment, output Entailment; for Contradiction, output Contradiction; for Neutral, output Neutral.
The NLI prompt employed by GPT series models in experiments on WikiRetr datasets is presented above. The agreement rates with human annotaters of GPT models in Table 5 and the precision of GPT models in Table 6 are based on this.
6You can find the original prompt at https://github.com/princeton-nlp/ALCE/blob/main/eval.py

heading: B Detailed Information About Adopted Datasets
text: More Statistics of WikiBio GPT-3 dataset are shown in Table 7, and statistics of WorldKnowledge subset in FELM are summarized in Table 8.
In Table 9, we provide an example of HaluEval dataset.
The utilized modules in different datasets are summarized in Table 10.

heading: C Case Studies on HaluEval Dataset
text: We conduct case studies on two cases in the evaluation of the HaluEval dataset, presented in Table 18 and Table 19, respectively.

heading: D Discussion about the reliability of WikiRetr datasets
text: After constructing WikiRetr datasets, we randomly select 100 samples from each of the two datasets, and conduct manual annotation to assess whether the original passages can support the rewritten claims by three annotators. More specifically, each sample is initially annotated by two human annotators. In cases where there are discrepancies between the annotations provided by the two annotators, the final label is determined by consensus among three annotators.
Labeling results show that 90% and 94% of generated claims are supported by original documents. This consistency is exceptionally high. For example, in the FELM dataset, the average interannotator agreement for labels is 90.7%, and in the Pinocchio dataset (Hu et al., 2023), the average label accuracy for sampled data is 92.7%, and the inter-annotator agreement is 85.6%.
The green part is the content of the regeneration.
The green part is the content of the regeneration.

heading: Future Work
text: In the future, we plan to further expand the corpus to support more scenarios. Limitations
Our study has several limitations: 1) Restricted retriever and corpus: In our experiments, we do not employ a fine-tuned specific retriever for posthoc methods, and our method utilizes only the Wikipedia corpus, limiting the applicability of our framework to general knowledge-based questionanswering scenarios but only demonstrate the effectiveness of our model. 2) Our experiments are conducted on existing benchmarks and manual annotations, where we do not propose new QA datasets for the verification of regeneration performance. 3) The adopted NLI method in the citation generation module inherently relies on the LLM’s world knowledge. More powerful NLI methods can contribute to better performance. 4) Prompting to regenerate and Using NLI technology to generate citations both incur API cost. API costs incurred for conducting our method and creating the WikiRetr datasets are shown in Table 11