Citation-Enhanced Generation for LLM -based Chatbots (2024)

Abstract: Large language models (LLMs) exhibit powerful general intelligence across diverse scenarios, including their integration into chatbots. However, a vital challenge of LLM-based chatbots is that they may produce hallucinated content in responses, which significantly limits their applicability. Various efforts have been made to alleviate hallucination, such as retrieval augmented generation and reinforcement learning with human feedback, but most of them require additional training and data annotation. In this paper, we propose a novel post-hoc Citation-Enhanced Generation (CEG) approach combined with retrieval argumentation. Unlike previous studies that focus on preventing hallucinations during generation, our method addresses this issue in a post-hoc way. It incorporates a retrieval module to search for supporting documents relevant to the generated content, and employs a natural language inference-based citation generation module. Once the statements in the generated content lack of reference, our model can regenerate responses until all statements are supported by citations. Note that our method is a training-free plug-and-play plugin that is capable of various LLMs. Experiments on various hallucination-related datasets show our framework outperforms state-of-the-art methods in both hallucination detection and response regeneration on three benchmarks. Our code and datasets can be found at https://github.com/Tsinghua-dhy/CEG.

## introduction large language models (llms) have experienced rapid development in recent years, which show powerful general intelligence in various scenarios @xcite . current llm-based chatbots, epitomized by chatgpt and † weizhi ma (mawz@tsinghua.edu.cn) and yang liu (li-uyang2011@tsinghua.edu.cn) are corresponding authors. gpt-4, demonstrate impressive capabilities across distinct domains in communicating with humans. there is a growing consensus that llm-based chatbots can be the next generation of information acquisition methodology. however, a critical and unsolved challenge of llm-based chatbots is the hallucination problem @xcite , which indicates these chatbots may generate hallucinated content in responses randomly. as the underlying mechanisms of hallucinations remain unclear, this problem has substantial constraints on the deployment of llm-based chatbots in various sensitive scenarios, such as healthcare and education, where reliability is paramount. previous approaches have attempted to mitigate this issue through retrieval augmentation @xcite and value alignment (rlhf) @xcite in response generation, but these often require additional training and extensive data annotation. for example, instructgpt @xcite utilize rlhf to alleviate hallucinations in model output, but needs extra training. @xcite attempt to reduce hallucination through adding retrieved related documents and citations before generation, while the pre-hoc way of incorporating citations may potentially harm the model performance, resulting in poor response results with hallucinations. in this work, we propose a novel method to alleviate hallucination in llms, which leverages retrieval augmentation and natural language inference (nli) technologies to implement citation-enhanced generation (ceg) in a post-hoc way. figure 1 is an illustration. differing from previous studies, the retrieval augmentation module of the ceg framework works after generation (post-hoc), and ceg prompts the model to regenerate the answer when necessary. this approach is effective and easy to use, which can reduce the hallucination in the model's output for various llms. we conduct experiments on distinct hallucination-related benchmarks, including detection and response regeneration, where our method achieved state-ofthe-art performance. further analyses demonstrate the usefulness of each module on ceg. in summary, the main contributions of our work can be summarized as follows: ## conclusion in this study, we propose a novel post-hoc citationenhanced generation framework to reduce hallucinations in llms, which involves retrieval augmentation and natural language inference technologies. different from previous hallucination studies, our framework is post-hoc and flexible, which can be applied to distinct llms without additional training or annotations, making it hold significant practical implications. experiments on three hallucination-related benchmarks and our datasets demonstrate that our ceg framework achieves state-of-the-art performance in hallucination detection and regeneration. further analyses show the effectiveness of our proposed modules and adopted models. in the future, we plan to further expand the corpus to support more scenarios.