Effective Large Language Model Adaptation for Improved Grounding and Citation Generation (2024)

Abstract: Large language models (LLMs) have achieved remarkable advancements in natural language understanding and generation. However, one major issue towards their widespread deployment in the real world is that they can generate “hallucinated” answers that are not factual.Towards this end, this paper focuses on improving LLMs by grounding their responses in retrieved passages and by providing citations. We propose a new framework, AGREE, Adaptation for GRounding EnhancEment, that improves the grounding from a holistic perspective. Our framework tunes LLMs to self-ground the claims in their responses and provide accurate citations to retrieved documents. This tuning on top of the pre-trained LLMs requires well-grounded responses (with citations) for paired queries, for which we introduce a method that can automatically construct such data from unlabeled queries. The self-grounding capability of tuned LLMs further grants them a test-time adaptation (TTA) capability that can actively retrieve passages to support the claims that have not been grounded, which iteratively improves the responses of LLMs. Across five datasets and two LLMs, our results show that the proposed tuning-based framework generates superior grounded responses with more accurate citations compared to prompting-based approaches and post-hoc citing-based approaches.

## introduction recent advancements in large language models (llms) have yielded demonstrably groundbreaking capabilities in natural language processing (nlp) @xcite . their ability to understand, generate, and manipulate text at unprecedented scales and depths has established them as a transformative force * work done during an internship at google cloud ai. there has been a growing amount of interest in making llm-generated responses more trustworthy by grounding and adding citations. one line of work uses instruction tuning @xcite or in-context learning @xcite to instruct llms to generate grounded responses with citations to retrieved passages, following the retrieval-augmented generation @xcite @xcite framework. as llms are required to perform this challenging task from just instructions and few-shot demonstrations, such directions often lead to mediocre grounding quality @xcite . another line of work is on post-hoc citing @xcite , which links support passages to the claims in responses using a natural language inference (nli) model. this paradigm heavily relies on llms' parametric knowledge and might not extend well to less-known knowledge @xcite . we propose a new learning-based framework, agree, adaptation of llms for grounding enhancement. as shown in fig. 1 , our framework fine-tunes llms to generate citations, as opposed to prompting or relying on an external nli model used in a post-hoc way. at the training learning-based: agree(ours) prompting-based post-hoc citing training adaptation (section 4.1) finetuning corpus unlabeled queries ?? base llm adapted llm nli model training data automated data construction base llm query base llm prompting the llm to include citations in the response add citations post-hoc using an nli model corpus nli model response without citations response [citation] response [citation] phase, agree collects well-grounded responses for unlabelled queries automatically from a base llm with the help of an nli model. next, the collected data are used for supervising llms to generate grounded responses based on the retrieved passages as well as include citations in their responses. as a test-time approach, we propose an iterative inference strategy that allows llms to seek for additional information based on the selfgrounding evaluation so as to refine its response. the tuning and test-time adaptation together enable llms to effectively and efficiently ground their responses in the corpus. we apply agree framework to adapt an api-based llm, text-bison, and an open llm, llama-2-13b, with training data collected using unlabelled queries from three datasets. we conduct evaluation on both in-domain and outof-distribution datasets, comparing the proposed agree framework against competitive in-context learning and post-hoc citing baselines. the experimental results highlight that agree framework successfully improves grounding, in citation recall & precision, compared to the baselines by a substantial margin (generally more than 20%). we find llms can learn to add accurate citations to their responses with our carefully designed tuning mechanisms. furthermore, the improvements in grounding quality achieved by tuning using certain datasets can generalize well across domains. to summarize, our main contributions include: ## related work hallucination is a prevalent issue for generative language models on many tasks @xcite @xcite @xcite @xcite . it has been evaluated in different ways, investigating the grounding in generated responses @xcite @xcite . various approaches have been proposed to mitigate hallucination and improve the factuality of llm-generated responses. among these, our work particularly focuses on providing citations to attributable information source @xcite . unlike existing work that largely relies on zero-shot prompting or few-shot prompting @xcite or use an additional nli model @xcite to add citations, we propose a learning-based approach that tunes llms to generate better-grounded responses supported with citations. more broadly, recent work also investigates methods for improving factuality of llms without using external knowledge, including inference-time intervention @xcite , cross-exam @xcite , self-verify @xcite , or reinforcement learning @xcite . our work differs from them in providing citations to external knowledge in the responses. additionally, there is past work that also uses external knowledge (e.g., knowledge base) to reduce hallucination by injecting knowledge into prompts @xcite . while the external knowledge used for generating a response can possibly serve as a coarse and general reference, these approaches also do not offer granular, sentence-level citations as in our work. lastly, the proposed framework is a form of a retrieval augmented generation approach. while past work has explored using retrieval to improve llm generation quality @xcite @xcite @xcite or factuality @xcite @xcite , our approach further enables llms to generate citations and self-generated citations to guide retrieval. ## problem & background our proposed framework aims to adapt a pretrained llm m b to m a that is able to provide grounded responses with citations. given a text query q and a corpus @xmath0, the adapted llm m a is required to generate a response a to the query that is factually grounded in the corpus d as well as providing citations c together with its response. following past work @xcite , we segment llms' output into statements by sentences and require each of the sentences to cite a set of passages from the corpus. specifically, let s 1 , . . . , s n be the statements in the answer @xmath1, . . . , s n . the citations @xmath2, . . . , e n } links each statement s i to a set of evidence passages recall that our adaptation aims to provide better grounded responses. with citations c, we can quantify the grounding quality of a response a by a grounding score g: ## agree framework the proposed agree framework takes a holistic perspective for grounding, proposing a model tuning approach that adapts the base llm to include citations in its responses, and introducing a testtime adaptation (tta) mechanism that leverages the citation information for actively retrieving from the corpus and iteratively refining the responses. ## conclusion we introduce a novel framework, agree, that adapts llm for improved grounding. the proposed framework tunes a pre-trained llm to selfground its response in retrieved passages using automatically collected data. the integrated capability for grounding their responses further enables the llm to improve the responses at test time. our evaluations across five datasets demonstrate the benefits of the proposed learning-based approach compared to approaches that solely rely on prompting or the parametric knowledge of llms. agree uses created grounded responses to llms via supervised fine-tuning, as we demonstrate it leads to strong empirical results. it is also possible to treat grounding as a preference and rlhf @xcite to tune llms, which we leave to future work. agree tuning incurs additional cost that is a one-time requirement for adapting the llm. considering the substantial grounding improvements, we believe this would be acceptable for most applications, especially for those with high-reliability requirements. future work can possibly explore training a separate universal improved grounding model beyond task-specific adaptation. we have mainly considered open domain question answering datasets focusing on information seeking tasks in english. generalization to other long form generation tasks and other languages can be important future work directions. lastly, adding citations to llm-generated responses in agree might carry a shared risk with related research -a seemingly plausible but incorrect citation could potentially make an unsupported statement more convincing to users.