Transparentize the Internal and External Knowledge Utilization in LLM s with Trustworthy Citation (2025)

Abstract: While hallucinations of large language models could be alleviated through retrieval-augmented generation and citation generation, how the model utilizes internal knowledge is still opaque, and the trustworthiness of its generated answers remains questionable. In this work, we introduce Context-Prior Augmented Citation Generation task, requiring models to generate citations considering both external and internal knowledge while providing trustworthy references, with 5 evaluation metrics focusing on 3 aspects: answer helpfulness, citation faithfulness, and trustworthiness. We introduce RAEL, the paradigm for our task, and also design INTRALIGN, an integrated method containing customary data generation and an alignment algorithm. Our experimental results show that our method achieves a better cross-scenario performance with regard to other baselines. Our extended experiments further reveal that retrieval quality, question types, and model knowledge have considerable influence on the trustworthiness in citation generation.

## introduction large language models (llms; @xcite have demonstrated remarkable questionanswering (qa) capabilities, providing users with helpful response @xcite . however, due to the hallucination of llms, it is crucial to improve the trustworthiness of the responses @xcite . retrieval augmented generation (rag) with explicit citations can utilize the retrieved external knowledge and link the response to the knowledge to improve the transparency of llm's response and increase user trust @xcite . however, previous work (appendix a) on leveraging context and prior knowledge in llm generation pays minor attention query: what album did olivia rodrigo release, which included the songs "vampire" and "obsessed" ? vampire is from guts, the second studio album by olivia rodrigo, an american singer and songwriter. the album was released in 2022 and it received 8.0/10... @user_8263: "vampire" was written around more serious themes -abusive relationships and drawback s to fame -that rodrigo hadn't delved into back in 2021... interpretability of prior knowledge utilization. prior knowledge, or parameter knowledge, referring to the knowledge encoded in the model's parameters, could serve as a supplement when the retriever fails to meet the needs of the question in rag @xcite . for example, as indicated in figure 1 , the question "what album did olivia rodrigo release, which included the songs 'vampire' and 'obsessed'?" contains two constraints, but the external documents only provide clues about one of them. if the model has supplementary prior knowledge, it can articulate the knowledge, thereby making the answer accurate and providing verifiable evidence. despite the significance of llm's prior knowledge, previous studies @xcite @xcite @xcite have largely overlooked its interpretable utilization in citation generation tasks. therefore, appropriately articulating and citing reliable parameter knowledge in an interpretable way remains challenging. ## dataset we construct the dataset from the three latest rag datasets: crag @xcite , frames @xcite , and situatedfaith-fulnesseval @xcite ) (sfe), as they provide diverse, challenging questions. we combine the three datasets, equip each data point with 5 retrieved documents, and annotate whether the document is ground truth. (i.e., contains the answer). according to whether the data point contains a ground truth document or not, we split the dataset into 2 settings gt and gt, respectively. detailed dataset profile and annotation step are shown in appendix d. ## method based on the proposed task requirements, we designed a generation paradigm and an integrated method for aligning open-source models. we propose a rational attribution and elaboration (rael) paradigm to align models with the requirements of our proposed task. specifically, we asked to review the context and scrutinize parameter knowledge to help selectively use context and faithfully state parameter knowledge. then, the model extracts context and recites parameter knowledge to provide trustworthy references. we design intralign (interpretability-trustworthiness alignment, a pipeline using reject sampling to generate customary data from @xcite and use the data tailored for the specific target model to enhance its performance, as in figure 4 . ## experiments we conduct comprehensive experiments on three llms with baselines and our method. then, we present the results along with a detailed and thorough analysis. ## human evaluations we design a webpage for human evaluations, as shown in figure 8 , and conduct human evaluations to justify our automatic metrics. three participants with fluent english proficiency took part in the evaluation, each of whom underwent preliminary testing and was assigned 100 examples. ## conclusion in this paper, we present a context-prior augmented citation generation task, requiring llms to generate citations and fine-grained references from both external contexts and internal parameter knowledge. our comprehensive evaluation of answer helpfulness, citation faithfulness, and reference trustworthiness reveals the challenge for llms to generate trustworthy citations. we also propose rael paradigm for this task and a method, intralign, to unleash the model's capacity to cite parameter knowledge with trustworthiness, facilitating a more transparent citation generation. in-depth studies reveal the significance of the quality of knowledge sources and highlight the llm's selective utilization of external and internal knowledge in citation generation.