Model Internals-based Answer Attribution for Trustworthy Retrieval-Augmented Generation (2024)

Abstract: Ensuring the verifiability of model answers is a fundamental challenge for retrieval-augmented generation (RAG) in the question answering (QA) domain. Recently, self-citation prompting was proposed to make large language models (LLMs) generate citations to supporting documents along with their answers. However, self-citing LLMs often struggle to match the required format, refer to non-existent sources, and fail to faithfully reflect LLMs’ context usage throughout the generation. In this work, we present MIRAGE – Model Internals-based RAG Explanations – a plug-and-play approach using model internals for faithful answer attribution in RAG applications. MIRAGE detects context-sensitive answer tokens and pairs them with retrieved documents contributing to their prediction via saliency methods. We evaluate our proposed approach on a multilingual extractive QA dataset, finding high agreement with human answer attribution. On open-ended QA, MIRAGE achieves citation quality and efficiency comparable to self-citation while also allowing for a finer-grained control of attribution parameters. Our qualitative evaluation highlights the faithfulness of MIRAGE’s attributions and underscores the promising application of model internals for RAG answer attribution. Code and data released at https://github.com/Betswish/MIRAGE.

## introduction retrieval-augmented generation (rag) with large language models (llms) has become the de-facto standard methodology for question answering (qa) in both academic @xcite and industrial settings @xcite . this approach was shown to be effective at mitigating hallucinations and producing factually accurate answers @xcite @xcite figure 1 : mirage is a model internals-based answer attribution framework for rag settings. context-sensitive answer spans (in color) are detected and matched with contextual cues in retrieved sources to evaluate the trustworthiness of models' answers. @xcite . however, verifying whether the model answer is faithfully supported by the retrieved sources is often non-trivial due to the large context size and the variety of potentially correct answers @xcite . in light of this issue, several answer attribution foot_0 approaches were recently proposed to ensure the trustworthiness of rag outputs @xcite @xcite . initial efforts in this area employed models trained on natural language inference (nli) to automate the identification of supporting documents @xcite . however, being based on an external validator, this approach does not faithfully explain the answer generation process but simply identifies plausible sources supporting model answers in a post-hoc fashion. following recent progress in the instruction-following abilities of llms, self-citation (i.e. prompting llms to generate inline citations alongside their answers) has been proposed to mitigate the training and inference costs of external validator modules @xcite . however, self-citation performance is hindered by the imperfect instruction-following capacity of modern llms @xcite , and resulting attributions are still predicted in an unintelligible, post-hoc fashion. this is an important limitation for these approaches, since the primary goal of answer attribution should be to ensure that the llm is not 'right for the wrong reasons' @xcite . in light of these considerations, in this work we introduce mirage, an extension of the contextreliance evaluation pecore framework @xcite that employs model internals for efficient and faithful answer attributions. mirage first identifies context-sensitive tokens in a generated sentence by measuring the shift in lm predictive distribution caused by the added input context. then, it attributes this shift to specific influential tokens in the context using gradient-based saliency or other feature attribution techniques @xcite . we adapt this approach to the rag setup by matching context-dependent generated sentences to retrieved documents that contribute to their prediction and converting the resulting pairs to citations using the standard answer attribution (aa) format. we begin our assessment of mirage on the shortform xor-attriqa dataset @xcite , showing high agreement between mirage results and human annotations across several languages. we then test our method on the open-ended eli5 dataset @xcite , achieving aa quality comparable to or better than self-citation while ensuring a higher degree of control over attribution parameters. in summary, we make the following contributions: in rag settings, a set of documents relevant to a user query is retrieved from an external dataset and infilled into an llm prompt to improve the generation process @xcite . answer attribution @xcite @xcite aims to identify which retrieved documents support the generated answer (answer faithfulness, @xcite , e.g., by exploiting the similarity between model outputs and references. 3 simplifying access to relevant sources via answer attribution is a fundamental step towards ensuring rag trustworthiness in customer-facing scenarios @xcite . @xcite and @xcite propose to approximate human aa annotations with nli systems such as true @xcite , using a source document as premise and an llmgenerated sentence as entailment hypothesis. aas produced by these systems were shown to correlate strongly with human annotations, prompting their adoption in aa studies @xcite . despite their effectiveness, entailment-based methods can be computationally expensive when several answer sentence-document pairs are present. moreover, this setup assumes the nli model ability to robustly detect entailment relations across all domains and languages for which the llm generator is used. in practice, however, nli systems were shown to be brittle in challenging scenarios, exploiting shallow heuristics @xcite @xcite , and require dedicated efforts for less-resourced settings @xcite . for example, nli may fail to correctly attribute answers in multi-hop qa settings when considering individual documents as premises @xcite . ## agreement with human answer attribution annotations we begin our evaluation by comparing mirage predictions to human-produced answer attributions. importantly, our aim is not to compare several aa approaches to claim optimal faithfulness, but rather evaluate how our proposed framework fares against existing approaches at the task of producing answer attributions from model internals. we employ the xor-attriqa dataset @xcite , which, to our knowledge, is the only open dataset with human annotations over rag outputs produced by a publicly accessible lm. 6 we limit our assessment to open-weights llms to ensure that mirage answer attribution can faithfully reflect the model's inner processing towards the natural production of the annotated answer used for evaluation. 7 moreover, while cross-linguality is not the focus of our work, xor-attriqa allows us to assess the robustness of mirage across several languages and its agreement with human annotations compared to an entailment-based system. ## conclusion in this study, we introduced mirage, a novel approach to enhance the faithfulness of answer attribution in rag systems. by leveraging model internals, mirage effectively addresses the limitations of previous methods based on prompting or external nli validators. our experiments demonstrate that mirage produces outputs that strongly agree with human annotations while being more efficient and controllable than its counterparts. our qualitative analysis shows that mirage can produce faithful attributions that reflect actual context input: prompt + retrieved docs (@xmath0...] document [3] [...] what will happen if you accidentally set off your security system? the siren will sound and it will be loud, but you should be able to stop the siren by entering your code into your keypad. [...] document [4] [...] query: how does a small paper library bar code set off the security alarm when you haven't checked a book out? answer attribution results self-citation: [...] false alarms can be prevented by entering the code on the keypad, as documented in [3]. [...] mirage: [...] false alarms can be prevented by entering the code on the keypad [∅] [...] nli (true model): [3] entails the answer sentence. in conclusion, mirage represents a promising first step in exploiting interpretability insights to develop faithful answer attribution methods, paving the way for the usage of llm-powered questionanswering systems in mission-critical applications.