Why Uncertainty Estimation Methods Fall Short in RAG : An Axiomatic Analysis (2025)

Abstract: Large Language Models (LLMs) are valued for their strong performance across various tasks, but they also produce inaccurate or misleading outputs. Uncertainty Estimation (UE) quantifies the model’s confidence and helps users assess response reliability. However, existing UE methods have not been thoroughly examined in scenarios like Retrieval-Augmented Generation (RAG), where the input prompt includes non-parametric knowledge. This paper shows that current UE methods cannot reliably estimate the correctness of LLM responses in the RAG setting. We propose an axiomatic framework to identify deficiencies in existing UE methods. Our framework introduces five constraints that an effective UE method should meet after incorporating retrieved documents into the LLM’s prompt. Experimental results reveal that no existing UE method fully satisfies all the axioms, explaining their suboptimal performance in RAG. We further introduce a simple yet effective calibration function based on our framework, which not only satisfies more axioms than baseline methods but also improves the correlation between uncertainty estimates and correctness.

## introduction large language models (llms) have recently demonstrated promising capabilities in various tasks, including question-answering, and various classification and clustering tasks @xcite @xcite . however, llms are prone to generating incorrect information for multiple reasons, such as lack of parametric knowledge @xcite , temporal knowledge shifts @xcite , or noisy information introduced through retrieved documents in retrieval-augmented generation (rag) @xcite . as a result, the trustworthiness of llm-generated responses has ## axiomatic framework the assumption of an axiomatic framework for ue is that by satisfying a set of formal constraints, a ue method would likely have an optimal correlation with correctness for both rag and no-rag setups. to define the framework, we introduce five axioms based on a set of functions that form our search space for an optimal ue. these axioms, while necessary, do not represent an exhaustive list, as increasing the number of axioms can, in reality, introduce stringent, contradictory, or biased constraints. in the following, we introduce the functions and constraints of our axiomatic framework. ## derivation of a calibration function in this section, we derive a calibration function that improves existing ue methods using our axiomatic framework. to recap, our formal constraints are built around four functions that are examined for llm responses without rag (r 1 ) and with rag (r 2 ). in the reference-free instantiation of our framework (cf. sec. 3.3), we showed that these functions are of two types: (i) equivalence that examines the relation between two llm-generated responses, represented as e(r 1 , r 2 ), and (ii) other functions that examine entailment, contradiction, and independence relations between context, query, and an llm generated response, represented as r(c, q, r). we define a calibration coefficient by searching the space of our axiomatic constraints using these two types of functions: the hyper parameters k 1 -k 4 are set to satisfy the axioms using a validation set. this calibration enables increasing the uncertainty score of rag for samples associated with axioms 2 and 4 while decreasing it for samples related to axioms 1 and 3. ## results to assess this bias further, we report on average uncertainty values of these methods in tables 1 and 5 . the results reveal that incorporating any context results in lower uncertainty values. even the inclusion of irrelevant contexts, which do not enhance accuracy, leads to a significant reduction in uncertainty scores. this suggests that current ue methods produce lower uncertainty values in the rag setup without adequately accounting for the relevance of the context. ## discussion and conclusions in this paper, we examined existing uncertainty estimation (ue) for the rag setup and showed they systematically generated low uncertainty values in the rag setup without considering the relevance of the given context to the query. we further proposed an axiomatic evaluation framework for ue in the rag setup and defined five formal constraints that a ue method should satisfy when processing both parametric and non-parametric knowledge. these axioms were empirically validated across multiple representative datasets, ue methods, and llms. our results showed that none of the existing ue methods pass all the axiom, pinpointing the problem in these methods. we further derived a calibration function for adjusting ue methods in the rag setup and improvements in both axiomatic evaluation and correlation with correctness. future work includes developing a ue method designed to naturally conform to the estab-