SIMBA UQ : Similarity-Based Aggregation for Uncertainty Quantification in Large Language Models (2025)

Abstract: When does a large language model (LLM) know what it does not know? Uncertainty quantification (UQ) provides measures of uncertainty, such as an estimate of the confidence in an LLM’s generated output, and is therefore increasingly recognized as a crucial component of trusted AI systems. Black-box UQ methods do not require access to internal model information from the generating LLM and therefore have numerous real-world advantages, such as robustness to system changes, adaptability to choice of LLM, reduced costs, and computational tractability. In this paper, we investigate the effectiveness of UQ techniques that are primarily but not necessarily entirely black- box, where the consistency between a generated output and other sampled generations is used as a proxy for confidence in its correctness. We propose a high-level non-verbalized similarity-based aggregation framework that subsumes a broad swath of UQ approaches suitable for complex generative tasks, as well as introduce specific novel techniques from the framework that train confidence estimation models using small training sets. Through an empirical study with datasets spanning the diverse tasks of question answering, summarization, and text-to-SQL, we demonstrate that our proposed similarity-based methods can yield better calibrated confidences than baselines.

## introduction uncertainty quantification (uq) approaches help inform reliability of model predictions and are therefore critical for deploying large language models (llms). uq refers to a broad suite of techniques that yield measures of uncertainty; here we are interested in assessing the confidence of an llm's generations for a user-specified task. while confidence is not always a well-defined quantity in the uq literature, it typically captures some measure of faith in an llm's response to a query as represented by a number between 0 and 1. in this work, we consider tasks where there is a notion of whether an llm's response to a user query is correct or not, and interpret confidence of a response as the probability that it is correct. this is in contrast to approaches that estimate the uncertainty of llm generations in response to a query, which measure the variability of the output. an important category of uq techniques are black-box methods, which only assume access to the model being used without requiring information such as the model parameters or even the token log probabilities. such techniques have numerous practical advantages as they are robust to the constantly evolving landscape of llms, computationally lightweight, and quickly deployable at inference time. as a result, black-box uq has become increasingly popular for tasks such as question answering @xcite @xcite . much of the research on black-box uq can be viewed as consistency-based, where the idea is to use the consistency between a generation and other sampled generations as a proxy for its confidence. the implicit underlying assumption behind such approaches is that when a generated response is more different from others, it is more likely to be incorrect, implying that responses that are consistently similar are more likely to be correct; this has been explored for various use cases involving self-consistency @xcite @xcite . recent work has referred to this assumption as the consistency hypothesis, formalizing it through statistical tests and evaluation metrics, and empirically validating its prevalence across a suite of tasks and datasets @xcite . in this paper, we provide a fresh perspective on consistency-based approaches through the lens of aggregating pairwise similarities. figure 1 outlines the procedure for a high-level framework that aims figure 1 : framework for uq yielding confidence estimates for generations from an llm in response to a natural language query. an illustrative natural language query from the spider dataset for the text-to-sql task is shown, along with example outputs at each step of the pipeline. jaccard is chosen as the similarity metric, and confidences are estimated using a proposed 'aggregation by classification' method with random forests (described later). at exploiting the afore-mentioned consistency assumption for uq. first, multiple outputs/samples are generated by the llm through some sampling procedure. pairwise similarities between samples can then be computed using any similarity metric of choice. finally, these similarities are leveraged to provide confidence estimates for each generation of interest. our methodological contributions are primarily in the third phase of figure 1 . in particular, rather than viewing the third phase as clustering outputs like in closely related work @xcite , we propose similaritybased aggregation as a framework for estimating confidences. in contrast to aggregating verbalized confidences @xcite , we aggregate pairwise similarities between generations, making our approach non-verbalized and therefore avoiding some empirically observed concerns around potential overconfidence when asking llms for probabilities @xcite . verbalized confidence aggregation is known to struggle for complex generative tasks. confidence estimates from uq can be evaluated in various ways depending on how they will be used by the system or end user. we are primarily interested in approaches yielding confidences that are well calibrated, as gauged by how closely they align with the empirical accuracy of the predictions @xcite . we also evaluate our proposed approaches on metrics that gauge how confidence estimates benefit when used to select from a set of generations or predict whether a generation is correct or not. our contributions are summarized as follows: • we introduce a high-level similarity-based aggregation (simba) framework, unifying and generalizing various consistency-based uq approaches by positioning them as different ways to represent (and possibly learn) confidence as a function of similarities with other generations. ## related work procedures for uq estimate measures like the variability or confidence of llm outputs, and can be categorized as either white-box or black-box. white-box methods assume access to the llm's internal components, such as model weights, logits, or embeddings. in contrast, black-box methods rely only on outputs, inferring confidence through alternative means. an orthogonal distinction is between verbalized and non-verbalized methods, where the former type prompts an llm to express uncertainty in natural language, such as using phrases ("i don't know" or "most probably") or quantitative indicators ("low" or "50%" or "90%"). white-box methods. early work on calibration in deep learning and transformers laid the groundwork for using token-level probabilities -derived from model output logits -as a signal for estimating the reliability of model predictions @xcite @xcite . more recently, @xcite propose semantic entropy based clustering on multiple samples generated from the model and then estimating confidence estimates by summing the token-level probabilities in each cluster. the use of token-level probabilities for uq has become a rapidly burgeoning area of research, driven by growing interest in the predictive confidence of llms @xcite @xcite . @xcite suggest a verbalized method where the llm generates responses and then evaluates them as either true/false; the probability that the model assigns to the generated token determines confidence. other approaches consider the llm's internal state such as embeddings and activation spaces. for instance, @xcite fit the embeddings for both inputs and outputs in the training data to a gaussian distribution, and estimate the model's confidence by computing the distance of the evaluated data pair from this gaussian distribution. some methods probe the model's attention layers to discriminate between correct and incorrect answers @xcite @xcite . although these methods provide insights into the model's linguistic understanding, they require supervised training on specially annotated data. ## conclusion assessing the confidence of llm generations in response to a query is a crucial endeavor for enabling trusted ai. real-world systems can benefit greatly from uq modules that are flexible in terms of applicability to diverse generative tasks and models, and also reasonably performant for varied downstream applications that use confidence estimates. we have proposed a general high-level similaritybased aggregation framework for uq, leveraging pairwise similarities between multiple generated samples, as well as specific novel approaches within that framework. one such approach views confidence estimation as a probabilistic classification task, where the objective is to predict the correctness of a generation using similarities with other generations for the same query as features. our methods do not rely on asking an llm for its confidence about a generation and can be categorized as consistency-based, since they rely on using consistency between generations as a signal for confidence. through an extensive empirical evaluation with 9 datasets spanning the tasks of qa, summarization, and text-to-sql, we show that using similarity features results in confidence estimates that fare well on various uq evaluation metrics, particular around minimizing calibration error. our data-driven methods are designed under the assumption of no data uncertainty and necessitate a small training set with samples generated in the same way for both training and testing. since many real-world applications often provide at least a small number of in-domain samples, and our focus in this work is specifically on no-data or lowdata scenarios, we have limited our experiments to in-domain settings. investigating out-of-domain performance with such methods is important for future work as it involves challenges such as distribution shift and domain adaptation that warrant separate study. finally, we note that results show only moderate gains over baselines on the ats and auroc metrics, indicating room for improvement when using the estimated confidences to distinguish between correct and incorrect responses. further studies are needed to understand fundamental limitations of consistency-based uq for llms.