SPUQ : Perturbation-Based Uncertainty Quantification for Large Language Models (2024)

Abstract: In recent years, large language models (LLMs) have become increasingly prevalent, offering remarkable text generation capabilities. However, a pressing challenge is their tendency to make confidently wrong predictions, highlighting the critical need for uncertainty quantification (UQ) in LLMs. While previous works have mainly focused on addressing aleatoric uncertainty, the full spectrum of uncertainties, including epistemic, remains inadequately explored. Motivated by this gap, we introduce a novel UQ method, sampling with perturbation for UQ (SPUQ), designed to tackle both aleatoric and epistemic uncertainties. The method entails generating a set of perturbations for LLM inputs, sampling outputs for each perturbation, and incorporating an aggregation module that generalizes the sampling uncertainty approach for text generation tasks. Through extensive experiments on various datasets, we investigated different perturbation and aggregation techniques. Our findings show a substantial improvement in model uncertainty calibration, with a reduction in Expected Calibration Error (ECE) by 50% on average. Our findings suggest that our proposed UQ method offers promising steps toward enhancing the reliability and trustworthiness of LLMs.

## introduction large language models (llms) @xcite @xcite @xcite have demonstrated remarkable success in various natural language processing tasks, such as text generation and question answering. however, a significant concern with llms is their proclivity to hallucinate, or make confidently wrong predictions @xcite @xcite , even for advanced models like @xcite . to address this 1 https://github.com/intuit-ai-research/spuq figure 1 : overview of uncertainty quantification techniques: one-pass @xcite @xcite , sampling-based @xcite , and our spuq method. spuq addresses both epistemic (via perturbation) and aleatoric (via sampling) uncertainties. aggregation yields the total uncertainty, distinguishing spuq from traditional methods focused mainly on aleatoric uncertainty. uncertainty in machine learning models can be categorized into aleatoric (data-wise) and epistemic (model-wise) uncertainty @xcite . in the case of llms, aleatoric uncertainty arises from the inherent variability of natural language, where multiple or even an infinite number of valid outputs can exist for the same input. existing work has proposed methods relevant to aleatoric uncertainty. a common approach is to use the (normalized) generation likelihood (or the reciprocal of perplexity @xcite ) as an uncertainty measure. however, many popular llms, such as chatgpt and gpt-4, do not provide access to token probabilities in their apis, rendering this approach inapplicable. additionally, sampling-based methods @xcite @xcite examine the deviation among multiple outputs generated from the original input. however, when an llm makes a confidently incorrect prediction, resampling tends to yield similar results, leading to an overconfident and poorly calibrated score. epistemic uncertainty, on the other hand, is due to the lack of knowledge or suboptimal modeling. it captures the uncertainty that arises from limitations in the model's ability to learn from the data and generalize to new situations. existing literature has linked epistemic uncertainty with adversarial examples @xcite and quantified it by introducing perturbation during inference time @xcite @xcite . however, there has been limited exploration of this uncertainty in the context of llms. in this paper, we introduce a novel approach, sampling with perturbation for uncertainty quantification (spuq), as depicted in fig. 1 . spuq addresses both aleatoric and epistemic uncertainties in llms by amalgamating existing methods that assess uncertainty from disparate angles. our method tackles epistemic uncertainty through a perturbation module tailored for llms. this module, inspired by (cremades @xcite , gauges model sensitivity to input perturbations. coupled with this is our approach to aleatoric uncertainty, which adopts the principles of sampling methodologies @xcite , enhanced by an aggregation module. within the perturbation module, we adjust temperature and/or prompts using a variety of techniques, such as paraphrasing, dummy tokens, and perturbed system messages. our aggregation module not only generalizes the"exact match" method used in existing sampling approaches @xcite as a more general intersample matching method but also incorporates intra-sample metrics @xcite . in essence, spuq distinguishes itself from traditional sampling-based uq methods through its innovative perturbation and aggregation modules. the former ensures a comprehensive treatment of epistemic uncertainty, while the latter adapts the method to a wider array of text generation tasks. we perform thorough experimental studies on different perturbation and aggregation techniques using multiple question-answering datasets for various llms, such as gpt and palm series @xcite . through comparing our perturbation-based uq method with existing baselines, we exhibit the efficiency of our approach in enhancing model uncertainty calibration, reducing expected calibration error (ece) by 50% on average. in summary, our key contributions in this work are threefold: 1) we introduce a novel perturbation sampling-based uncertainty quantification (spuq) framework tailored for llms. this framework effectively addresses both aleatoric and epistemic uncertainties, leading to improved model uncertainty calibration. 2) we unify and generalize existing methods, integrating them into our perturbation and aggregation modules, making them adaptable to a broad range of llm tasks. 3) we demonstrate significant improvement in uncertainty calibration through comprehensive experimental studies on multiple datasets across different llms. ## spuq: sampling with perturbation for uncertainty quantification figure 2 : options associated with the perturbation (section 2.1) and aggregation modules (section 2.2) of the spuq method. in this section, we introduce the sampling with perturbation for uncertainty quantification (spuq) technique, as depicted in fig. 1 and elaborated in algorithm 1. spuq operates on an llm's original input temperature t 0 and prompt x 0 . it derives perturbed variants (t i , x i ) using the perturbation module and aggregates the outputs y i from both the original and perturbed llm inputs to compute a confidence score, c. this score rep- ## experimental setup large language models (llms) we conduct experiments using five llms foot_5 : gpt-3 @xcite ) (text-davinci-003), chatgpt (ope-nai, 2022) (gpt-35-turbo-v0301), gpt-4 (ope-nai, 2023) (gpt-4), palm2 @xcite ) (text-bison), and palm2-chat (chowdhery et al., 2022) (chat-bison). datasets we evaluate our methods on a summarization task dataset, xsum @xcite , and four question-answering datasets: dev set tuned spuq hyperparameters test performance t perturbation x perturbation aggregation ece ↓ correlation ↑ i +0.3 paraphrasing inter-sample, rougel 0.082 0.690 ii +1.3 paraphrasing inter-sample, rougel 0.102 0.670 iii +0.3 paraphrasing inter-sample, rougel 0.082 0.690 v +1.3 dummy tokens inter-sample, rougel 0.103 0.672 iv +0.3 paraphrasing inter-sample, rougel 0.082 0.690 baselines our spuq method is benchmarked against several established baselines: ## conclusion we introduced the spuq method to enhance uncertainty calibration in llms, achieving a notable reduction in expected calibration error by 30% to 70%. our ablation study linked this improvement largely to our perturbation mechanism, underscoring its role in addressing epistemic uncertainty. the application of spuq offers a path to more reliable llm outputs. future work should expand spuq's applicability on beyond present datasets with simple prompts, exploring its effectiveness across diverse tasks and complex prompt structures.