Uncertainty Quantification for In-Context Learning of Large Language Models (2024)

Abstract: In-context learning has emerged as a groundbreaking ability of Large Language Models (LLMs) and revolutionized various fields by providing a few task-relevant demonstrations in the prompt. However, trustworthy issues with LLM’s response, such as hallucination, have also been actively discussed. Existing works have been devoted to quantifying the uncertainty in LLM’s response, but they often overlook the complex nature of LLMs and the uniqueness of in-context learning. In this work, we delve into the predictive uncertainty of LLMs associated with in-context learning, highlighting that such uncertainties may stem from both the provided demonstrations (aleatoric uncertainty) and ambiguities tied to the model’s configurations (epistemic uncertainty). We propose a novel formulation and corresponding estimation method to quantify both types of uncertainties. The proposed method offers an unsupervised way to understand the prediction of in-context learning in a plug-and-play fashion. Extensive experiments are conducted to demonstrate the effectiveness of the decomposition. The code and data are available at: https://github.com/lingchen0331/UQ_ICL.

## introduction large language models (llms) have revolutionized diverse domains by serving as general task solvers, which can be largely attributed to the emerging capability: in-context learning. by providing demonstrations of the task to llms as part of the prompt, llms can quickly grasp the intention and make corresponding responses to the particular task @xcite . in this paradigm, llms can quickly adapt to solve new tasks at inference time (without any changes to their weights). advanced llms, e.g., gpt-4 and llama, have achieved state-of-the-art results on lambada (commonsense sentence completion), triviaqa (question answering) @xcite , and many tasks in other domains @xcite . while in-context learning has achieved notable success, llms remain vulnerable to well-known reliability issues like hallucination @xcite . uncertainty quantification has emerged as a popular strategy to assess the reliability of llm responses. in the past two years, numerous works @xcite @xcite @xcite have been proposed to quantify the uncertainty of llm response. these approaches could return a confidence score or directly compute variance/entropy across multiple llm responses; however, they often overlook the complex nature of llms and their reliance on provided demonstrations in in-context learning, so that existing methods may not provide insights into the underlying causes or the interactions among different factors contributing to uncertainty. a natural question therefore arises: when llm uses in-context learning to predict a wrong answer with high uncertainty, can we indicate if it is caused by the demonstration examples or by the model itself? given llm's responses to a particular task, it's essential to decompose the uncertainty into its primary sources to address the question. specifically, aleatoric uncertainty (au) refers to variations in the data, often linked to the demonstration examples. as shown in figure 1 (a), llm's output can easily be disturbed by inappropriate demonstrations since the provided demonstrations do not cover all possible labels. the noise and potential ambiguity of these demonstrations could bring uncertainty, which, in turn, may hinder the accuracy of the response. in contrast, epistemic uncertainty (eu) stems from ambiguities related to the model parameters or different configurations. as depicted in figure 1 [0: sadness; 1: joy, 2: love; 3: anger]. ## uncertainty decomposition of in-context learning we first formulate the process of in-context learning as bayesian neural networks with latent variables. based on the formulation, we propose to decompose the predictive uncertainty into its epistemic and aleatoric components from the mutual information perspective, followed by a novel way to estimate both uncertainties based on the entropy of the prediction's distribution. ## experiments we evaluate the uncertainty decomposition procedure on realistic natural language understanding problems. by comparing state-of-the-art uncertainty quantification methods, we aim to examine what type of uncertainty is the most promising indicator of high confidence for llms. in addition, we also provide generalization analysis and two specific out-of-distribution detection applications. due to the space limit, extra experiments and experiment settings are provided in the appendix. ## conclusion we provide a novel approach to decompose the predictive uncertainty of llms into its aleatoric and epistemic perspectives from the bayesian perspective. we also design novel approximation methods to quantify different uncertainties based on the decomposition. extensive experiments are conducted to verify the effectiveness and better performance of the proposed method than others. we believe this research stands as a significant stride toward harnessing the full potential of llms while being acutely aware of their performance boundaries. for future works, we plan to extend our method to other forms of data @xcite and tasks @xcite to quantify the uncertainty.