Uncertainty Unveiled: Can Exposure to More In-context Examples Mitigate Uncertainty for Large Language Models? (2025)

Abstract: Recent advances in handling long sequences have unlocked new possibilities for long-context in-context learning (ICL). While existing research predominantly focuses on performance gains driven by additional in-context examples, the impact on the trustworthiness of generated responses remains underexplored. This paper addresses this gap by investigating how increased examples influence predictive uncertainty—an essential aspect in trustworthiness. We begin by systematically quantifying uncertainty across different “shot” configurations in ICL, emphasizing the role of example quantity. Through uncertainty decomposition, we introduce a novel perspective on performance enhancement, focusing on epistemic uncertainty (EU). Our results reveal that additional examples reduce total uncertainty in both simple and complex tasks by injecting task-specific knowledge, thereby diminishing EU and enhancing performance. For complex tasks, these advantages emerge only after addressing the increased noise and uncertainty associated with longer inputs. Finally, we investigate the progression of internal confidence across layers, uncovering the underlying mechanisms that drive the reduction in uncertainty.

## introduction in-context learning has emerged as a pivotal paradigm for modern large language models (llms) in addressing real-world challenges @xcite . by presenting a few learning examples through carefully crafted prompts, llms achieve remarkable performance without requiring weight updates. the latest techniques of equipping llms with long-context capabilities have made strides @xcite , including continued fine-tuning @xcite , position extrapolation @xcite and innovative architectures @xcite this raises a natural question: can additional examples similarly reduce uncertainty in llms? one such area is long-context icl, also known as many-shot icl, which involves feeding llms with hundreds or even thousands of input-output pairs. this regime of icl allows llms to learn from large quantities of data once and could be deemed as a comparative alternative to fine-tuning methods. despite its potential, the properties of many-shot icl remain largely unexplored. while several studies have initiated preliminary investigations in this area, which mainly focus on performance gains from extra examples @xcite , critical aspects such as trustworthiness and reliability of generations by llms @xcite remain unexamined. systematic investigation of these aspects is essential for advancing our understanding of long-context icl and paves the way for its wider adoption in high-stake applications. to fill this blank, we quantitatively examine the impact of increasing scales of in-context examples on llms' confidence through faithful uncertainty quantification (uq) approaches. by incorporating model parameters, configurations, and various demonstration sets, we approximate the predictive distribution in the output space. then we compute entropy to measure total uncertainty (tu). building on the framework proposed by @xcite , we employ a bayesian framework to disentangle two core components from tu for many-shot icl: epistemic uncertainty (eu) and aleatoric uncertainty (au). eu arises from insufficient evidence or knowledge during model training, while au stems from the inherent randomness and variability of the data @xcite in fig. 2 . our analysis reveals that the reduction in llms' uncertainty with more examples is primarily driven by a main decrease in eu. these examples enrich task-specific knowledge, thereby lowering eu, which in turn reduces tu and enhances performance. furthermore, we demonstrate that the performance gains are attributed to increased informational content rather than extended context length. to explore the mechanisms behind reduced uncertainty, we project the residuals from all model layers into the vocabulary space, visualizing the evolution of internal confidence. the results reveal that long-context icl enables llms to concentrate more logit mass on the correct answer and amplify the disparity between the correct response and distractors, effectively reducing uncertainty in predictions. this study represents one of the earliest efforts to examine long-context icl through the lens of uncertainty. the core research questions addressed are as follows: long-context icl the significant advancements in equipping llms with long context capabilities have expanded the potential for research in previously constrained areas, such as repositorylevel code understanding and multi-document qa. for icl, an important emergent ability for llms @xcite , the extrapolation of context length enables the investigation into its performance limits and learning dynamics as the number of demonstrations scales. several studies have initiated preliminary investigations in this area. @xcite , for instance, demonstrates notable performance gains with many-shot prompting across various generative and discriminative tasks using gemini 1.5 pro @xcite . in parallel, @xcite offers valuable insights into the properties of many-shot icl, particularly examining the influence of example retrieval and demonstration order. on a more optimistic note, @xcite concludes that many-shot icl can facilitate efficient adaptation of multimodal foundation models to new applications and domains. however, the benefits of long-context icl are not universally positive. @xcite argues that long-context models encounter difficulties with extreme-label classification tasks, especially when large label spaces are involved. uncertainty quantification uq has been extensively studied in traditional machine learning @xcite @xcite , which predominantly concentrates on estimating models' confidence and uncertainty in its prediction, called total uncertainty. total uncertainty can be decomposed into two key components: epistemic (model) uncertainty and aleatoric (data) uncertainty @xcite . the advent of llms has introduced new challenges in quantifying uncertainty, particularly due to the sequential and context-dependent nature of generative processes. recent advances in uq research can be categorized into two main approaches: black-box and white-box methods. black-box uq quantifies uncertainty by measuring the agreement across multiple generation samples @xcite , whereas white-box approaches assess internal model states or logits to capture intrinsic uncertainty @xcite . ## interpretability view for uncertainty in k-shot icl to investigate the mechanisms by which increased in-context demonstrations reduce uncertainty in llms, we aim to delve into the models' internal states, unraveling the underlying processes governing answer selection and generation in in-context learning, thereby offering a comprehensive and interpretable analysis of this phenomenon. ## further discussion clarifications while our work builds upon the framework in @xcite , our research specifically investigates the evolution of uncertainty in long-context icl, a topic that has not been examined to date. in contrast, ling et al. primarily focus on introducing a framework for decomposing uncertainty in few-shot icl. by shifting the focus to long-context scenarios, our study explores how uncertainty evolves as the number of in-context examples increases, thereby addressing an important yet understudied dimension of icl. ## conclusion this study investigates the impact of extra demonstrations on the confidence of llms in their responses. experimental results demonstrate that additional examples significantly reduce tu across both simple and complex tasks by integrating taskspecific knowledge. this reduction is primarily attributed to decreased model uncertainty, which enhances overall performance. however, in complex tasks, many-shot icl faces challenges in reducing tu due to a concurrent increase in au. analysis of the internal representations of llms reveals that many-shot icl not only reallocates greater logit mass toward correct responses but also enlarges the logit margin between correct answers and distractors, reflecting an increase in model confidence.