M onte C arlo Temperature: a robust sampling strategy for LLM ’s uncertainty quantification methods (2025)

Abstract: Uncertainty quantification (UQ) in Large Language Models (LLMs) is essential for their safe and reliable deployment, particularly in critical applications where incorrect outputs can have serious consequences. Current UQ methods typically rely on querying the model multiple times using non-zero temperature sampling to generate diverse outputs for uncertainty estimation. However, the impact of selecting a given temperature parameter is understudied, and our analysis reveals that temperature plays a fundamental role in the quality of uncertainty estimates. The conventional approach of identifying optimal temperature values requires expensive hyperparameter optimization (HPO) that must be repeated for each new model-dataset combination. We propose Monte Carlo Temperature (MCT), a robust sampling strategy that eliminates the need for temperature calibration. Our analysis reveals that: 1) MCT provides more robust uncertainty estimates across a wide range of temperatures, 2) MCT improves the performance of UQ methods by replacing fixed-temperature strategies that do not rely on HPO, and 3) MCT achieves statistical parity with oracle temperatures, which represent the ideal outcome of a well-tuned but computationally expensive HPO process. These findings demonstrate that effective UQ can be achieved without the computational burden of temperature parameter calibration.

## introduction large language models (llms) have fundamentally transformed the way we interact with artificial intelligence, revolutionizing various domains, from content creation to complex problem-solving tasks @xcite @xcite . however, these powerful models can sometimes produce unreliable or incorrect outputs, raising concerns about their deployment in critical applications @xcite ; xiao and * equal contribution @xcite . while significant research efforts have focused on improving llms' accuracy through techniques like chain-of-thought prompting @xcite and retrieval-augmented generation @xcite , parallel work has emerged on developing uncertainty quantification (uq) methods to estimate model confidence as an indicator of potential errors @xcite @xcite . existing uq methods for llms can be used to predict the correctness of a llm's output, either under white-blox or black-box assumptions. they fall into two broad categories: single-sample and multi-sample approaches. single-sample methods analyze a single generation using metrics like perplexity or evaluating model's weight activations. in contrast, multi-sample methods, which we focus on in this work, rely on querying the model multiple times with the same input and non-zero fixed temperature sampling, to induce and measure diversity in the generations. to assess the effectiveness of uq methods in distinguishing between correct and incorrect model outputs, they are typically evaluated as a classification procedure using the area under the receiver operator characteristic curve (au-roc) metric @xcite . however, the impact of selecting a specific fixed temperature parameter is understudied, and our analysis reveals that temperature plays a fundamental role in the effectiveness of different uq methods across scenarios in which different llms are employed to solve different tasks. figure 1 exemplifies this behavior over four question-answering datsets and three models using the semantic entropy method foot_0 @xcite . the figure highlights three critical observations: (1) for a given model and dataset, performance varies significantly with changes in temperature; (2) no single temperature consistently optimizes performance across datasets for a given model; @xcite (3) no universal temperature yields optimal results across models for a given dataset. for instance, the falcon-40b model achieves peak performance on the triviaqa dataset at a temperature of 0.6, but requires a lower temperature of 0.3 for the svamp dataset. similarly, within the same triviaqa dataset, optimal temperature values differ across different models: falcon-40b performs best at 0.6, while falcon-7b-instruct achieves superior results at 1.0. this lack of robustness in maintaining consistent performance across different scenarios poses significant challenges for practitioners attempting to implement uq methods and highlight the need for more robust approaches to temperature selection. to address the challenges of selecting a specific fixed temperature in uq methods, we introduce monte carlo temperature (mct), a sampling strategy that dynamically varies the temperature during multiple sentence generations, allowing uq methods to generalize more effectively to different model-dataset combinations. this approach reduces sensitivity to specific temperature values and ensures more reliable uncertainty estimates. we evaluate mct against an oracle determined by selecting the temperature that yields the best results on the test set. by using an oracle as reference, we place ourselves in the most challenging evaluation scenario, as it represents an idealized outcome that hyperparameter optimization (hpo) may not achieve in practice. beyond this comparison, we assess mct against two alternative model-dataset agnostic approaches, that do not require hpo: the best on average temperature, which selects a single fixed value performing well across multiple models and datasets, and the fixed random temperature approach that randomly chooses a single temperature. our results demonstrate that mct consistently achieves statistical parity with the oracle, eliminating the need for expensive hpo. additionally, mct outperforms both the best on average temperature and the fixed random temperature strategies, further highlighting the benefits of structured temperature sampling. the paper is structured as follows: in section 2, we present an overview of multi-sample uq methods. in section 3, we introduce the mct approach and describe its implementation. section 4 details the experimental setup, including the llms, datasets, and evaluation metrics used. section 5 presents the results of our experiments. finally, in section 6, we discuss the implications of our findings, acknowledge limitations, and outline potential future research directions. ## multi-sample uq methods in this section, we present an overview of popular multi-sample uq methods that we selected to evaluate the mct sampling strategy. these methods represent a diverse set of approaches commonly employed for estimating uncertainty in llms. in this section, we define the concept of robustness in the context of uq methods and formalize the mct sampling strategy. ## experimental setup this section outlines the experimental framework employed to evaluate the performance of mct and related uq methods. we detail the configurations used for answer generation, the llms and datasets selected for evaluation, and the specific entailment and evaluation models utilized in the study. ## results in this section, we present the results of the mct sampling strategy, comparing its performance against three baselines: (1) the oracle temperature, selected to maximize test set performance, (2) the best on average temperature across modeldataset combinations, and (3) the fixed random temperature approach. first, we assess how closely mct approximates the oracle temperature and achieves statistical parity. then, we compare mct to the two baselines that do not rely on hpo. our results reveal that a previously optimal temperature does not necessarily generalize well across different model-dataset settings, as the best on average temperature still underperforms relative to mct. meanwhile, the random baseline highlights the drawbacks of uninformed selection, showing that arbitrary temperature choices lead to unpredictable and often suboptimal results.