Adaptive P latt Scaling with Causal Interpretations for Self-Reflective Language Model Uncertainty Estimates (2025)

Abstract: As large language models (LLMs) are consumed by more users and deployed in increasingly autonomous capacities, their ability to self-monitor and ask for human intervention is of vital importance. Underlying this capability are fundamental skills like self-reflection and expression of uncertainty. In this work, we provide a formal analysis of LLM self-reflection for uncertainty estimation, using domain adaptation theory to model the shift between base predictions and reflective judgments. We use this to motivate a temperature scaling algorithm that calibrates uncertainty using comparisons between base predictions and LLM self-reflections. We evaluate our approach on challenging question-answering tasks requiring reasoning, demonstrating that our methods can improve calibration of uncertainty estimates and also offer improvements in human interpretation. More broadly, this use case shows how domain adaptation presents a promising analytical tool for understanding the underlying statistical properties of LLM self-reflections.

## introduction as llms are deployed to more users and given greater autonomy, their ability to self-monitor gains increasing importance. models need to recognize when they might be wrong and externalize this uncertainty in interpretable ways, reducing users' cognitive loads to enable reliable llm use @xcite . for example, this is especially true in safety-critical settings like healthcare @xcite . despite this, we still lack a clear understanding of how llm self-monitoring works, how to interpret it, and how to use it for reliable uncertainty estimation. existing methods primarily treat model reflections and uncertainty estimates as static, isolated outputs, overlooking the dynamic 0 sicilia partially completed this work at northeastern. 1 code: https://github.com/anthonysicilia/adaptive-platt nature of llm inference, where models can iteratively reassess answers to potentially improve accuracy @xcite . our work addresses this gap by introducing a theoretically-grounded method, which recognizes llm self-reflections as a source of dynamic, interpretable uncertainty. specifically, we propose a new calibration algorithm that leverages selfreflective outputs to improve the accuracy of uncertainty estimates, comparing these to base predictions during platt scaling @xcite . by formalizing self-reflection through the lens of domain adaptation @xcite , we explicitly account for changes between initial predictions and reflective judgments, framing llm self-reflections as a form of distribution shift. our method is a practical, lightweight instantiation of our theoretical inquiry that can be implemented via a single additional regression layer, incurring negligible computational costs to typical self-reflection workflows. interestingly, we also demonstrate how this technique can offer improved human interpretation, framing the technique as a causal analysis for when llms may be more likely to make errors. we apply our methods to a range of question-answering tasks with a focus on reasoning, challenging llm self-reflections to recognize when reasoning may be uncertain or incomplete. domains include logical tasks; difficult problems in topics like science, business, or history @xcite ; and tasks requiring reasoning about the future @xcite . to summarize, our contributions include: 1. we introduce a domain adaptation framework for analyzing llm self-reflections; 2. we use our analytical tool to propose an uncertainty estimation method that improves calibration by using reflective outputs; 3. we suggest an associated method for interpretation, drawing connection to causal analysis; and, we publish code to enable future applications. foot_0 2 background problem setup we focus on expressing uncertainty as a basic form of self-monitoring capability. in this setting, a language model is presented with a random query q and generates an answer a, with the goal of estimating the probability that a is correct. we follow typical evaluations @xcite @xcite , which treat the model's estimate as a probabilistic classification, measuring quality using proper scores like the brier score @xcite : basic approach uncertainty estimation methods for language models typically involve extracting a model-derived signal correlated with correctness, then calibrating this signal to output a probability @xcite . we focus on direct forecasts, where the model is prompted to directly express confidence via tokens in its generated output (e.g., a number 1-10). this signal z qa is parsed and calibrated by platt scaling @xcite : ## proposed methods self-reflection we use self-reflection to broadly describe any prompting technique that asks language models to re-consider, reflect on, correct, revise, etc. their own previously generated answer @xcite @xcite . for example, this strategy would fall under post-hoc self-feedback in the survey by @xcite and is distinct from strategies that use reflection to improve de-coding @xcite or fine-tune a model @xcite . under this broad umbrella, with a focus on uncertainty estimation, we will assume the re-prompted language model generates a new string r conditional to the pre-existing dialogue qa, and then a new uncertainty signal ẑqar is formed. a simplified example is given below: q : what is 42 plus 111? a : ...so it's 156, i'm 70% sure. user : re-analyze & report confidence. r : ...due to this, i'm only 40% sure. here, ẑqa = 70% and ẑqar = 40%. we report our precise prompt for reflection, examples, and other prompts for our experiments in the appendix. theory our key insight is to recognize the shift in modeling problem that self-reflection entails. in conditioning our uncertainty signal ẑqar on the reflection r, 1 we intend to model the distribution with ϵ = | pqa -p qa |. this form of inequality is referred to as an adaptation bound @xcite . it communicates how errors change when distributions shift. in the current context, it tells us that even if we minimize ϵ perfectly by learning an effective calibration function ( eq. 3) , our error | pqa -p qar | can still be large because of the unaccounted for term involving f (q, a, r). reflective calibration to remedy this problem, we aim to account for the missing information in our platt scaling algorithm. the obvious solution is to use ẑqar in the platt scaling algorithm, since this is implicitly conditioned on all of q, a, and r via the language model. this baseline solution rewrites the right-hand side of eq. ( 3 ) as below causal interpretation under appropriate assumptions, the theory and estimation procedures discussed allow us to identify interpretable causes of llm errors. the basic idea is that uncertainty signals derived from llms can accurately reflect true uncertainties in language data due to llms large-scale training exposure, making them useful in estimating missing information for purpose of z qa z qar d c qa causal analysis @xcite . applied to our setting, we adapt this idea to the signals ẑ * and their interactions. so far, we've used our theory, and results from @xcite , to hypothesize the signals ẑ * are accurate enough to capture all information pertinent to predicting c qa -except for other slack in our bound, which is accounted for by using data-labels d. translated to the causal setting, we can think of ẑ * as co-variates and d as treatments of interest (figure 1 ). causal analysis allows us to use the model in eq. ( 9 ) to estimate "treatment effects," which in our case, correspond to the causal effects of data properties encoded in d. to do this, define the causal (log) odds ratio ## experiments models we use meta's llama 3.1 with 8b parameters and llama 4 scout @xcite . we also use google's gemma 2 with 9b parameters @xcite and mistral small 3. we access models via together ai and set @xmath0.7. ## discussion our experiments across bbh, mmlu-pro, and fortunedial show that reflective calibration (eq. 9) consistently improves brier skill score, with gains that are statistically significant in several cases. while improvements are sometimes modest in magnitude, they are robust across domains, and, importantly, come from an extremely lightweight and simple methodolgical choice: an added regression layer with (at most) tens of parameters. while beneficial in most cases, increasing the number of learned parameters inherently impacts the variance of performance (i.e., there is no free lunch, @xcite . this is evidenced by increased dispersion between the min and max model performance in practical considerations generally speaking, the method is easy to implement in real-world pipelines. it requires only two ingredients that practitioners already have access to: 1. model confidence signals, which can be either direct forecasts verbalized by the llm (e.g., i'm 70% sure) or logit-based scores for the model responses -we study the former in § 4, but other works have succesfully used logit-based scores during calibration @xcite ; 2. a single regression model, which requires fitting a logistic model with interaction terms by numerical mle, using hundreds of data points. with these in hand, inference requires one additional reflection prompt, doubling the number of forward passes for the llm, which is the clear computational bottleneck. thus, the total overhead is small compared to common llm costs and negligible if added to an existing reflection workflow. the impact of scale while we evaluated medium-scale llms on relatively small data sources, we expect scaling either of these dimensions would preserve our findings. in past work, larger models maintain consistent performance gains during both self-reflection @xcite and uncertainty calibration @xcite , so we do not expect our combination of these methodologies to present significantly different findings. the fact that the calibrating regression layer is independent of model size strengthens this hypothesis. with respect to data, we observe the (internal) diversity of the datasets we consider, whose subsets span law, engineering, health, history, social dialogues, task-oriented dialogues, etc. our statistically significant results on these varied data provide evidence that our method should remain applicable to other data and tasks. interpretation gains beyond raw calibration improvements, our causal framing provides interpretability not offered by other methods. for example, in mmlu-pro we were able to identify that computer science and law questions pose larger systematic challenges to llms than history or psychology. on average, a model will do worse on these domains, speaking to the contexts where a practitioner can trust their model outputs. this intuition is also precisely quantified through the regression layer's coefficients, which allowed us to rank domain difficulty in our case study. other potential applications could utilize these coefficients in llm interfaces to dynamically warn users about potential errors, use them in training pipelines to refine data (e.g., as a form of active learning), or use them in myriad model diagnostics, such as longitudinal studies of performance across model versions. compared to raw performance metrics split by data, our regression coefficients attempt to control for potential confounders -these would typically be commonalities across data domains (e.g., the presence of equations in both computer science and physics problems) that can bias interpretation of traditional performance metrics. lastly, we note that even when performance drops slightly (e.g., on fortunedial with mistral small), the interpretive benefits remain. a practitioner can always revert to simpler calibration strategies if performance decreases, but retain our method for interpretability. ## conclusion this work provides theory and conclusive empirical evidence to motivate a new reflective calibration algorithm that improves llm uncertainty estimates and enables causal explanation of model errors. both come at negligible computational cost. however, beyond lightweight diagnostic tooling, our contributions might be viewed as an entryway into rethinking how llms engage with their own understanding. the very act of asking a model to reconsider is not a simple a calibration trick, but a way to alter reasoning. indeed, our experiments suggest that reflection alters the model's reasoning distribution itself, uncovering uncertainty signals that static inference obscures. this suggests reflective uncertainty may function as a core element, and design lever, for multi-step llm reasoning.