SelfReflect: Can LLMs Communicate Their Internal Answer Distribution?
Authors: Michael Kirchhof
Luca Füger
Adam Golinski
Eeshan Gunesh Dhekane
Arno Blaas
Seong Joon Oh
Sinead Williamson
Venue: ICLR 2026 Poster (2026)
Keywords: LLM
uncertainty quantification
subjective uncertainty
benchmark
language model
Primary area: foundation or frontier models, including LLMs
TL;DR: We measure whether LLMs can output a string that summarizes their distribution of strings P_\theta(A|q) that they would output in response to a question.

Abstract: The common approach to communicate a large language model's (LLM) uncertainty is to add a percentage number or a hedging word to its response. But is this all we can do? Instead of generating a single answer and then hedging it, an LLM that is fully transparent to the user needs to be able to reflect on its internal belief distribution and output a summary of all options it deems possible, and how likely they are. To test whether LLMs possess this capability, we develop the SelfReflect metric, an information-theoretic distance between a given summary and a distribution over answers. In interventional and human studies, we find that SelfReflect indicates even slight deviations, yielding a fine measure of faithfulness between a summary string and an LLM's actual internal distribution over answers. With SelfReflect, we make a resounding negative observation: modern LLMs are, across the board, incapable of revealing what they are uncertain about, neither through reasoning, nor chains-of-thoughts, nor explicit finetuning. However, we do find that LLMs are able to generate faithful summaries of their uncertainties if we help them by sampling multiple outputs and feeding them back into the context. This simple approach shines a light at the universal way of communicating LLM uncertainties whose future development the SelfReflect score enables. To support the development of this universal form of LLM uncertainties, we publish the code that implements our metric for arbitrary LLMs under https://github.com/apple/ml-selfreflect .

Published as a conference paper at ICLR 2026
SELFREFLECT: CAN LLMs COMMUNICATE THEIR INTERNAL ANSWER DISTRIBUTION?
Michael Kirchhof Apple
Luca Füger
Adam Goliński Apple
Independent Researcher
Eeshan Gunesh Dhekane Apple
Arno Blaas Apple
Seong Joon Oh Tübingen AI Center
Sinead Williamson
Apple
ABSTRACT
The common approach to communicate a large language model's (LLM) uncer- tainty is to add percentage number hedging word to its response. But is this all we can do? Instead of generating a single answer and then hedging it, an LLM that is fully transparent to the user needs be able to reflect its internal belief distribution and output summary of all options it deems possible, and how likely they are. To test whether LLMs possess this capability, we develop the SelfReflect metric, an information ceermation-thoortiic distance between a given summary and a distribu- tion over answers. interventional and human studies, we find that SelfReflect indicates even slight deviations, yielding a fine measure of faithfulness between a summary string and an LLM's actual internal distribution over answers. With SelfReflect, make resounding negative observation: modern LLMs are, across the board, incapable of revealing what they are uncertain about, neither through reasoning, nor chains-of ofttuuggsss that LLMs are able to generate faithful summaries of their uncertainties if we help them by sampling multiple outputs and feeding them back into the context. This simple approach shines a light at the universal way of communicating LLM uncer- tainties whose future development the SelfReflect score enables. To support the development of this universal form of LLM uncertainties, we publish our metric under ht httttt:///ttthhhit com www.pppppleeeeeem sew.ee/eeeeelllffeeeeleeeee
explicit finetuning. However,
do find
INTRODUCTION
When large language models (LLMs) are uncertain about a response, either because the query is ambiguous or because they are factually unsure, they should indicate Consider the example in The LLM's internal distribution comprises a variety of answers, so it is not enough to just output the greedy response. While existing uncertainty quantification approaches augment the greedy response (or any other single sample from the distribution) with a numerical measure of uncertainty (Aichberger et al., 2024 Fadeeva al. 2023 Fomicheva et al., 2020 Malinin and Gales] 2020) or verbalize the confidence in the response (Lin al. 2022 Yona et al. 2024), this offers limited insight into the model's beliefs: we do not see the full range of cities the LLM believes are plausible, nor the variety of supporting information (c.g., that Paris hosts the French government).
Fig. 1
We believe we can do better than this. As motivation, consider the following comment on Gödel's proof on the incompleteness of number theory.
Gödel had the insight that a statement of number theory could be about a statement of number theory (possibly even itself), if only numbers could somehow stand for statements.
Hofstadter 1979)
Gödel's key idea was that statements of number theory are expressive of much more than just integers. The same holds for strings: An answer string generated by L.IM is expressive enough to descrihe

---

Published as a conference paper at ICLR 2026
User query q: What is the main city of France?
Paris.
The capital of France is Paris.
It's Paris, which bosts government and many commercial hubs.
LLM's internal distribution Py
Its main city is Paris.
The capital of France is Paris.
Marseille one of France's is most popular and vibrant cities.
It's Toulouse.
Normal (greedy) answer: "The cupital of Frunce is Paris!
Numerical uncertainty: ('The capital of France is Paris., 75%)
Verbalized uncertainty: Tmm very sure that the capital of France Paris."
Self-reflective uncertainty: Tm 75% sure that it's Paris, its capital and commercial hub, could also Toulouse or Marseille,'
Figure/ I:
LLMs have internal answer distributions about user queries. Rather than just sampling an output, possibly combined with percentage, LLMs should generate string that is self-reflective of their internal distribution, summarizing all possibilities and which they find the most likely.
distribution over all answer strings the LLM could generate. We can therefore use single string query q. We see this in the
to summarize the LLM's distribution Pg (A q) over responses "self-reflective uncertainty" example of Fig. A single string conveys the relative degrees of belief in different cities, and covers the detailed facts of all answers of the distribution. Communicating uncertainty like this, through string rather than number, is quantification so novel that there exists no way to benchmark it. Our contribution is thus twofold:
paradigm for uncertainty
First, define benchmark that evaluates whether given self-summary string faithfully represents an LLM's internal distribution over possible response The underlying challenge here is to measure whether a single string "carries the same information" as distribution over strings, in some information-theerrtiic sense that takes into account both mentioned facts and their relative likelihoods. Our theoretical analysis yields the SelfReflect metric. It scores how well sulf-summary string is predictively sufficient of distribution of answer strings. To ensure that this measure of faithfulness robust practice, we conduct controlled experiments on both free-form and closed-form question datasets. We find that the SelfReflect score precisely discriminates good from bad (and almost- good) summaries of answer distributions, and that agrees with human judgements, in both cases outperforming other possible benchmark metrics such as LM judges and embedding distances.
Second, we use the SelfReflect metric to test whether 20 modern LLMs can generate self-reflective uncertainty strings. We make resounding negative observation: Neither explicit prompting, nor reasoning, nor SFT and DPO fine-tuning enable an LLM to faithfully summarize its internal beliefs. Its output may have summary-style format, but it mentions arbitrary possibilities, not those that the LLM actually believes in. It is, however, possible to give honest insights into the internal answer distributions by explicitly i.i.d. sampling an LLM and returning this back for summarization.
These findings mark but the start of this new avenue uncertainty quantification, and, in extension, of fundamentally making LLMs aware of their internal uncertainties. We expect that future advances along our SelfReflect benchmark metric will unlock more honest and trustworthy LLM interactions.
RELATED WORK
2.1 UNCERTAINTY IN LLMS
Most work on uncertainty in LLMs associates a single numerical expression of uncertainty to specific string like the greedily decoded response. Since LLMs are, in essence, probabilistic next- token classifiers, one can attempt to read their uncertainty off their token logits (Aichberger et al.
'Throughout the paper, we use anthropomorphised language like "what the LLM deems possible" or "being honest". We do this only for brevety and giving intuitions. Technically, always mean "summarizing the answers that L.M could give if sampled multiple times".
2

---

Published as a conference paper at ICLR 2026
2024, Fadeeva et al., 2023
Fomicheva et al., 2020 Malinin and Gales
2020). These methods can
be extended to longer LLM
answers for example by searching for fact tokens and extracting their
logits (Fadeeva al. 2024) and made more human-readable by transforming the numeric uncertainty into a string like "I am very sure that..." " (Lin et al., 2022 Yona et al. 2024). Still, these approaches quantify the uncertainty of only single element of the L.M's internal distribution.
So how can the full uncertainty of the LLM's distribution be captured? Farquhar et al. (2024) cluster answers sampled from the LLM's internal distribution semantically and calculate an entropy over the clusters. This considers the full distribution over strings, but it still reduces the uncertainty to a single number and presents this number alongside a single string from the distribution. Moving towards richer uncertainty explications, Xu et al. (2024) generate multiple samples from an LLM, use GPT-4 to summarize the distribution of samples and train the I.I.M to output such summaries. Similarly, Yang et al. (2024b) train an L.M to output strings that delineate which facts it is uncertain about. This arguably one of the richest ways to express an LLM's uncertainty. But both papers, focusing on the generation of summaries rather than on evaluation, use simple LM judges to rate the summary strings. As we show in Section 4.1 LM judges can not discern how faithfully a string reflects a distribution over strings beyond relatively simple good vs bad cases. Our SelfReflect gives fetter-founded and more precise metric to compare whether a summary string contains the same information as the LLM's internal distribution, enabling further develop this new avenue of LLM uncertainties.
2.2 SUMMARIZATION
Testing whether summary of long document is good has long history in natural language processing (NLP) (Zhang et al.] 2024). Summaries are traditionally rated in terms of consistency with the long document, relevance of the chosen information, and fluency and coherence of their sentences (Fabbri et al., 2021). as rated by humans or recently by LM judges (Jain et al.) 2023). In modern L.M-generated summaries, fluency and coherence usually granted, so that the focus lays on the consistency and relevance of the summary, in other words, whether it contains the same information as the long document. This fundamental question dates back the Cloze test (Taylor 1953). This test, originally designed for human language learners, masks out words from the long document and asks to fill them in. Summarization metrics like BLANC (Vasilyev et al., 2020) run this test twice, once when conditioning an NLP model on the summary and once without. If the summary contains correct information, the NLPP model should fill in better words. The masked-out performance can be quantified either as accuracy gain Vassilev et al. 2020) or, more softly, pscudo bigliikeldhood (Shin et al. 2019; Wang and Cho | Salazar et al. 2020 Kauf and Ivanova 2023). Other recent metrics use masked-out tasks to estimate pointwise mutual information (Jung et al. (024).
Since our SelfReflect metric also quantifies the quality of summary, we base it off Cloze-like masked-out tasks. But there is twist: The summary string does not summarize another string but a distribution over strings Pa (A q). This means we must go beyond comparing s to a specific string ~ Po (A q). to quantifying how faithfully 8 represents the density over the string space that Po (A q) defines, i.e., to all possible answers and how likely they are. To this end, we re-think masked-out tasks from the lens of sufficient statisties in the following section.
DISTANCES BETWEEN SUMMARY STRINGS AND DISTRIBUTIONS OF STRINGS
Our main challenge to find distance that quantifies the extent to which a summary string carries the same information an LLM's internal answer distribution. We build theoretical foundation for sufficient statistics in string spaces in Section 3.1 and develop the SelfReflect metric in Section 3.2
3.1 SUMMARIES AS PREDICTIVE SUFFICIENT STATISTICS
Suppose we have an LLM (which we denote LLMg), prompted with a random query Q. We posit that this puts us in a state Oq, which allows us to sample random responses B. We are interested in summarizing this distribution over responses. Let (1:NN) = (A(1) A(N) E x be set of responses sampled from LLMg, where X' is the space of finite strings Consider a summarization
"These N samples may be generated independently and identically B, but we do not require this; for example, the distribution over subsequent answers could depend on the previous answers.
3

---

Published as a conference paper at ICLR 2026
function : XN + X that, given A(N,, generates summary S := (A(:.)) What criteria should d satisfy if its summaries are to exactly capture LLM3 distribution over B?
Continuing the example from Fig. we summary of A (1:N) should neither omit important details from the answer distribution nor add extra details. For ex- ample, a summary stating "The capital of France is Paris" would ignore the LLM's belief Marseille or Toulouse, whereas summary stating "The capital of France is Paris but for period in history, it was Orléans" would be adding unfaithful details. The same holds for the relative likelihond of answers: the ideal summary should state that the capi- tal of France is most likely Paris, and Toulouse Mar- seille, because this answer has a higher probability mass in the LLM's internal distribution. This indicates that an ideal summary should capture exactly the same information about the answer distribution as that con- tained the sampled answers. We can formalize this in terms of mutual information,
that an ideal
eo
4(:N)
S
B
Figure 2: Graphical model for the suf-
ficiency that SelfReflect quantifies.
Definition 3.1 (Ideal summary). An ideal summary S of answers A(IN) of an LLM satisfies
I [{A():); BB (S;
Here, I (Y; } denotes the mutual information between Y and Z. Intuitively, for any subsequent answer B from the LLM, the information about B contained in Al:N) is exactly captured by S.
This definition is closely tied to the notion of predictive sufficiency (Lauritzen 1974). whereby
statistic () of observations ( is called sufficient if it satisfies p (X X ((::))
p (X (((())))) for any subsequent observation X. In fact, we can reframe Definition 3.1
Proposition 3.1 (Connection to predictive sufficiency). For an ideal summary S of answers A (1-N),
I {A(I:)) I [S: = p (B AU:N) p (B|S)
Intuitively, the ideal summary a predictive-sufficint statistic of the answers A(:N) for B.
From Definition 3.1 and Proposition 3.1 we see that a measure of how much (B (1:N)) diverges from p (B) |S) would be a good metric for measuring how faithfully S reflects the sampled answers A(I:N) Towards this, we formulate a Cloze-task based on masked-token prediction that constitutes simple yet equivalent characterization of the desired predictive sufficiency. Let B; denote the ith word of B and let B_; == (By) denote all other words of the answer. We propose predicting the missing word B, from the rest of the words B_, with the extra context of either the sampled answers (IIN) their summary S. Identical behavior in this masked-token prediction task turns out to be equivalent to predictive sufficiency (and hence, Definition 3.1):
Proposition 3.2 (Informal; towards the SelfReflect metric). For answers AlI:N) and their summary S, under mild conditions on all involved distributions and support of B, we have:
p B A p (B
for all masking indices
(B. A0.N) B_.) p (Br (S (3)
Full details and proofs of Propositions 3.1 and 3.2 are given in Appendix A Proposition 3.2 motivates
us to measure the divergence between the distributions (B; S, B-:) and (B; A B_,) as
a tractable metric for the quality of a summary, forming the basis of the SelfReflect metrie.
3.2 THE SELFREFLECT METRIC
Proposition 3.2 tells us we can sequence of masked-out tasks to quantify whether a summary s contains the same information about LLM3's 's distribution Py (B 9) as a sequence of samples from that distribution. We approximate this task using second judge LLM, LLM,, to estimate the conditional distribution over masked-out words. Intuitively, irrespective of whether we show the sampled answers their ideal summary, judge LLM should predict the same masked tokens.
Concretely, we sample a new response at temperature from LLM,, mask out one word B,, and ask I.IM,, to predict B, given the remainder of the answer B_g, the query g, and either the summary
4

---

Published as a conference paper at ICLR 2026
slaat fire: fustrall lar prise wirletory carple 2o snarera tiia qosatico.
the Fire: fostral wirletery andis
question
the sustatat
fiLs insastation
cindidite
Ste that the firet fastral prime winiater
L.d. samples
f_3 "hhe firet histralian peine sinister, Ri
could alss Anstrww riaker Idrand Deak in
from py(Alg)
end)
pasted into contects
4_30 "The fiese pacaco to officially sacve as deiza Acstemisa
ci scart
miosing
Pill
task with maskeddout
vord
the
vord only baaed on the anamer you gave abora:
vord only baaed so the anaver you gave abore:
The lostralian Mistater Rdwind
was Australion Miniater " shole ssntecce.
poocide niasing word tot shols
naswer d..
Pleass provide <lim andi>
<i andl>
scart Paasaiatent
<iin
"aaccon" "Fister"
"Baztes"
"2aabar*
Predicted token vector: py00 b.).) = (0.70, 0.28, 0.01, ...)
Predicted token vector: Py(B; (g.a ).) (0.78, 0.18, 0.01, ...)
"uaskie
"Daakin"
Misggnofor W (B, 9.8, Py B: 9, (H,8), (5.)))))...))
Figure 3: To test whether summary string s contains the same information as a set of samples a (1:N), SelfReflect prompts an LLM twice. First, it provides the summary as context; next, it provides the concatenated samples. SelfReflect then compares the resulting distributions via a masked-out task.
8 or a sequence (1:N) of N samples from Po (A(1:N) [4). see Fig. This yields two distributions Py (B, Q= q, A = (1::N), B_, b-.) and p, (B, Q q, 5, B_ b_i). over the vo- cabulary space of L.L,,, which we compare using the W-Wasserstein distance We marginalize over B and index to satisfy the requirements of Proposition 3.2] Finally, we take the expectation over all summaries that summarization strategy &i writes for each question in the dataset:
MaeetRefleet (1)
E Q,AG B,
[W (p, Q (Q)) B; Q, A0:N) B B=.)]]
(4)
Here, zb is any method that makes LLMg output a summary of its internal distribution in response to a query We put set of N = 50 samples (l:N) per query into the reference context of LLM, and estimate Eq. (4) via Monte Carlo sampling. We iterate over M 50 samples of B and each of their words (except stopwords) to create masked-out tasks. We repeat this over 1000 queries per dataset average the final SelfReflect benchmark score. These are relatively conservative settings that take 67 minutes to compute on a node of 8 NVIDIA A100 GPUs In Appendix B we show that we can also reduce minutes or even below one minute the goal is not to reach a benchmark metric precise to multiple digits but rapid development or reward signals. Further, literature notes that Cloze- like evaluations are often limited by synonyms (Kauf and Ivanova 2023). so we post-hoc flatten p. with 5. We quantitatively find this improves discriminability since especially Instruction-tume LLM, models otherwise place too much mass on one specific masked-out word. We discuss further design choices in Section 7
We explore different choices of L and find that SelfReflect is robust to the exact choice, see the quantitative results in Appendix C and the qualitative example in Appendix We find that Qwen 2.5 Instruct (Yang et al. 20224) captures both textual details and the implicit relative certainties in summaries concatenated samples in its context even when they subtle. The 7B model provides results almost on par with the 72B model, so we choose it for efficiency.
4 CAN SELFREFLECT SCORES QUANTIFY HOW GOOD SUMMARIES ARE?
We now verify that the SelfReflect metric works as a benchmarking tool, based on three pillars: Distinguishing hand-crafted good and bad summaries on free-form questions, simplified study on multiple-choice QA answer distributions, and comparison to which summaries humans deem faithful. In all studies, we compare SelfReflect to several other possible benchmarking metrics.
'If IL.M is black-box model that only returns the top-predicted word, i.e., pJ are one-hot vectors, 1 Wasserstein comparison simplifies into accuracy that tests whether the two predicted words are equal. This why we choose I-Wasserstein distance over KL divergence, which is quantitatively equally good.
WWhile the link to sufficiency only holds if 1 depends only on a (L:N), the metric is well-defined whether the summary generation involves taking samples in-hetween or generating summary answer for g in other ways.
5

---

Published as a conference paper at ICLR 2026
Table 1: Given pairs of good and bad summaries, we measure how often the SelfReflect score, and
other benchmark metrics, correctly assign a better score to the good summary to verify that they work
as benchmarking metrics. We test multiple pairs of good and bad summaries, e.g., lacking possibilities or lacking details. Mean + 95% confidence interval. Per-dataset results are in Appendix G
Good summaries summaries
Good almost-good
Detailed truncated
Verbalized uncortainty vs only majority answer
Verbalized vs or-concateeatted
Pereentage vs or-concutenated
Metric
Summarization
93.33% s03
39. 28.72%--a.a 2755
53.05% 0415
19. 20.90=a---0
58. 38.125fga..om .004
64, 64.92% 26 793
InfoSSumm
99.87% 123
49, 49.24%%
15.71
27.75 e4.30s
10.99%
LM Judge
98.33%
47.3 17.322% 11.31%
59.92%
19.37 19.37%% s.
34.55%
35 08% =
Opt. Tramsport
80.16%
60.78% 60.78%
39 39.699-51..mm
48. 65..69
52. 22.88%-=7706 27.00r
69.1 19% 2
Embedding
96.50% eo.ors
65.49% 1.22%
65.65%
10.99% 4.4445
43.98% -7.045
36.65% 6.668
SR-log]
96.37%
85. $1.90%
86 66.649%-4..22%
58. 12% 12%
40,84 40,84%
SR-PMI
88.40% 1906
33 13.6455-2 a1%
53.44%
25.65% 1958
14.14% medk
20.42%%-25 77%
SR-sam SR-saaplling-free
88.26% =.18%
54.85% 1..0%
73 73.28%%-s..1o a..9%
38.74% s.0t
35 =8.77%
38.22% = a93t
SR-P(True)
65.29% mos
81.91%
69.475h-a 1e%
87. %
71.73%
86.39 %
SelfReflect
98.77% c07t
93.20 200
93.13% =333%
85, 85.34% lerk
72.77 96 a115
80. 80.10% 00:5
Baselines. While developing SelfReflect, we experimented with approaches from various roots for comparing a summary string 8 to set of strings ((2:N), First, we compare against multiple metrics from summarization literature that treat (1::N) as single document that summarized by s. Summarization (Jain et al. 2023) uses the judge LLM,, to rate the summary in terms of consistency. fluency, relevance, and coherence. LM Judge prompts LLM,, to rate how well s matches a (1:N) in one prompt, following the chain-of-thoughts prompt of Xu al. (2024). InfoSumm (Jung et al. 2024 uses masked-out task to estimate pointwise mutual- -ntiuualliioormttion between summary and document. Next, we turn to the neighboring field of calibration. Wang and Holmes (2024) argue that calibration can be seen as distance to a centroid. We implement this in Embedding by comparing embeddings of s to (1:N), Finally, from an Optimal transport perspective (Peyré et al. 2019), we let LLM,, split into a "distribution" over atomic statements and likelihoods, compute a pairwise entailment matrix and return the Earth Mover's 's distance to Pa (A q).
Ablations. We also ablate key characteristics of SelfReflect (SR). SR-logl replaces the Wasserstein distance over the whole logit vector with only the log probability assigned to the masked-out word given either context. SR-PMI (SelfReflect with Pointwise Mutual Information) even removes the one-by-one masked-out task and directly compares the log likelihoods of the full answers (1:N); analogous to Proposition 3.1, SR (R-samplinng-ree uses the masked-out task, but compares the masked- out logit vectors given the summary to predictions of LLM, given q, instead of putting sampled answers into the context of LLM,. SR-P(True) changes from a generative to a discriminative masked- out task, asking LLM,, whether several candidates words fit or not (via the P(True) method of (Kadavath et al. 2022) given either the summary or the samples. More details are in Appendix E
4.1 STUDY 1: DISTINGUISHING GOOD FROM BAD AND ALMOST-GOOD SUMMARIES
We first conduct an interventional study to test whether summaries that we know are good are judged as better than summaries that we know are bad. To this end, take 3x 1, 000 open-ended questions from Natural Questions (K (Kwiatkowski et al.) 2019)). TriviaQA (Joshi et al. 2017). and SimpleQA (Wei et al., 2024), and let Qwen 2.5 7B Instruct generate 50 answers each. We then give these answer distributions to Gemini 2.0 Flash and prompt it to generate good summaries, containing all possibilities, details, and relative likelihoods, and bad summaries, which alter key facts of the good summaries, but keep their remaining style (human- -writtnn summaries reach equivalent results in Appendix [F). We then calculate which score SelfReflect gives to the summaries, and in how many percent of the good-bad pairs it correctly gives the good summary better (lower) score than the bad one.
Table I shows that SelfReflect correctly discriminates good from bad in 98.77% of cases. But several other baseline metrics also score over 90%. So we make the task harder by comparing good to almost-good summaries, which only contain facts that are faithful to the answer distribution, but leave out some possibilities and details SelfReflect gives the good summary better score than the almost-good good summaries in 93.2% of all questions. The other metrics, including the LM judge used in literature, can no longer distinguish these fine-grate -na-ine quality differences and are thus not good for benchmarking. We ablate this multiple times, finding the SelfReflect score also correctly notices when summary does not mention all written details, when it does not mentions all options
6

---

Published as a conference paper at ICLR 2026
but only the majority answer. Even when summary mentions all options ("It is ... ... or "..",, SelfReflect assigns a yet better score to a summary that also faithfully delineates in words or numbers which options are how likely. The SelfReflect score picks these subtle differences (last five columns) up better than all other benchmarking metrics, matched only in some cases by its own SR-P(True) ablation. Further, all these tests checked whether SelfReflect can distinguish the quality of individual summaries. In later benchmarks, which average over thousands of summaries per method, averaged Sel/Reflect scores will become even more exact by the law of large numbers, making Sel/Reflect a precise benchmarking tool that allows to iteratively develop cenmary-generating methods.
4.2 STUDY 2: DISTANCES OF MULTIPLE-CHOIC DISTRIBUTIONS
Next, we investigate SelfReflect in a narrower setup. We generate 2 1,000 answer distributions for MMLU (Hendrycks et al. 2021), a multiple-choice dataset with choices A, B, C. and D for each question, with Gemma 3 12B (non- (non-Instruct) (Gemma Team et al. 2025), and multiple-
Table 2: Agreement (rank corr. between SelfReflect, and others, and special bench- mark metric for MMLU. Mean + 95%.
Metric
Per Question
Whole Dataset
Qwen 2.5 7B Instruct. Since MMLU choice dataset, we can sample the LLM multiple times to obtain a simple categorical distribution. We then create summaries that either talk about this distribution "The answer is most likely C (54% sure), but it could also be B (32% sure) or A (14% sure)." or that mention the most likely answer only, are overconfident, or give random percentages. This gives range of different- quality summaries that the benchmark metrics have to tell apart. The categorical setup of MMLU also allows calculate a reference benchmark metric, namely the Wasserstein distance between the percentages mentioned in the summary and that of the real distribution. This lets us test if the SelfReflect metric and the other baselines agrees with the true distance in this special case. Specifically, we report the rank correlation between them and the reference metric.
Summarization
0,45+0.0s
0.80=0.00
LM Judge
0.76+0.c2
1.00-0.00
Opt. Transport
0.67+
0.82=0.00
Finhedding
-0,24gooo1
0.19_.0.2
SR-logl
0,5922.ca
1.00=.0.00
SR-PMI
0.07
0,20
SR-sampling-fice SR-P(True)
0,51+
0.80-0.00
0,57+00.a
1.00=z0.0
SelfReflect
0.65+s..o
1.00=0.00
Table shows that most metrics have a positive rank correlation with the reference metric. The LM judge metric even slightly outperforms SelfReflect on this particular task, indicating that SelfReftect may be slightly noisy on individual questions when summaries contain exact probabilities. However, when we compute the average score across all questions, as it will later be used in the benchmark, SelfReflect, like two of its ablations and LM Judge, achieves a perfect agreement with the reference metric. This shows SellReflects generic power benchmark metric, even in this special case.
4.3 STUDY 3: DO THE RATINGS ALIGN WITH HUMAN RATINGS?
Finally, we assess whether SelfReflect scores are aligned with human judgements. We conduct user study using 200 open-ended questions from the TriviaQA dataset (Joshi et al.) 2017). For each question, we generate ten sample responses using Phi-4 (Abdin et al. 2024), and four summaries: good summary and had summary, generated using Gemini 2.0 Flash as Section 4.1 a greedy summary, i.e., the greedy response of Phi-4; and Chain of Thought (CoT) summary, using Phi-4 to reason about possible answers and then summarize its reasoning. Note that the greedy and CoT summaries not based on the actual samples. All prompts provided in Appendix E Raters were shown the question, the ten sample answers, and two of the summaries, and asked to choose which best summarized the set of samples. Each question/summary combination was evaluated by 5 raters. To assess agreement between human raters, we calculate Krippendorff's a Alternative agreement metrics such as Cohen's 's kappa Fleiss' kappa are not appropriate here since each rater only rates a subset of the combinations. We then calculate Krippendorff's s a between the majority human preference and that of SelfReflect and other scores. Further details are in Appendix !|
As we see from Table 3 SelfReflect has the highest overall alignment with the majority human judgement (a 0.690). This is close to the inter-human alignment (a = 0.723) and significantly higher than any of the competing methods or ablations. Looking into the individual summary types, we see all metrics other than SR-P(True) have good alignment with humans on the bad vs good, bad vs greedy, and bad vs CoT comparisons. However, the other metrics show poor agreement
7

---

Published as a conference paper at ICLR 2026
Table 3: Agreement of metrics with human preference (consensus over five raters) on pairwise summary preference task, using Krippendorff's 's (values in [-1, 1]; positive numbers indicate agreement). Also shown is Krippendorff's a between individual human raters. Mean + 95% CI.
all
bad vs good
bad vs greedy
bad vs CoT
good vs greedy
good vs CoT
greedy vs CoT
Summarization
0.4
0.950_..0s
0,910ga ose
0.9402000
-0.21 150
-0.067.0
0.260
LM Judge
0.940=v..s
0.920=0.0x
0,930zu00
-0.063aw.1ez
-0.015
0,267 c2e
Opt. Transport
0.487
0.850_..006
0.779 م0 oes
0.679 101
0.098 1ss
0.265 us
0.191 166
Embeddings
0,435
0.750=uv0st
0.79920.cer
0.477
-0.36322011e
0.33 =
0.490
SR-PMI
0.4 0.436eauoss
0.820
0.890 =0.007
0.769 0.769=a.050
-0 -0,246-0.1as
0.029
0.246ea 114
SR-sampling-free
0.530
0.829
0.870
0.799
0.025 n.:14
0.241
0.340
SR-P(True)
-0.032=0.02
-0,029
-0.335
-0.477-a0.12p
0,3 L0.447
0.409
-0.024ao.145
SelfReflett
0.690
0.990
0.850
a.850
0.489 co.151
0.599
0.329
Human vs human
g.7
0.988_.0013
0.906 eas
0.441 1000
0.636 o61
0,45210000
with humans the more nuanced good greedy and good vs CoT. For all pairs of summary type, SelfReflect is close to inter-human agreement and either the most aligned with the majority human preference, or has overlapping 95% confidence intervals with the most aligned metric.
5 CAN LLMs GENERATE SELF- RESPONSES?
Now that we have metric to benchmark how well summaries summarize the distribution of LLM answers, we explore the performance of different summarization methods, that is: can one (somehow) make LLMs reflect on and summarize their own internal distributions?
5.1 EXPERIMENTAL SETUP
We distinguish two broad categories of methods: A) Sample & summarize: draw multiple independent samples from the LLM,, and then feed them back to the LLM to summarize them, B) Single-dccoding: methods which utilize only one decoding, requiring the I.IM to reflect on its internal distribution on its own. We consider three single-decoding methods: a) Basic: prompt asking the LLM for a summary of all possible answer options; b) CoT: prompt inducing chain-of-thoughh reasoning about the possible answers and then summarizing them; c) Greedy: Simply return the answer without summarizing all possibilities; we use this as baseline. The Greedy baseline is. in fact, strong: On questions where model has unimodal distribution on a specific answer, Greedy is in fact the best possible summary of this distribution and achieves a competitive SelfReflect score. To account for this, we report the percentage of answers where we observe such "Po (A q) unimodal" distributions per LLM. We evaluate all summarization methods via the SelfReflect score 3x 1000 randomly chosen questions from Natural Questions, SimpleQA, and TriviaQA (retrieval-aus -augreeee--eeeeeeed generation experiments on HotpotQA are in Appendix M). We use the same LLM to sample the answers to the question and generate the summaries in order to assess whether LLMs can access and describe their own internal distributions. We publish all benchmarking code upon publication. More details are in Appendix J
5.2 RESULT: LLMS CAN ONLY ACCESS THEIR INTERNAL DISTRIBUTIONS WITH SOME HELP
As we see in Table Sample & summarize is able to create summaries that faithfully reflect the model's 's internal uncertainty, consistently outperforming the Greedy baseline. In fact, its score matches that of humans asked to summarize samples from an LLM distribution, with humans achieving 90 10-3 when summarizing Qwen 2.5 72B Instruct answer distributions and Sample & summarize achieving 88- 10 on the data-split of Appendix F However, Sample & summarize helps the LLM in so far that it explicitly samples it i.idd. and provides the samples back as context summarize.
is of particular interest we can generate such self-reflective outputs without needing to sample in- between, for runtime and elegancy. Table unveils resounding negative result: No single-decoding methods is able to out-perform the Greedy baseline, corroborating that LLMs are not able to fully verbalize their own uncertainty by themselves, despite our best efforts optimize the prompts.
Maybe this task is too complex for an instruction-tunii LLM. We thus turn to reasoning models, asking them to reason about all possibilities and then output summary. But Table 5 reinforces
8

---

Published as a conference paper at ICLR 2026
Table SelfReflect score + (x10-a for readability). The results in small font are relative to Greedy. Pa (((g) unimodal is the proportion of questions for which the LLM always gives the same answer.
Model
Pe (A g)
Single-decoding methods
Sample summarize
unimodal
Greedy
Basic
CoT
20
Qwen2.5 0,5B Instruct
Yang et al. 2024a
7%
96
95-1
94-2
96-o
96
Qwen2.5 1.5B Instruct
Yang el a Yang
2024a
94
94
87
Qwen2.5 3B Instruct
27%
97
99
99-22
91
Qwen2.5 7B Instruct
Yang ct al 20248
36%
96
9943
101
91-5
90-c
Qwen2.5 5 14B Instruct
Yang Cf a1
2024B
52%
92
97+5
99-7
86-6
85-7
Qwen2.5 32B Instruct
Yang
20244
49%
96
102+a
105-
91-5
91-3
Qwen2.5 72B Instruct
Yang
20244
91
94+8
96-5
85-6
84
Phi 14B (Abdin
36%
92
92-0
93-1
85-7
84-
Ministral 8B Instruct 2410
Ciang et al.
2024
25%
107
106-1
105-2
101-6
100-7
Llama 3.1 70B Instruct
(Mel 20/44
51%
92
92-0
95-3
87-5
87-4
Llama 3.3 70B Instruct
(Meta AT 2024b
63%
94
98 (4
104
89
Llama Scout
16e
Instruct (Mcla
2025
91
961
101 130
88
Gemma 3 IB Instruct
(Gemma Team
2025
20%
129..13
1294.3
117-1
111
Gemma 3 4B Instruct
(Gemma Team ct
2025
52%
108
124.18
128+20
101-7
100-
Gemma 3 12B Instruct
(Gemmna Team el al.
2075
59%
105
116
121
102.
101
Gemma 3 27B Instruct
(Cremma Ieam ct al.
2025
71%
100
113
120
97--
96
Generation time (soconds)
1.56
1.59
2.48
3.65
4,50
Length (characters)
104.79
195.12
303.09
174.70
219.22
Table 5: SelfReflect score 1 (x103)) of RIVR models averaged over TriviaQA, NQ & SimpleQA. Greedy is generated w/o reasoning. Basic and Sample & Summarize reason and output a summary.
Model
Single-decoding methods
Sample & Summarize
Greedy
Reasoning
N 10
N 20
QwQ 32B (Qwen Team 2025b
96
105
91-6
90-6
DeepSeck RI Distill Qwen 32B
DecpSeck-Al
2025
108-12
91-5
90-s
Qwen3 32B (Reasoning enabled)
(wen Ieam 2025g
96:
85
Qwen3 8B (Reasoning enabled) Qwen Team 2025a
103
104-1
90-13
89-14
Generation time (scconds)
1.96
3.60
6.99
8.57
Length (characters)
107.56
224.98
287.31
350.98
negative result. Reasoning models do not perform any better. Qualitatively, summaries produced by reasoning models are similar to the instruction-tun LLMs with CoT prompts: they list possibilities, as prompted, but these possibilities are not faithful to the LLM's actual internal distribution.
Last, we attempt to explicitly train LLMs output self- summarums...s. We take a dataset of 10,000 Sample & summarize summaries from TriviaQA or Natural Questions (SimpleQA is too small) as good examples and perform supervised finetuning (SFT) and/or direct preference optimization (DPO, against 10,000 greedy answers as negative examples) Qwen 3 8B non-reasoning model. Appendix shows that SFT reduces the SelfReflect score on the train data but neither on held-out validation questions from the same dataset nor out-of-domain questions from the other dataset. This suggests that the model memorizes individual summaries rather than learning a general mechanism for accessing and summarizing its internal distribution. These experiments show that generating self- reflective summaries that are faithful to the model's internal uncertainty is a non- trivial new challenge.
5.3 IF IT FAITHFUL, THEN WHICH ANSWERS DOES CHAIN-OP-THOUGGI LIST?
To understand our resounding negative result further, compare summaries and the true internal distributions. As example case, we use CoT summaries from Qwen2.5 5 72B Instruct, our largest model.
We first test whether Col' correctly captures the spread of the answer distribution, i.e., whether it focuses on a single answer when the true distribution is unimodal and includes multiple options when the true distribution is multimodal. We let Gemini 2.0 Flash classify whether the CoT summaries and a (I:N) are certain (only mentioning different options, see also Appendix K). Figure 4 shows that for 36% of the questions, its summary is uncertain even when the answer distribution samples not, meaning it suggests multiple answers
answer option) uncertain (mentioning semantically

---

Published as a conference paper at ICLR 2026
options that do not have high probability under the true distribution. The same holds in reverse: in fact, the cross-table reveals that CoT generates certain or uncertain summaries nearly independently of whether the model's ' internal distribution is actually certain or uncertain.
Second, we investigate the possibilities mentioned a sum- mary. The most important possibility cover is the ground- truth answer, so we use it as an anchor for this analysis. Fol- lowing the best practices of Santilli the RougeL-Recall on Natural Questions short answers, i.e.
(2025), measure
Answer Distribution uncertain
Cortain
18%
36%
the longest substring of the true answer that appears mary, as percentage of the true answer's 's length. We find that
sum-
Greedy answers have an average overlap of 59.5% with the true answers. Basic summaries have 62.0%, Coff summaries 64.0%, and Sample & Summarize summaries 65.6%. Evalu- ating with an LM Judge instead of RougeL-Recall shows the same trend, rating that 71.3%. 72.2%. 74.1%, and 76.0% of the summaries include the true answer. In other words, summaries of the LLM's s internal distributions are going in a promising direction in that they cover the true answer more often, they are just not faithful to the model's ss internal uncer- tainties (yet).
7%
39%
Cortain Uncertain CaT Summary
Figure 4: How often Qwen2.5 72B In- struct answer distributions span mul- tiple possibilities vs how often their CoT summaries do.
OUTLOOK
We present SelfReflect, a metric that judges how faithfully a single string represents a distribution over output strings. SelfReflect is intended to guide the field Developing methods to make LLMs honestly describe all possible answers to a prompt in one string. We have seen in our benchmark that this is hard task, but solution to this problem would be fundamental building block in many applications: provides a numan-interpretable account of LLM be useful in building appropriate trust in the L.M'' 's outputs. The string can
avenue of expressing uncertainties:
uncertainty, which also be fed back to the LLM. for example to reason about follow-up questions when user query is ambiguous. Listing all output possibilities also core necessity for conformal approaches, which are popular for classification but less explored for LLMs where the span of possible outputs is not immediately available. Finally, an accurate description of distribution can also be recast into numeric uncertainty value, thus generalizing traditional numeric and verbalized uncertainties.
LIMITATIONS AND DESIGN CHOICES OF THE SELFREFLECT SCORE
To outline the limitations of work, first note that I-Wasserstein-baase SelfReflect scores not directly interpretable without basclines. A simplified version, like the percentage of equal top- predicted words using either summary answer samples, would give more standardized values in [0,1]. However, we found that such an approach is less sensitive to differences in good vs almost- good summaries, rendering less useful benchmark metric.
Second, seen from a summarization literature perspective, our SelfReflect metric intends to capture whether a summary faithfully represents the information of the model distribution. It does not intend to capture how short a summary is, so that concatenating 50 i.i.d. sampled answers an adversarial summary would probably optimize the SelfReflect score without being a useful summary. From summarization literature metric like the summary length, so we appeal to always report summary lengths and qualitative samples along with the SelfReflect metric, as we do in this paper's results tables.
know that this is an orthogonal aspect that is better captured in a second
Third, we repeat that the faithfulness we measure is with respect to an LLM's s subjective uncertainty. We intentionally did not develop SelfReflect to quantify objective truthfulness, with the outlook that larger LLMs approximate their training datasets better and better, such that more faithful summaries of subjective uncertainties will ultimately lead to better objective uncertainties.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
We intend to lay a foundation for a new avenue of communicating uncertainties with our work, and enable future researchers contribute to it. Thus, publish code compute SelfReflect scores for arbitrary LLMs and summary-generaine code, we have added all prompts used throughout our experiments in the appendix, as well as all hyperparameters, and exemplary SelfReflect computations broken down to the word-level.
methods to enable standardized benchmarking. Besides
REFERENCES
M. Abdin, J. Aneja. H. Behl, S. Bubeck, R. Eldan, S. Gunasekar, M. Harrison, R. J. Hewett, M. Javaheripi, P. Kauffmann, al. Phi-4 technical report. arXiv preprini arXiv:2412.08905, 2024.
L. Aichberger, K. Schweighofer, and S. Hochreiter. Rethinking uncertainty estimation in natural language generation. arXiv preprint arXiv:2412.15176, 2024.
J. M. Bernardo and A. F. Smith. Bayesian theory, volume 405. John Wiley & Sons, 2009.
T. M. Cover. Elements of information theory. John Wiley & Sons, 1999.
DeepSeek-AL, D. Guo, D. Yang. H. Zhang. J. Song. R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang. X. Bi, X. Zhang. X. Yu, Y. Wu, Z. F. Wu, Z. Gou, Z. Shao, Z. Li, Z. Gao, A. Liu, B. Xue, B. Wang. B. Wu, B. Feng, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan, D. Dai, D. Chen, D. Ji, E. Li, F, Lin, Dai, F. Luo, G. Hao, G. Chen, G. Li, H. Zhang, H. Bao, H. Xu, H. Wang, H. Ding, H. Xin, H. Gao,
H. Qu, H. Li, J. Guo, J. J. Wang, J. Chen, J. Yuan, J. Qiu, J. Li, J. Cai, J. Ni, J. Liang, J. Chen, K. Dong, K. Hu, K. Gao, K. Guan, K. Huang, K. Yu, L. Wang, L. Zhang, L. Zhao, L. Wang,
L. Zhang, L. Xu, L. Xia, M. Zhang. M. Zhang, M. Tang, M. Li, M. Wang, M. Li, N. Tian, P. Huang,
P. Zhang. Q. Wang, Q. Chen, Q Du. R. Ge, R. Zhang, R. Pan, R. Wang. R. Chen, R. L. Jin, R. Chen, S. Lu, S. Zhou, S. Chen, S. Ye, S. Wang, S. Yu, S. Zhou, S. Pan, S. S. Li, S. Zhou, S. Wu, Pei, T. Sun, T. Wang, W. Zeng, W. Zhao W. Liu, W. Liang, W. Gao, W. Yu, W. Zhang, W. L. Xiao, W. An, X. Liu, X. Wang, X. Chen, X. Nie, X. Cheng, X. Liu, X. Xie, X. Liu,
S. Ye, T. Yun,
X. Yang, X. Li, X. Su, X. Lin, X. Q. Li, X. Jin, X. Shen, X. Chen, X. Sun, X. Wang, X. Song, X. Zhou, X. Wang. X. Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Y. Zhang, Y. Xu, Y. Li, Y. Zhao, Y. Sun,
Y. Wang. Y. Yu, Y. Zhang. Y. Shi, Y. Xiong, Y. He, Y. Piao, Y. Wang, Y. Tan, Y. Ma, Y. Liu, Y. Guo,
Y. Ou, Y. Wang. Y. Gong, Zou, Y. He, Y. Xiong, Y. Luo, Y. You, Y. Liu, Y. Zhou, Y. X. Zhu, Y. Xu,
Y. Huang. Y. Li, Y. Zheng, Y. Zhu, Y. Ma, Y. Tang, Y. Zha, Y. Yan, Z. Z. Ren, Z. Ren, Z. Sha, Z. Fu,
Z. Xu, 7. Xie, 7. Zhang, 7. Hao, 7. Ma, 7. Yan, 7. Wu, 7. Gu, 7. Zhu, Liu, 7. Li, 7. Xie, 7. Song, Z. Pan, Z. Huang, Z. Xu, Z. Zhang. and Z. Zhang. Deepscek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. 2025. URL htttt:tt////rrt
org/ abs/2501 12948
K. Enevoldsen, I. Chung, Kerboua, M. Kardos, A. Mathur, D. Stap, J. Gala, W. Siblini, D. Krzemiński, G. I. Winata, S. Sturua, S. Utpala, M. Ciancone, M. Schaeffer, G. Sequeira, D. Misra, S. Dhakal, J. Rystrom, R. Solomatin, Omer Çağatan, Kundu, M. Bernstorff, S. Xiao, A. Sukhlecha, B. Pahwa, R. Poéwiata, K. K. GV, S. Ashraf, D. Auras, B. Plüster, J. P. Harries, L. Magne, Mohr, M. Hendriksen, D. Zhu, H. Bisselot-Boukhlef,, T. Aarsen, Kostkan, K. Woj- tasik, T. Lee, M. Suppa, C. Zhang, R. Rocca, M. Hamdy, A. Michail, J. Yang, M. Faysse, A. Va- tolin, N. Thakur, M. Dey, D. Vasani, P. Chitale, S. Tedeschi, N. Tai, A. Snegirev, M. Günther, M. Xia, W. Shi, X. H. Lů, Clive, G. Krishnakumar, A. Maksimova, S. Wehrli, M. Tikhonova, H. Panchal, A. Abramov, M. Ostendorff, Z. Liu, S. Clematide, I. J. Miranda, A. Fenogenova, G. Song, R. B. Safi, W.-D. Li, A. Borghini, Cassano, H. Su, J. Lin, H. Yen, L.. Hansen, S. Hooker, C. Xiao, V. Adlakha, O. Weller, S. Reddy, and N. Muennighoff. MMTEB: Massive multilingual text embedding benchmark. arXiv preprint arXiv:2502. 2025.
A. R. Fabbri, W. Kryšciński, B. McCann, C. Xiong, R. Socher, and D. Radev. Summeval: Re- evaluating summarization evaluation. Transactions of the Association for Computational Linguis- tics, 9:391 2021.
E. Fadeeva, R. Vashurin, A. Tsvigun, A. Vazhentsev, S. Petrakov, K. Fedyanin, D. Vasilev, E. Gon- charova, A. Panchenko, M. Panov, T. Baldwin, and A. Shelmanov. LM LMlpooygraph:: Uncertainty es- timation for language models. In Y. Feng and E. Lefever, editors, Proceedings of the 2023 Confer- ence on Empirical Methods in Natural Language Processing: System Demonstrations, Dec. 2023.
11

---

Published as a conference paper at ICLR 2026
E. Fadeeva, A. Rubashevskii, A. Shelmanov, S. Petrakov, H. Li, H. Mubarak, E. Tsymbalov, G. Kuzmin, A. Panchenko. T. Baldwin, P. Nakov, and M. Panov. Fact-checking the output of large language models via token-level uncertainty quantification. In L.-W. Ku, A. Martins, and V. Sriku- mar, editors, Findings of the Association for Computational Linguistics: ACI. 2024, Aug. 2024.
S. Farquhar, J. Kossen, L. Kuhn, and Y. Gal. Detecting hallucinations in large language models using semantic entropy. Nature, 630(8011))x65--66) 2024.
R. Flamary, N. Courty, A. Gramfort, M. Z. Alaya, A. Boisbunon, S. Chambon, L. Chapel, A. Corenflos, K. Fatras, N. Fournier, L. Gautheron, N. T. Gayraud, H. Janati, A. Rakotomamoniy, I. Redko,
A. Rolet, A. Schutz, V. Seguy, D. J. Sutherland, R. Tavenard, A. Tong, and T. Vayer. Pot: Python optimal transport. Journal of Machine Learning Research, 22(78):1-8, 2021. URL
http://3m (wm.mmmrroomm www.gror.p...e wwtgg/aaperss. stterr/vv2/// 20222220-222 450000151000 .htmll
M. Fomicheva, S. Sun, L. Yankovskaya, F. Blain, F. Guzmán, M. Fishel, N. Aletras, V. Chaudhary, and I. Specia. Unsupervised quality estimation for neural machine translation. Transactions of the Association for Computational Linguistics, 8:539-555, 2020.
Gemma Team, A. Kamath, J. Ferret, S. Pathak, N. Vieillard, R. Merhej, S. Perrin, T. Matejovicova, A. Ramé, M. Rivière, et al. Gemma 3 technical report. arXiv preprint arXiv:2503. 19786, 2025.
D. Hendryeks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt. Measuring massive multitask language understanding. Proceedings of the International Conference on Learning Representations (ICLR), 2021.
D. R. Hofstadter. Godel, Escher, Bach: An Eternal Golden Braid. Basic Books, Hassocks, England, 1979.
S. Jain, V. Keshava, S. M. Sathyendra, P. Fernandes, P. Liu, G. Neubig, and C. Zhou. Multi- dimensional evaluation of text summarization with in-context learning. arXiv preprint arXiv:2306.01200 2023.
A. Jiang, A. A. Chahine, A. Sablaymoless, A. Taenet, A. Boissonnet, A. Kothari, A. Héliou, A. Lo, A. Peronnin, A. Meunier, A. Roux, A. Faure, A. Paul, A. Darcet, A. Mensch, A. Herblin-Stoop, A. Garreau, Birky, A. Sooriyarachchi, B. Rozière, Conklin, B. Bouillon, B. de Beauregard, C. Rambaud, C. Feldman, C. de Freminville, C. Mauro, C.-K. Yeh, C. Bamford, C. Auguy,
C. Heintz, C. Dubois, D. Chaplot, D. L. Casas, D. Costa, E. Arcelin, B. Hanna, E. Metzger,
E. O. Autran, F. Lesage, G. Gourdel, G. Blanchet, G. D. Vidal, G. M. Lengyel, G. Bour, G. Lample,
G. Denis, Rajaona, H. Jaju, I. Mack, I. Mathew, J.-M. Delignon, J. Facchetti, J. Chudnovsky,
J. Studnia, J. Murke, K. Khandelwal, K. Chiu, K. Riera, L. Blier, L.. Suslian, L.. Deschaseaux,
L. Martin, L. Ternon, L. Saulnier, L. R. Lavaud, S. Yang, M. Jennings, M. Pellat, M. Torelli, M. Janiewicz, M. Felardos, M. Darrin, M. Hoff, M. Seznec, M. J. Kenyon, N. Derwiche, N. C. Zaragoza, N. Faurie, N. Moreau, N. Schuhl, N. Raghuraman, N. Muhs, O. de Garrigues, P. Rozé, P. Wang, P. Platen, P. Jacob, P. Buche, P. R. Muddireddy, P. Savas, P. Stock, P. Agrawal, R. de Peretti, R. Sauvestre, R. Sinthe, R. Soletskyi, S. Vaze, S. Subramanian, S. Garg, S. Ghosh, S. Regnier, S. Antoniak, T. L. Scao, T. Gervet, T. Schueller, T. Lavril, T. Wang, T. Lacroix, V. Nemychnikova, W. Shang, W. E. Sayed, and W. Marshall. Un ministral, des ministraux. 2024. URL httttttss//mistratt
ai hwtp://ww//eeeeeeeeeee wtw.wwswwmiiistreeeeee Putm. htt.://trrauuuttmmoooooooooo-- source=tldrai
M. Joshi, E. Choi, S. Weld, and L. Zettlemoyer: Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, Canada, July 2017. Association for Computational Linguisties.
Jung, X. Lu, L. Jiang, F. Brahman, P. West, P. W. Koh, and Y. Choi. Information distillation for reference-less summarization. In First Conference on Language Modeling. 2024. URL / net/ forum? id= Jx=JXxXJJJSui
S. Kadavath, T. Conerly, A. Askell, T. Henighan, D. Drain, E. Perez, N. Schiefer, Z. Hatfield- Dodds, N. DasSarma, E. Tran-Johnson, S. Johnston, S. El-Showk, A. Jones, N. Elhage, T. Hume, A. Chen, Y. Bai, S. Bowman, S. Fort, D. Ganguli, D. Hernandez, J. Jacobson, J. Kernion. S. Kravec, L. Lovitt, K. Ndousse, C. Olsson, S. Ringer, D. Amodei, T. Brown, J. Clark, N. Joseph, B. Mann,
12

---

Published as a conference paper at ICLR 2026
S. McCandlish, C. Olah, and J. Kaplan. Language models (mostly) know what they know. arXiv, 2022.
C. Kauf and A. Ivanova. A better way to do masked language model scoring. In A. Rogers, J. Boyd- Graber, and N. Okazaki, editors, Proceedings of the 61 Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), July 2023.
T. Kwiatkowski, J. Palomaki, O. Redfield, M. Collins, A. Parikh, C. Alberti, D. Epstein, I. Polosukhin, J. Devlin, K. Lee, K. Toutanova, I.. Jones, M. Kelcey, M.-W. Chang, A. M. Dai, J. Uszkoreit, Q. Le, and S. Petrov, Natural questions: benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7, 2019.
L. Lauritzen. Sufficiency, prediction and extreme models. Scandinavian Journal of Statistics, pages 138,134,7 1974.
Z. Li, X. Zhang, Y. Zhang, D. Long, P. Xie, and M. Zhang. Towards general text embeddings with multi stage contrastive learning. arXiv preprint arXiv:2308. 033881, 2023.
S. Lin, J. Hilton, and O. Evans. Teaching models to express their uncertainty in words. arXiv preprint arXiv:2205. 14334, 2022.
A. Malinin and M. Gales. Uncertainty estimation in autoregressive structured prediction. arXiv preprint mrXiv:2002.076500 2020.
Meta AL. LJama 3.1 model card. 2024a. URL https: hwwp://wwwwwaaaccoooooooossss
mode www.arrs-aao and -prampddddppp----- www.tttttssooooom..mmm..oooo.rtt ama3
Meta AI. Llama 3.3 model card. 2024b. URL
mode 1- cardss and-aand- pwoopt-ropp---oooo formats/llama3 http:://ww...aa.ccoom//////
Meta AI. Llama 4 model card. 2025. URL https:/ htt:://www... www. 11a.alaaaa...mmm cot.am/ccco///css
mode 1- and--crrd--ddd--ooooo prompt wwwpopttfoooattttttttep wow.atoottssssoo /llmma4///
G. Peyré, M. Cuturi, et al. Computational optimal transport: With applications to data science. Foundations and Trends@ in Machine Learning, 11(5-6)3355-60 607, 2019.
Qwen Team. Qwen3, April 2025a. URL https:t//////. ptttpp://qwennm... www.gwewwm.mmmmm..m htt.://www....ta.i.o.......aeeee
Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025b. URL
https: / http:///weemm..mmmm qithub
P. Sahoo, A. K. Singh, S. Saha, V. Jain, S. Mondal, and A. Chadha. A systematic survey of prompt engineering large language models: Techniques and applications. arXiv preprint arXiv:2402.07927, 2024.
J. Salazar, D. Liang, T. Q. Nguyen, and K. Kirchhoff. Masked language model scoring. In D. Jurafsky, Chai, N. Schluter, and J. Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, July 2020.
A. Santilli, A. Golinski, M. Kirchhof, F. Danieli, A. Blaas, M. Xiong, L. Zappella, and Williamson. Revisiting uncertainty quantification evaluation in language models: Spurious interactions with response length bias results. arXiv preprint (tt::2504.136777 2025.
Shin, Y. Lee, and Jung. Effective sentence scoring method using bert for speech recognition. In W. S. Lee and T. Suzuki, editors, Proceedings of The Eleventh Asian Conference on Machine Learning, volume 101 of Proceedings of Machine Learning Research, pages 1081-1093. PMLR, 17-19 Nov 2019. URL httpttt/pproceeeiiii ngs wwtp:/gw.................p.p..te ptress/vvcl/shati 1a.119.hhh. html
W. I.. Taylor. Cloze procedure: A new tool for measuring readability. Journalism quarterly, 30(4): 415-433, 1953.
O. Vasilyev, V. Dhamidharka, and J. Bohannon. Fill in the BLANC: Human-free quality estimation of document summaries. In S. Eger, Y. Gao, M. Peyrard, W. Zhao, and E. Hovy, editors, Proceedings of the First Workshop Evaluation and Comparison NLP Systems, pages 11-20. Online, Nov. 2020. Association for Computational Linguistics. doi: 10.18663/ https: //os://cc antho ogy. ogw.org//000..... An
URL
13

---

Published as a conference paper at ICLR 2026
A. Wang and K. Cho. BERT has mouth, and it must speak: BERT Markov random field language model. In A. Bosselut, A. Celikyilmaz. M. Ghazvininejad, S. Iyer. U. Khandelwal, H. Rashkin. and T. Wolf, editors, Proceedings of the Workshop on Methods for Optimizing and Evaluating Neural Language Generation, June 2019.
Z. Wang and C. Holmes. On subjective uncertainty quantification and calibration in natural language generation. arXiv preprint arXiv:2406.05213. 2024.
Wei, N. Karina, H. W. Chung, J. Jiao, S. Papay, A. Glaese, Schulman, and W. Fedus. Measuring short-form factuality in large language models. arXiv preprint arXiv:2411.04366 2024.
T. Xu, S. Wu, S. Diao, X. Liu, X. Wang, Y. Chen, and J. Gao. Sayself: Teaching Ilms express confidence with self-reflective rationales. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 5985-5998, 2024.
A. Yang. B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, C. Li, D. Liu, F. Huang, Wei, H. Lin, J. Yang. J. Tu, J. Zhang, J. Yang. J. Yang. J. Zhou, J. Lin, K. Dang, K. Lu, K. Bao, K. Yang, L. Yu, M. Li, M. Xue, Zhang. Q. Zhu, R. Men, R. Lin. T. Li, T. Xia, X. Ren, X. Ren, Y. Fan, Y. Su, Y. Zhang, Y. Wan, Y. Liu, Z. Cui, Z. Zhang, and Z. Qiu. Qwen2.5 technical report. arXiv preprint ar 24X1v:2412.15115, 2024a.
R. Yang, C. Zhang, Z. Zhang, X. Huang, S. Yang. N. Collier, D. Yu, and D. Yang. Logu: Long-form generation with uncertainty expressions. arXiv preprint arXiv:2410.1 243.::9990.,,30, 2024b.
7. Yang, P. Qi, S. Zhang, Y. Bengio, W. W. Cohen, R. Salakhutdinov, and C. D. Manning. HotpotQA: A dataset for diverse, explainable multi-hop question answering. In Conference on Empirical Methods in Natural Language Processing (EMNLP), 2018.
G. Yona, R. Aharoni, and M. Geva. Can large language models faithfully express their intrinsic uncertainty in words? arXiv preprint arXiv:2405. 16908, 2024.
Y. Zhang, H. Jin, D. Meng, J. Wang, and J. Tan. comprehensive survey process auto- matic text summarization with exploration of llm- based methods. arXiv preprint arXiv:2403.02901. 2024.
14

---

Published as a conference paper at ICLR 2026
APPENDIX CONTENTS
A
SelfReflect and predictive sufficiency: propositions and proofs
17
A.I
Setup, notations, and assumptions
17
A.2
Predictive sufficiency and equivalent characterizations
18
A.3
SelfReflect metric and equivalence predictive sufficiency
19
A.4
Modeling with LLM: From derivation to implementation
22
B
Convergence the SelfReflect metric
24
B.
Reducing both N and M
24
B.2
Reducing only M
24
C
Which LLM, judge to to generate SelfReflect logits
28
D
Example of SelfReflect scores per masked-out word
29
E
Implementation details
31
SelfReflect score
31
SR sampling-fre score
31
SR-PMI score
E
SR-P(True) score
31
Embedding score
E,6
6
Summarization score
32
E.7
LM Judge score
33
E.8
Optimal Transport score
34
9
InfoSumm score)..
35
10
Licensing information
F
Rating good and bad summaries written by humans
36
G
How well does SelfReflect distinguish good from bad summaries
40
MMLUU tests of the SelfReflect metric per dataset
40
User study details
41
J
Automatic summary generation
42
Experimental details
42
J.2
Prompts used
42
0.3
Results per dataset
43
K
Ex Experiment details of CoT deep dive
46
Results per dataset
46
Prompts used to classify certainty vs. uncertainty
46
L
Finetuning to generate self-reflective summaries
51
15

---

Published as a conference paper at ICLR 2026
M
Results on Retrieval-auggmentdd Generation
51
Behavior of Sample Summarize
52
Results with self-critique
53
16

---

Published as a conference paper at ICLR 2026
A SELFREFLECT AND PREDICTIVE SUFFICIENCY: PROPOSITIONS AND PROOFS
In this appendix, we provide details of the propositions from the main text and their proofs. We begin with the definition of predictive sufficiency and provide proof of its two equivalent characterizations in the context of the SelfReflect metric. We then prove an equivalence between solving the masked- token prediction task of the SelfReflect metric and the desired predictive sufficiency of the summary. providing theoretical foundation for the design of the SelfReflect metric.
QQ
S
B
Figure 5: The graphical model for the setting of SelfReflect metric. The figure reproduced from Figure of the main text for the sake of better readability of the formalization that follows.
A.1 SETUP, NOTATIONS, AND ASSUMPTIONS
Recall that prompting given LLM with question Q puts it in state Qq. from which we sample N answers ((1:N), A summarization mechanism function vi generates the summary of these answers as (L::N)) For developing the SelfReflect metric, we generate another sample B from the same state 9g and require an ideal summary S to capture all the information about B that captured by the samples (1:N), Now, we formalize this setup of the SelfReflect metric by setting the notation, listing the assumptions of the setup, and providing their justifications.
SETUP AND NOTATION
1. Firstly. Figure 2 shows the graphical model of this setup, which we also reproduce here in Figure 5 for better readability. In this graphical model, observed variables are shaded gray, which includes the sampled answers (1:N), their summary 5, and a subsequent answer B, whereas unobserved/latent variables unshaded, which includes the LLM state Qq.
2. We will upper-case non-boldface letters (like B S) to represent random vari- ables/vectors and the corresponding lower-case non-boldface letters (like or s) to represent particular samples from their underlying distributions.
3. For a random variable Y. the sampling of particular value will be denoted as y Y or y E supp (Y), where supp (Y) represents the support the random variable Y.
4. Let denote a finite vocabulary of words (or tokens), which is used to generate questions, the corresponding answers, and their summaries.
5. Let Q denote the random variable for question.
6. Prompting the given LLM with this question Q assumed to put it state, which is represented with the random variable θg. From this state, which are then used to define the SelfReflect metric.
sample multiple answers,
7. The random variables A (1:N) = (A AA())) are used to denote the N answers sampled from the LLM in state Oq. These samples may be sampled in an i.i.d. manner but do not necessitate this. In fact, can sample each answer A(n) conditioned on all previous samples A (1=x-1) well. We allow for this generality because throughout our derivation, we will always consider these answers jointly as A (1:N).
8. A summarization mechanism inputs the sampled answers and generates their summary S. 9. Suppose denote subsequent sample from the LLM in the same state θq. For the SelfRefiect metric, we require an idea summary of sampled answers A (1:N) to capture all information about this subsequent answer B.
17

---

Published as a conference paper at ICLR 2026
ASSUMPTIONS
1. The support of question Q is assumed to be the set of all finite-length sentences generated from V. which we denote by X.
2. The support of each A is also assumed to be X, the set of all finite-length sentences generated from vocabulary V.
3. The summarization mechanism that inputs the sampled answers ( and generates their summary 5 is assumed to be a function vb Formally, & : - X' inputs any N sampled answers A (1-N) from the L.M and generates their summary 5 as S == ((((L))) Note that the support of the summary S, will be subset of the set of all finite-length sentences, i.c., supp (S)= C X. This condition models our setup sufficiently well, where we have a candidate summary S per set of answers (1:N), However, we acknowledge that it is a restrictive condition in that it doesn't 't allow for modeling conditional distribution over all summaries given the answers. Generalizing our SelfReflect metric for this case or proving its generality in this case is an interesting direction for future work.
4. We define the support of the subsequent new answer B to be the set X2 == V of all possible sentences from the vocabulary V that are of length L. Despite being slightly restrictive, this assumption is not unreasonable; all LLMs have maximum context length, which can be viewed as an upper limit on the length of the answer B. Also, sentences with smaller lengths are usually padded to achieve the maximum context length.
5. Throughout our derivations, we will assume all required marginal and conditional distribu- tions to be strictly positive. This assumption is reasonable for our setting because in prac- tice, we would be implementing corresponding distributions using the given LLM. For in- stance, p (W) would represent the probability of sentence W under the given LLM. Further, p (Y Z) would represent the probability of sentence Y when the LLM is prompted with the context Z. Since the LLMs generate distribution over the entire vocabulary V, all the condi- tional distributions will have strictly positive values, albeit extremely small in certain cases.
A.2 PREDICTIVE SUFFICIENCY AND EQUIVALENT CHARACTERIZATTION
Now, having set the notations and assumptions, we define the notion of sufficiency and connect with the definition of an ideal summary.
Definition A.1 (Bayesian and Predictive Sufficiency (Bernardo and Smith 2009)). Consider distribution parumeterized in terms of a parameter é) Let (1:MM) denote M (i.id.) samples from this distribution. A statistic (function) T(X X1:M is called a Bayesian sufficient statistic of samples (11:M) for & if and only if we have: p (Ø X (1::)) = =((1: = p (ó (Xx()))) = (1:())))) On the other hand, it is called predictive sufficient statistic of samples X(I:M) if and only if have: p (X- x ((:M) = (1EM)
p (X = x (( = f(x(1.M)) for any subsequent sample X (with concrete value x: E supp (X) from the same distribution.
Note that our Definition 3.1 of an ideal summary is closely related to predictive sufficiency as defined in Definition A. However, it turns out that Bayesian and predictive sufficiency notions are not exactly equivalent. In light of this, our reason for defining an ideal summary be predictive sufficient, rather than Bayesian sufficient, is as follows. An LLM trained on a huge corpus of data contains information about a wide array of aspects. However, through the summary, we are interested in capturing only those aspects of the state Oo of the L.LM that are related to answering the given question Q For this, requiring the summary to be predictive sufficient serves the purpose precisely,
Now, in the context of the Definition A.I of predictive sufficiency, Definition 3.1 of ideal summary, and the graphical model of Figure 5 we prove Proposition 3.I] which asserts the equivalence in the information theoretic and conditional distribution based formulations of the ideal summary. We begin by proving a lemma about the graphical model of Figure S
Lemma A.1 (Conditioning on (1:N) and S). Under the graphical model given in Figure have:
p B A(L:N),
(B) Aa:N)
18

---

Published as a conference paper at ICLR 2026
Proof. Consider the following manipulations:
p B A(:N)) 5)) = I [a0 p B, 02 = 1
-12) S do P (00 = 0, B, A(I:)), B,B,A
p (A(1:N),
(3) ! /a0 p p00 = 0) p (B 10 = 0) p (A (1) 100= = 0) p (5) A0:m)
p (A(1::)) D (S) A(:N))
(4) / do
p p0Q = 0) p (B 1 = 0) .p (A(I.)) (0q=0
p (A(L:N))
(5) / do P (0q = =0, B, A(ND)
p (A(I:N))
(6) ! dB p B, QQ 0 1a00)) -(7) p (B) A(:N)
Here, steps (2), (5). (6) follow from chain rule. Step (4) follows by cancellation of the common
terms. Steps (1), (7) follows from integrating out variable Og. Step (3) follows from the graphical
model of Figure 5 Finally, an analogous derivation would follow by replacing integration with
summation in the case of Oo being a discrete variable.
Now, we prove Proposition 3.1 establishing the equivalence of the information theoretic and condi- tional distribution based formulations of the desired predictive sufficiency.
Theorem A.1 (Connection of SelfReflect to Predictive Sufficiency). Consider the graphical model given in Figure 5 Under this graphical model, for ideal summary S of answers A A(:N),
I IfAUN B) = [S: = (B) Al:M) = p (B (S)
Proof. Consider following steps:
log p (AAI:N), .B)
=gs,n log (S, B)
I { (umm B) B}= = I I(S B) (1)
(2) E BAUNN
p (A(::)) p (B)
p (S) p (B)
(A(1:N), B) p (S)
p (B | A(::))
- + BBBAUN, log
(2) B,AGN,,
log p S)
0
p (S, -p (A(I:N))
log p (B) A(L:N),))
- I I {A:A(IUN 883=0
E BBA.UN)
p (BB] S)
(5) p B, (1:N)
1s) p (B SS).p (A(:N) (s)
(6) p B A(I:N),
.s) p (B S) (7))
= (7) p B ((a)))) SS)
Here, step (1) follows from the definition of mutual information, steps (2) and (6) from chain rule, steps (3) and (7) from Lemma A. step (4) from the definition of conditional mutual information, and step (5) from the equality condition of conditional mutual information. For details on mutual information and conditional mutual information, we refer the reader to Cover (1999).
A.3 SELFREFLECT METRIC AND EQUIVALENCE TO PREDICTIVE SUFFICIENCY
Now, we demonstrate that the masked-token prediction task of SelfReflect is equivalent to the above notion of predictive sufficiency. For the SelfReflect metric, we consider the random variable B for a new subsequent sample from the LLM in state eg and dissect it in terms of its words. In particular, have: B (Bi,-.. Bz), where L is length of the sentence B (which, as saw, could be chosen to be the maximum context length for the LLM). Here, B, represents the random variable for the t-th word of the sentence B for each value of i E {I,... L.J. For each we use
19

---

Published as a conference paper at ICLR 2026
the shorthand notation B_, ; to represent the variable for all the words in the sentence B except for B,, ie., B_; := (B1,... ....... BB,-1, Bi+1. BL) = -BB)) Note that Bt, which represents the E-th word of sentence B, is not to be confused with A(A), which represents the k-th sampled answer from the LLM. For each B,, its support is going to be the vocabulary V and the supports of B_; and B are V and V = X₁, respectively. With this setup, we can prove Proposition 3.2 which asserts that under assumptions from subsection A.1 SelfReflect metric provides an equivalent formulation of the desired predictive sufficiency of ideal summary S. This is done as follows.
Theorem A.2 (SelfReflect Metric and Predictive Sufficiency). Suppose all involved conditionals are modeled via the given LLM and hence, are strictly positive. Then, we have:
p B ( p (B (S) = for all masking indices
(Bs) A B-.) (Br |S, B-i) (7)
Proof.
(=) Suppose we are given that p (B) A0:N) = (B) S). Consider the following steps:
p B A(LN)) = (B S) - p (BI... ... Br. | = (B)....,,,,
Σ Enevp B2,..., B, =
(BI.... .., ,B = P(B... B, = by ...... Bg SS)
1 (1) p (B-i 14 (B)
(8)
Here, step (1) follows from integrating out variable B; Combining this result with the premise gives:
p B ((NN))) = (BIIB))) (0-=|Au0))) = =p (B
p B A ( ()
p (B=, A(IN)) p (B_ (S) -1) p B: (BLM)BB.) =p (B; S, B-;) (9)
Here, step (1) follows because B is formed of the i-th word B; and the rest of the words B-i. Since we can carry out these steps for any index i, we prove the forward direction of the theorem.
(=) Now, to prove the converse, suppose we are given that for all masking indices i, we have: p (Br) ((=:N), B):) p [B; S, B-.) and we have to prove that (B A(1:N) = (B ()) Since this is an equality of the random variables, we prove the equality of random variables by proving it for any and all choices of the samples of those random variables. Note that this works because of the assumption of summary mechanism S being function of A (I:N), which allows us to use the given condition as well as prove the desired result by assuming particular instantiations of (1::N) = (l:N) and using the corresponding summary S 5 == ((a(l:N)) Pick any instantiations of sampled answers from their support as (1:N) ((1:N), Since the summary mechanism is function, it gives us concrete sample (a(((t))) E X. Now, suppose want to prove the desired result for any particular given sample ~ B with b ;= (b,,... by.) E VL. Consider fixed sentence by E VL with b² == (bj,.. bj.)) Now, we define a sequence of sentences as follows:
x (0) (b1,b2, bn.) PEVE
x (1) = (bi, ... b1) E V'.
a (2) = (bi, Ug, b1) E VL
-b)==b' (4) = (bi, (.))) 6'
(10)
Intuitively, we create a sequence of sentences where each subsequent sentence z(0) differs from the previous sentence and the next sentence in exactly one word and as we go from sentence (0)
to ,.(L), we change the given sentence b to the fixed sentence b². Now, we consider the following
20

---

Published as a conference paper at ICLR 2026
manipulations for p (B bl A(lMM) all:M)):
B = b) b|A(1 = =c(1)0)
[ = =x)) [A = =((l-N))
p p x((1 |A(2:N) (1:00)
=(1) B = ,00) A11iM) = a(um)) II 1-1 p (B= = x(1) | |A1:N)) = a(l:0))
=2) L. p (B = (f(11) A(I-N) =
p B = bi | = =a(iN))
(11)
p [B- = x({ AAI:N) = =ali.))
In an exactly analogous way, we get following manipulations for p (B=b) = S= =s):
p ( = (BB=bSS-1 = p 0( (B = (0) 1 =
L pB = x(() | S =
=(1) p B = x0 |s ss)) .II [=1 PB - x(() | S ==
=2) L (B = x (1-1)
S = s)
) p (B= b' 1 S= 8)
(12)
p B = x (8)
5 8)
Note that in both Equation 11 and Equation| 12 above, step (1) follows from multiplying and dividing by the same terms and step (2) follows from rearranging the terms and recognizing x(() b by definition. Now, we consider the f-th term from the Equation I1 and simplify it as follows:
PB x(t-1) | A (22:N)
(B = z(1) AAIN) = =l(iM))
-0) p (B
Bi-1 69_1, By bb, BB+1 b1t1.-.. ,B, = 1y | (11:N))
p (B, bij B-- b/ Bt bú, Be+1 bet1,'..
Br. br.
All:N) ((I:N))
(2) p (B_ = (bg, ... b ... ,,,b)) | A = a(l:N))
p (B_r (bj, ,b6_1. be41...... ,b,)) A(I:N) a(l:N))
x BB = be/ (1:N) = (1:N), B_ = (6), ... bī-1 be+1, ,b))
p (Be = b7 A(I.N) = a(I:N) B_t = (bf. .. b7 11 bet1, bl))
=(3) P (B. = be [I:N) = ((l:N), B (b), ... bū_1 1. bet1 bL))
(13)
(B:- = bū A(:N) = a(I:N), B_ (69 ... 63 bet1, bl))
Again, in exactly analogous way, simplify the e-th th terms of Equation 12 as follows:
p ( x(0 1 ==
s))
_(1) p PBB = bj. B = b B₁ = Be+1 = bi41, ... BB = by S = 8)
p PBB = bj. ... B b Br bj. Br41 bet1. B₁, = bj,
S 8)
= _2) P (B_+ = (bi, ... 0i_ 1. bet1, ... .b)) 15 ==
p (B_g e = (bf, (bf, ... 0 1. bet1, '. .b.) |S ==
x p B, = b, S = 8, B_t = (b)."" ... .b bett, ... b1))
p BB = b; S = s, B_r = (bi ... 6b be+1 ... bz)
(3) P (Be = by S = s, A_ = (bi. b7 1. bet1, bz))
(14)
p (B. = 67 S = s, B_ = (b),-.. .b bet1 b1.)
In both these simplifications, step (1) follows from the definition of the sentences (ff-1), (0), step (2) follows from chain rule, and step (3) follows from canceling the common terms. However, given
21

---

Published as a conference paper at ICLR 2026
equality (B, , = p pB, S, B-.) for all masking locations implies that for all E:
p BB = be AA = a (IN) B_t = (bi ... bi-1 bet1. ...
.bL)) ,
i = (bj 6 bo+1. . bb.)) and =p (Br- = by S = 8, B
(15)
BB = bi | A (1:N = =d (lN,) B_+ = (6) by be+1, .b.))
=P P (Be- = by S = 8, B_ = (bi, ... b 1. be+1, bL)) p (Bt by 1 a (1:N), B_ (bj, b/ bet1 ... bL))
(16)
(Br by A(IN)) = a(I.N), B_ = (0g,
by b1t1; br)
p (Be l9 S = 1 = (61, 6/ 1, be+1, bb))
(Be- bū S = B t (bj
.b be+1, ,,,
P B (f-1 |ALLN)= (l:NN)
p (B (!(11) =
for all l E f1, ... (17)
(B = x(0 A(I:N) = a(I.N))
p p x00)
|
Combining this with Equation
11 and Equation
12 we get interesting result:
p B b A(IN) (1:N))
(B = b
(I p( pB=xlee 11 11q(1:)=a (1:))
P (B =² A (1:N)
[B=r(C]AA (1(
a p( p(B=l(t S=s=s)
.p(B = bO 1 sS=s)
p( (B=
(1) p (B = 6" A (1:N) = a (2:N)
(18)
p p b' s)
Here, step (1) follows from canceling equal terms in both the numerator and the denominator. What
Equation 18 implies is that given A (1:N), thereby giving S = ((a(1)))) the ratio for any and all values of e v thereby making it constant ::= c(a(2:N)) (a constant that dependents on ((I:N)). Now, we can integrate out B and obtain the value of this constant as follows:
p (B=bb4 (i
equals the ratio p "B
p p[B=b
p(B=6-18=s)
For all b E VL P (B = 6 A(IN) = (11:N))
c(a(1:N))
( S=
-1 Enevn p P( B = bl (1:N)
= = ( c(all:N)) (BB=bl =)
=
= =((((
E Ebeva P (B (B= =b 5 8) =c(a(i:)). = o(al1:N)
(19)
This proves that in fact c(a (IN))
which gives that for all b B, we have: p (B- = 6 A (1:N) (1:N)) p (B = b s). Since this result holds for all b B, we can write the corresponding result with the underlying random variable as: p (B A(:N)) = (1:N3) =
(B S = s). However, since this result holds for any sample choice of A = a (I:N) (and corre- sponding S = s = (I:())))) we get the desired results involving all underlying random variables: p (B) A = p (B S). This proves the reverse direction of the equivalence.
A.4 MODELING WITH LLM: FROM DERIVATION TO IMPLEMENTATION
Now, having proved the equivalence of the basis of the SelfReflect metric and the desired predictive sufficiency of summary, we show the connection with the exact definition of the SelfReflect metric. Suppose we are given with a question Q = E X, which is shown to an LLM labeled LLM,. This puts LLM, in a state Og = 0,- from which we sample answers A (1:N) = (I:N), and a subsequent sample B- = b E V Now, to calculate the SelfReflect metric, the core idea is that conditional distributions of the form p (Y) Z) involved in the theoretical considerations above modeled by
22

---

one by one to obtaín the masked word Published as a conference paper at ICLR 2026
prompting the judge LLM,, with context Z and checking the probability of In our implementation, this LLM, will semperature-scatt with temperature = as mentioned in the main text in order to flatten its distribution and make it consider more synonyms. Then, build the prompt of LLM,, by including the question Q q and either the samples A (1:NN) or their summary S = s == (ball))) along with a description t of the masked-tokee -ooken prediction task to tell the LLM,, judge what needs to do. We then mask each word B Bmn = bm E V and the rest of the sentence B-m = b m E V Then, we model the required conditional distributions that appear in the derivation using the LLM,, judge as follows:
p Bm =bm |A(IN) (1:N), B_m = -b-m)
= PLLM, BB bm Q = q, (1:N) = (1:N), ,, B-m b-m). and
(20)
p (Bm- = bm 1S 8, B_ = b bmm)
PrIM, (Bm = bm Q S = t, B bmm)
This modeling along with Theorem A.2 demonstrates the efficacy of SelfReflect metric: Corollary A.1 (Efficacy of SelfReflect Metric). For any question Q. for all masking indices m,
(PLM, (Bm A. B.Bmm)) .PLLM (Bm S, (.ttBBmm) B-m)) =
(1) (1) PIIM, Bm Q,A(1.), ,t, B_Bm) = -PIM, (Bm Q, B-m)
(2) Bm A BBm) = (p (S)Bm)
(21)
(3) (BA(( = pB(B S)
(4) I { A(LN)) B) = I (S: B)
Proof. Step (4) follows from Theorem A.1 step (3) follows from Theorem A.2
step (2) follows
from modeling in Equation 20] and step (1)
follows from the fact that the W (1 (1-Wasserstein)
distance between two distributions is 0 if and only if the distributions are identical.
DISCUSSION
We conclude this section by discussing two important points about our derivation.
1. Firstly, LLMs are known to behave significantly better with careful design of prompts (Sa lho et al. 2024). Thus, in modeling of Equation 20 may try optimize the prompt- ing template and the task description in order to further obtain sharper versions of the SelfReflect metric. In this aspect, note that our derivation does not provide a mechanism for optimizing for the prompt template or task description L. In fact, irrespective of this detail, the derivation holds true.
2. Secondly, state the assumptions required for the derivation, stated in Appendix I. are needed for establishing the connection of SelfReflect metric with the notion of predic- tive sufficiency. However, these are not needed for defining, implementing, or using the SelfReflect metric. Users may find our SelfReflect metric useful even in cases where one or more of the assumptions are loosened. Also, further generalizing the SelfReflect metric in cases where the assumptions are loosened proving that the current formulation holds in those scenarios remains an interesting direction for future theoretical work.
23

---

Published as a conference paper at ICLR 2026
B CONVERGENCE OF THE SELFREFLECT METRIC
B.1 REDUCING BOTH N AND M
In the main paper, we evaluate SelfReflect on 1000 questions per dataset with N MM 50 conditioning and masked-out answers. This is based on a convergence analysis that we present in this section. We use Qwen 2.5 72B Instruct and Natural Questions as an example and calculate the average SelfReflect score across an increasing number of questions and conditioning and masked-out answers Figs. 6 to 10 The question is how many questions are needed to arrive at a stable average score.
It can be seen in Fig. 6 that at N = M = 50, the SelfReflect score converges at 1000 questions, our setup for the paper. One can of course reduce N and M, which will roughly linearly reduce the runtime required to compute the score. However, when for example reducing to N M = 20 questions in Fig. 7 convergence to the final value sets in only at about 2500 questions, which linearly increases the runtime, so that the runtime advantage vanishes. If one allows the score to be bit less converged, for example in development rather than in reporting test results, we suggest to use N M 10 and 500 questions. This reduces the runtime calculate SelfReflect to minutes node with 8 A 100 GPUs, compared to the 67 minutes of N = M = 50 and 1000 questions.
The only real outlier these trends is N M 1. Here, it is especially important that == i.e., in the context of the answer distribution prompt, there is only a single response. In this case, the ideal summary is actually to return exactly this response rather than a summary of the distribution. Hence, in Fig. 10 Greedy obtains better SelfReflect score than Sample & Summarize. This underlines the importance of why SelfReflect uses multiple samples from the answer distribution in the context.
B.2 REDUCING ONLY M
We can study this effect further by reducing only the number of answers that compute masked-out tasks over, M, and keeping the number of reference distribution answers in the LM judge context constant at N = 50. The results, and comparison all baselines, is shown in Table 6 that, similar as in the previous section, M can be reduced down to M 5 without losing much of the ability detect fine-grained quality differences. At M E (1,2), performance degrades slightly on good vs almost-good and percentage vs or-concatenated. But it remains above the baselines and unlike in the previous experiment, does not degrade verbalized vs only majority answer. This confirms that the collapse in the previous experiment was due to lowering the number of answers in the reference distribution to N in which case the task becomes top-1 matching task rather than a distribution matching task.
We observe
Table 6: We lower the compute spent to calculate the SelfReflect score by reducing the number of masked-out task answers M, keeping the number of reference distribution answers constant. Mean + 95% confidence interval. It can be seen that SelfReflect's 's performance stays roughly untouched from N 50 down to N 5, and stays above baselines even for E [1,2]. Qwen 2.5 7B Instruct distributions questions from the Natural Questions dataset.
50
Good summaries bad summaries
Good vs almost-good
Detailed truncated
Verhalized uncertainty only majority answer
Verbalized or-concatenated
Percentage vs or-coneatenated
Metrie
Summarization
97.40%
38.70% 28.70%
53.55%
11.57% as.nots 19.37% us
57.02%
65 29. a
L.M Judge
98. 38.
27.32% mi
92.929 1o0k
34.55%
05.088 19%
Opt. Transport
50.166% 4x76
60.78% a wore
39.69% ner
48, 89% 27.00m
52.88% (0f5:...00 52.88%=...oo
69. 1 10.1119-a 10..-...0
Fimhediing
96. 20
65.49% 11
65 65.65%
10, (0.99%=21444
43.98%
36 16.65%-44
SelfRefleci M -1
99. 00
94.23% a aare
96. 13% oct
84.30% c000
71.07% oers
78.5 78.515:44.00 2075
SeifReflect M
2
99.600 19.60%
96. 12 ank
98. 06%
90.91 19.1%
77. 6..66% 43%
73. 33.55% 49%
SelfReflect
99. 10,770S-gaa
97.90% mite
98.
71.90% 01%
90.99% wi5
SelfReflect M 10
99. 99.80%
98. 2.5 sin
98. 066% 17%
92, 22.366%
76.03% ents
82. 64.44 75%
SelfReflect M
99.80%
98.64%
98. 08.006%
95.04% 23.5%5a..00
71.07%
84. 3..3099-a
SelfReflect 99.90%
98.74%%
98.06%
25.04%=21.m7 1..m%
74.38% m.%
83.47%
24

---

Published as a conference paper at ICLR 2026
0.115
0.110
Average SelfReflect Score
0.105
0.100
CoT
0.095
Basic
0.090
Greedy
0.085
Sample Summarize 50)
1000
2000
3000
4000
5000
Number of Questions/Answer Distributions Averaged Over
Figure 6: Convergence of the SelfReflect score with N .M 50 and increasing number of queries we evaluate on. Answer Distributions of Qwen 2.5 72B Instruct on Natural Questions.
0.100
CoT
Average SelfReflect Score
0.095
Basic
0.090
Greedy
0.085
Sample Summarize (n 50)
0.080
1000
2000
3000
4000
5000
Number of Questions/Answer Distributions Averaged Over
Figure 7: Convergence of the SelfReflect score with N M 20 and increasing number of queries we evaluate on. Answer Distributions of Qwen 2.5 72B Instruct on Natural Questions.
25

---

Published as a conference paper at ICLR 2026
0.0975
0.0950
www.
CoT
Average SelfReflect Score
0.0925
Basic
0.0900
0.0875
0.0850
Greedy
0.0825
0.0800
Sample Summarize (n 50)
0.0775
1000
2000
3000
4000
5000
Number of Questions/Answer Distributions Averaged Over
Figure 8: Convergence of the SelfReflect with N MM 10 and increasing number of queries we evaluate on. Answer Distributions of Qwen 2.5 72B Instruct on Natural Questions.
0.100
0.095
Average SelfReflect Score
W1
CoT
0.090
Basic
0.085
0.080
Greedy
Sample & Summarize (n 50)
0.075
1000
2000
3000
4000
5000
Number of Questions/Answer Distributions Averaged Over
Figure 9: Convergence of the SelfReflect score with N M and increasing number of queries we evaluate on. Answer Distributions of Qwen 2.5 72B Instruct on Natural Questions.
26

---

Published as a conference paper at ICLR 2026
0.095
0.090
Average SelfReflect Score
MM
CoT
0.085
0.080
Basic
Sample Summarize (n 50)
0.075
Greedy
0.070
1000 Number of Questions/Answer Distributions Averaged Over
2000
3000
4000
5000
Figure 10: Convergence of the SelfReflect score with N M 1 and increasing number of queries we evaluate on. Answer Distributions of Qwen 2.5 72B Instruct on Natural Questions.
27

---

Published as a conference paper at ICLR 2026
C
WHICH ,LM, JUDGE TO USE TO GENERATE SELFREFLECT LOGITS
Table 7: To find out which L.M judge produces the best logits, we test how often SelfReflect correctly distinguishes a good (top) from a bad (bottom) summarywith different possible judges LLM,, that calculate the SelfReflect metric, across different LLM's LLMg whose answer distributions are being summarized. Automatically generated summaries on Natural Questions, following Table [ Results for Phi 4 14B as a judge for Llama 3.1 8B Instruct are pending and will be added.
sumasticss bad summarics
Gandd shound good
Deailed truncated
Verbalized uncertainyy
Vistalized or-concenennted
Percentage Vs os-omiiatioaaldd KT 33.885
L.M,,
LLM,
only unjurity anower
Ilams 3.1 Instruct
Lians 3.1 88 Instruet
99, 75%
96, 135
94 U2
97. 39
80.00 B078-- are
Phi 4 14B
Liana 3.1 8B Instrce
99,75 75.5%
97. 77.50%
1000000
96, 10.300--a U7.52% 179
S1, 855 amn
66, 67.67%
Qwen22 7B Instruct
Liana 3.1 8B Instrcct
99, 70%
94, 10%
1000000
47. 95.5 win
80 80.99%
Ilams 3.1 5B Instrutt
4li 148
87.7%
94, 13.
94 92
99. 99.13
X7 8555 sen
86 119%
Phi 14B Qwen22 5 7B Instruct
Phi 14B
100,000
94, 25.25
14. 44.4
48, 68.15%--11.m
59, 26.5
59, 26%
Phi 14B
99, 70%
93.10%
98.71% a
45.04' i
59. 50.5006 cn mork
75.21 aint
Llama 3.1 5B Instrct
Owen2.5 7B Instuct
100,000% eouass
95. 73,7258- .0%
US. 15.76/8- a
U5.65% c555-mm-7
NO 80.87% own
R5. 26.32266. am
Phi 4 14B Qwen225 7B Instruct
Owen2.5 7B Instuct
99, 25% aoll
96.75%
98.59% vpt
M4.44% 10.
70.575
77.76%. 11.0/
Owen2.5 7B Instrct
99,80%
94.20%
98.00' and
95.08% 1..r
74.38% mark
83.47% 1a.22
Tlams 3.1 SB Instruct
Qwen2.5 728 Instract
95 87 sox
15.17 s0%
97 46% win
99. 1991 not
86 99 10%
76 26/6
Phi 4 14B
Owen2.5 72B Instract
98. 75% Lines
97.50%
98.59% 12
96.30% is.
72,2246 a2.00
55,50%%
Qwen2:5 7B Instract
Qwen2.5 72B Instract
99,80%
94.90%
99.35% L1a/d!
99.17%
75.21%..v.oo av.
66. 00.9555.. kuantt
mandatory component to calculate the SelfReflect metric is a judge LLM,, that predicts which masked-out words are possible, given either summary or a concatenation of samples. This judge needs to be able to "understand" both the details of the answer and the probabilistic aspect of this task, all the while not overwriting its context information with its own world knowledge when making the prediction. The choice of the judge can thus be seen as a hyperparameter to be optimized to produce SelfReflect scores that are discriminative as possible between good and bad and almost- good summaries. We test four different judges in this section, Llama 3.1 8B Instruct, Phi 14B, Qwen 2.5 7B Instruct (which we ultimately use in the paper), and Qwen 2.5 72B Instruct. We generate answer distributions on Natural Questions for different LLM,, (Llama 3.1 8B Instruct, Phi 4 14B, and Qwen 2.5 7B Instruct), then use Gemini 2.0 to generate summaries like in Section 4.1 and calculate how often SelfReflect correctly tells apart good from bad (or almost-good) summaries.
Table 7 shows that Sel/Reflect is very robust the choice of the judge LLM: All judges can tell apart good from bad summaries in almost all cases. In particular, there is also no indication of a "home- bias", i.e., that a judge would perform better in judging answer distributions that it sampled itself. This, along with the fact that especially bad summaries, which explicitly introduce statements that are wrong and go against the judge's world knowledge, are almost always judged as worse than good summaries, shows that there is no world-knowlede. leakage. We attribute this to L.MSs' abilities to predict from their context, and to the fact that SelfReflect runs its prediction both conditional on the summary and conditional on the answer distribution, that should there be any world knowledge leakage, it would likely be equal and removed.
To make the choice of which LLM judge use, we pay particular attention the last three columns of Table Comparing a verbalized or percentage uncertainty answer to an or-concatenated answer is among the most subtle challenges and tests whether the judge correctly infers the relative probabilities in both the answer distributions and the summaries, even when they are not explicit. Here we see that the Qwen family sets itself slightly off Phi and Llama 1.1. Within the Qwen family, the 7B model is within the confidence interval of the 72B model (with a mean result better for percentage vs or- concatenated, and worse for the other two), so we use it in the main paper due its lower inference cost. We note that we also tried using Qwen 2.5 0.5B Instruct judge, however, this small model was not able to tell apart good from bad summaries. Finally, we note that there exists a research opportunity in developing an LLM judge specialized to perform the SelfReflect judging, either to compress the 7B model into smaller and faster one, or to improve the last bits of performance on challenging cases. However, increase the complexity of our method and add a dependency on a particular model (-checkpoint), which is likely to be outdated soon in the fast- dast-moving field of LLMs.
decide against this in this paper, since specialized model would
28

---

Published as a conference paper at ICLR 2026
D EXAMPLE OF SELFREFLECT SCORES PER MASKED-OUT WORD
To deepen the understanding of how the SelfReflect score judges summaries, we provide a worked example. We break down the SelfReflect score to the penalty it gives to each masked-out word. To simplify this educational example, we use only N = M = samples and make the answers in the conditioning of the prompt equal to the masked-out test answers.
The question posed to the LLM is "Who received the first Nobel Prize in physics?". As can be seen below, the LLM's answer distribution includes Wilhelm Conrad Röntgen as most likely answer, as well as Hendrik Antoon Lorentz and Pieter Zeeman or Henri Becquerel as additional possibilities, and details on their work. Let us now first look at how SelfReflect judges relatively bad summary of this distribution which just returns the greedy answer "Wilhelm Conrad Rönigen received the first Nobel Prize in Physics.". Overall, SelfReflect assigns this bad summary a distance of 0.102 (or taken x 1000 like in Table 4 102). This score is due to SelfReflect detecting that Hendrik Antoon Lorentz and Pieter Zeeman or Henri Becquerel are not predictable from the summary, and neither the details can in the per- word penalties below (darker red higher penalty).
of the works,
Conred
Rontpen
womiied
Nobel
Prize
Physics
The
First
hobel
Prize
in
Physic5
owar d
to
Wildhe
Corrad
körtgenn
WSheers
Corrad
Röntgen
recevved
Price
Physiss
his
discovery
Wilhhrs
Corrad
Ronigan
secaived
the
Trut Robel
Price
In
Physics In racogriliin hi
d scorvery
of
Xrays
which
Here
Becquend who recelued the
tixt Nobel
Prtze
In Physics
Hendrr
Antoon
Lorontz
Zooman
rocahred
firtt
hood
Prizs
Prysics.
Elenaric
Antoon
Lorentz
wnd
Fieber
findt
Noow
Prize
Physicx
effect
of
magnetic
Teids
Ight
enblted
by
stoms
known
effect
Figure 11: SelfReflect per-word penalties on how far the prediction of each masked-out word based on the summary "Wilhelm Conrad Rönígen received the first Nobel Prize in Physics." differs from the prediction based on the samples from the internal distribution. Total penalty: 0.102.
We can now improve this summary by adding the two other possibilities, namely "It's most likely that Wilhelm Conrad Röntgen received the first Nobel Prize in Physics. But the laureates could also have been Hendrik Antoon Lorentz and Pieter Zeeman or Henri Becquerel.". With this better summary, Hendrik Antoon Lorentz, Pieter Zeeman, and Henri
SelfReflect correctly removes the penalty Becquerel. But it correctly still penalizes the summary for not mentioning the details of any of the works. This results in overall score of 0.084 (or 84).
Wilherm
Conred
Rüntpen
incelved
the Test Robel
Frize
In
Physics
Physics
körtgen
Wilhhim
Conred
Röntgen iereived the Tedt Robel Frize
In
Physies
for
hi
dicovery
J-yyys
Binipan
Physirs
recosriltin
seewery
are
now named
afte sims.
Besauard
Nobel
Prise
Physks.
hondri is
Antoon Loronnz and Picter Zoomon rocalee the first hoze
Prize
In
Physis.
Hondrix
Ficter
Prize
effect
of
magnetic
Felds
0n
the
5pe.trum
of
lgft
enktee
by
atoms.
known
35
the
Zeamian
offect
Figure 12: SelfReflect per- word penalties on how far the prediction of each masked-out word based on the summary "It's most likely that Wilhelm Conrad Röntgen received the first Nobel Prize in Physics. But the laureates could also have been Hendrik Antoon Lorentz and Pieter Zeeman or Henri Becquerel." differs from the prediction based on the samples from the internal distribution. Total penalty: 0.084.
Having added all answer possibilities, we can now add details mentioned in the individual answers. As good summary, we give "It's most likely that Wilhelm Conrad Röntgen received the first Nobel Prize in Physics in recognition of his discovery of X-rays which are now named after him. But the laureates could also have been Hendrik Antoon Lorentz and Pieter Zeeman or Henri Becquerel." " SelfReflect removes the penalty X-rays, which the summary mentions. The remaining penalty of
29

---

Published as a conference paper at ICLR 2026
0.078 (or 78) is due to the summary still not mentioning the details the Zeeman effect, plus some remaining noise mostly on the names.
Wihhirs
Corrad
Röntgan
ccdlved
Physks
The
Tind
Nobell
Price
Physics
Wan
MAarded
16
Villeem
Corred
Rontieen
Milhain
oodived
Physks
Wihhimm
Conned
Rontgen
ieceived
Nobel
Prize
Physicx
recogrittin
dccoveryy
Xrays
whith
M35
Hann
Physks.
Trendric
Antoon
Lonentz
Wnd
Pieter
Zee-tan
THc Rored
the
fint
Noze
Price
Phyecs.
magnaticc
Ight by
stoms.
known
Figure 13: SelfReflect per-word penalties how far the prediction of each masked-out word based on the summary "It's most likely that Wilhelm Conrad Röntgen received the first Nobel Prize in Physics in recognition of his discovery of X-rays which are now named after him. But the laureates could also have been Hendrik Antoon Lorentz and Pieter Zeeman or Henri Becquerel." differs from the prediction based on the samples from the internal distribution. Total penalty: 0.078.
These examples demonstrate that SelfReflect punishes summaries for the correct reasons: Either when they don't mention all possibilities or all details of the actual internal answer distribution. We have seen in Sections 4.1 and 4.2 that SelfReflect also correctly punishes deviations from the relative frequencies. To this end, let modify the second summary which previously had score of 0.084 (or 84) and state that Henri Becquerel was the most likely first Nobel laureate, which is in conflict with the LLM's internal answer distribution: "It's most likely that Henri Becquerel received the first Nobel Prize in Physics. But the laureates could also have been Hendrik Antoon Lorentz and Pieter Zeeman or Wilhelm Conrad Röntgen.". ". This correctly leads to higher penalties on Wilhelm Conrad Röntgen and Henri Becquerel because both of their implied probabilities are off (while keeping the same penalties on Hendrik Antoon Lorentz and Pieter Zeeman, as well as the in both cases unmentioned details on their works) and worsens the SelfReflect score to 0.092 (or 92).
Wilsarms
Corrad
Rönigeni
scclived
the
Trut
Nobel
hrue
In
Physic
The
Finst
Nnbiel
Price
Willeem Conred
Rönigen
Wilherm
Conred
Ronipen
weeiued
Robel
Physirs for hi
dicroeery
X-wys
Wilhhrrs
Corrad
Röntgen
sodived
thc
Test Nobal
Prize In
Mhysks recogritiin
scrvery
Xirass
which
M35
Honni
Becouerd
w/o rocoived the fist Nobel Price in Physks.
llenaticc
Antoon
lorentz
Fleber
Nabw Prize
Antoon
Lorontz
rocahoed
Prizs
Plysics
effect
of
magnelic
Taids
coectrum
of
ight
#nilled
by
atoms.
known
the
Zeeman
effect
Figure 14: SelfReflect wor-word penalties on how far the prediction of each masked-out word based on the summary "It's most likely that Henri Becquerel received the first Nobel Prize in Physics. But the laureates could also have been Hendrik Antoon Lorentz and Pieter Zeeman or Wilhelm Conrad Röntgen." (note that Henri Becquerel is in fact not the most likely; it is Wilhelm Conrad Röntgen) differs from the prediction based on the samples from the internal distribution. Total penalty: 0.092.
This demonstrates that SelfReflect's score works as intended, not only on the dataset or question level as studied in the main paper, but also on a word-level granularity. This example is regular case, one of the 95%+ (see Table where SelfReflect correctly scores the summaries. We note, however, that there are around 5% of questions where it does not score correctly. most of these cases, the scores of good and slightly worse summary due to noise. We thus recommend to run SelfReflect over 1000 questions per dataset, as noted in Appendix B and the main paper, in order to smoothen some of the remaining noise.
very close to another and the mis mis-decision is mostly
30

---

Published as a conference paper at ICLR 2026
E IMPLEMENTATION DETAILS
E.1 SELFREFLECT SCORE
To calculate the SelfReflect score, in every masked-out task we run the two prompts in Fig. through a judge LLM, which is by default Qwen 2.5 7B In--rutruct. This makes the judge predict the logits over the vocabulary size for the current token of the fill-in word. If a fill-in word consists of multiple tokens, where we add the tokens of the true fill-in word one after another into the autoregressive context of the assistant answer. Given the two fill-in token vectors conditioned either on the summary or on the concatenated answers, we apply a temperature of flatten it. This is in order to give some weight to synonyms, since instruct-tuner LM judges otherwise would give nearly probability I to only one possible token (in which case SelfReflect would still be valid, but simplify into comparing whether the two contexts lead to predicting the exactly same word). We found that a temperature of T improves the SelfReflect score, making able to discern good from almost- good summaries more often validation dataset. We then softmax the flattened logit vectors and caleulate the 1- Wasserstein distance between the log probability vectors. Since these are categorical vectors, the I-Wasserstein distance simplifies into the L1 distance, times 0.5. We repeat this over all tokens of a masked-out word, then over all masked-out words of each of the M = 50 answers (that are not stopwords), then across all 1000 questions of dataset. The global average gives the SelfReflect score.
E.2 SR SAMPLING-FRED SCORE
The sampling-free free ablation of the SelfReflect metric also gives two prompts to a judge LLM to calculate the masked-in task. The difference is that the prompt which in SelfReflect contains the sampled answers does not contain sampled answers. Instead it just gives the question and then the masked-out task.
E.3 ER-PMI SCORE
The PMI ablation of the SelfReflect metric uses masked out task. Instead it poses the question, gives either the summary or the sampled answers as background information in context, and then measures the logit vectors assigned to each token of each of the M 50 answers. In other words, the answers are not given word-by-word with masked-out tasks, but measured one full answer. As for SelfReflect, we then calculate the I-Wasserstein distance between the flattened logit vectors and average.
E.4 SR-P(TRUE) SCORE
In the P(True) ablation of SelfReflect, turn the generative masked-out task into discriminative one. We first generate three candidate words to fill in the masked word: One is the true masked word, is word sampled from the masked-out task prompt given the summary and the last is word sampled from the masked-out task prompt given the distribution samples. With these candidate fill- in words, we then run two prompts, one conditional on the summary and one on the answers, to let the judge LLM predict how likely they fit in, see Fig. 15 As in the normal SelfReflect, this gives distribution over the vocabulary size, concentrated on "True" and "False" tokens. We then compare the two flattened logit vectors via the 1- Wasserstein distance and average as in the original SelfReflect.
E.5 EMBEDDING SCORE
We compare the Qwen2-7B-inntruee (Li al. 2023 embedding of the summary to the embedding of the samples of the distribution, normalize them and take the inner product to form cosine distances. We average over all samples. The reason why select this particular embedding model is that at the time submission it was the best-performing open-source model on the MTEB benchmark (Enevoldssen al. 2025).
31

---

Published as a conference paper at ICLR 2026
andis
the Fire:
question
fire: fustral lar prise wirletort carpie 20 snavera tiia qunatico.
(iLs first histralian price sinister,
cindidite
1he that the fixes fustral prins
Lid. samples
could alos Anstrww rinker Edeand Deak in
Brom py(Alg
pasted contexts
4,30
flere paraco to officially sacve
discininative
The fizas Austeation scine Mizsatec admind was slected sn 1861.
tok will
The fisas Austention Prine Miszates Rdaund was elacted in 1961.
meskefont
tere sbatiee candidate sould
wnrd
wouid dis vord iTruet or not (ralses, based on the anaver you gave sbowe.
abactes
unoer b.
hespord only
and candidaic werd
word ITros not oeausees based on the arsver you gave abowe hespord on
andis
<Lo soart (Pasatatent
eltm soart Dassaiatent
Predicted token voctor; p,( (g. 9)== (0.69, 0,30, ...)
Prodicted teken vector: p,0l a () (0.82, 0.16, ...)
"rou"" *Maloo"
"ceas" "valso"
W (ps (X 1. wo0))) PP (x 9, (1:N3 s1 )m))
mISRATTee (S)
The P(True) ablation of SelfReflect adds candidate word oj into the context and asks the judge LLM to classify whether this word fits as masked-ou word or not. It compares the probability vectors predicted given either the summary or the concatenated samples.
Figure 15:
E.6 SUMMARIZATION SCORE
We follow the prompts of Jain al. (2233) that prompt an LLM to judge a summary in terms of consistency, fluency, relevance, and coherence with few-shot example. We then normalize all scores [0, and average them to get the summarization score.
Prompt for the 'Summarization' metric in Table [ to judge fluency.
Fluency measures the quality of individual sentences, and whether they are well-written and grammatically correct. Rate the summary of given text on scale of to 1 fluency.
Here are some examples: few-shet examples>
Now here is the summary whose fluency you are supposed to rate: Summary: (summary)
Fluency:
Prompt for the "Summarization" metric in Table judge coherence.
ccherence, with a higher value correspending to higher coherence. Coherence is a collective quality of all sentences. To score highly on it, the summary should be well-structured and well-organized. It should not just be heap of related information, but shculd build from sentence sentence to about the topic.
Rate the following summaries
scale
coherent body of information
Here are some examples: <4 few-shot examples>
Now here is the summary whose coherence you are supposed to rate: Summary: (summary)
Coherence:
Prompt for the 'Summarization' metric in Table to judge consistency.
Consistency measures whether the details the sunmary reproduce the facts present in the text accurately. Rate the summary of
32

---

Published as a conference paper at ICLR 2026
given text scale from 0 to 1 consistency.
Here are some examples: few-shet examples>
Now here is the text and summary whese consistency you are supposed
to rate:
Text: We received many answers our question '(question)'. Here they are:
x_1 (_ (answer))
answers) (answer)'
Summary:
(summary)
Consistency:
Prompt for the 'Summarization' metric in Table [ to judge relevance.
Relevance is the quality of a sunmary to capture important information from a reference text. Rate the summary on scale from 0 to on relevance.
Here are some examples: few-shet examples>
Now here is the text and summary whese relevance you are supposed
to rate:
received many answers our quest on (question)".
they are: x_1 x_'=((ansee) (answer)'
(n answers) (answer]'
Summary:
(summary)
Relevance:
E.7 LM JUDGE SCORE
We follow Xu et al. (2024) to build metric that asks LM judge to chain-of-thoughts think and rate how well summary matches distribution of answers, including a few-shot example. The prompt is shown below.
Prompt for the "LM Judge' metric in Table
Your task to analyze whether summari zed answer correctly contains all the possibilities that len (answers) individual answers to question mention.
Note that some individual answers occur more often than other individual answers. You should output score from to 10, indicating whether the summarized answer mentions all possibilities and whother it correctly outlines which are the most often appoaring individual answers and which appear less often. A higher score is means the summarized answer matches the distribution of individual answers better.
Also note that some individual answers may be factually wrong. Do not correct those, just report how good the summari ri zed answer matches the individual answers.
You should first provide your reasoning for well the summarized answer matches the distribution over individual answers, and then assign score based on this reascning. The output should be in
33

---

Published as a conference paper at ICLR 2026
the Following format:
Reason: [REASON]
Score: [SCORE]
Here is example:
Question: <Example question>
Individual answers:
<Rxample answer samples>
Summarized answer: <Example summary>
Then your output can be:
Reason: The summarized answer mentions the most likely pessibility, and it also correctly mentions that this is the most likely one. For other possibilities, it mentions wilhelm Conrad Röntgen, but does not mention that got the award for his discovery of x-rays, which the individusl answers mention. It also does not mention the possibility of Bendrik Antoon Lorentz and Pieter Zeeman, which the individual answers mention. Score: 8
Now consider the following case: (question)
Individual answers:
x_1 (annserr)
x in answers) = ' [answer]"
Summarized answer: (summary)'
Please provide the reason and the score of how good the summarized answer matches the distribution of individual answers.
E.8 OPTIMAL TRANSPORT SCORE
The optimal transport metric consists of steps. First, break down summary into distribution of statements and their probabilities. This is done with the following prompt.
Prompt for the 'Optimal transport' metric in Table T to split summary into core statements and their probabilities.
Question:
(question)
Here is background information. This background information defines distribution of possible answers you can later sample from:
(summary)
Now, split this distribution up into its mutually fundamental statements and the explicitly or implicitly connected probabilitics.
Split it up such that each statement is mutually exclusive and the probabilities Include don't know" statement with the remaining percentage the background information explicitly mentions not being certain.
1.
Return json file with 1ist of dictionaries, where in every dictionary the first key is called 'prob' and includes the numerical probability and the second key 'statement' and includes a string the fundamental statement.
34

---

Published as a conference paper at ICLR 2026
In the second step, tween how much each sample answer entails each statement and vice versa. We multiply (1 entailment probability) of both directions to get distance score for each sample answer and state- ment. This defines distance matrix with the statements as rows and the sample answers as columns. Besides distance matrix, optimal transport also requires marginals for both rows and columns. For the rows, we use the probabilities assigned to the statements in the above prompt. For the columns, we assign each individual sample answer a uniform probability. We then compute the earth movers distance using Flamary et al. (202T). This matches sample answers to summary statements in such manner that the marginals are preserved and that overall all pairs in sum have the smallest possible distance. The resulting overall distance then tells how far the answer samples are from the summary.
NLI model to calculate an entailment probability in [0, 1] be-
E.9 INFOSUMM SCORE
InfoSumm (Jung et al. 2024) is a metric from summarization literature to compare the pointwise mutual information between summary and document string based on masked-out tasks, similar to SelfReflect in spirit but with some notable differences in the estimator. InfoSumm expects to have summary string and long document, which corresponds concatenated answers from the sampled distribution a (1:N) in our case. They then estimate PMI once via saliency and once via faithfulness. Saliency predicts masked words of the long document with an LLM judge that given the summary, and faithfulness predicts masked words of the summary with an LLM, judge given the long document" [6) After removing terms in the denominator that are independent of the summary in question, this gives log likelihood terms:
saliency_score log (PLIM, (ALL))) am
(I:N)
Q q. A(::N) a ((2:N)) .)): (22)
faithfulness_ score log (LLLM, (Sm 3m Q = q, S_m s-m Al1:N (1:N)
(23)
where n. denotes masked-out words and --m the remaining non-masked text. To make the masked- out task for saliency less ambiguous to the LLM judge (since there may be duplicated answers in (1:N), ) we compute the saliency score per answer:
saliency_score- Σ log (PLLM, (4% = am 11 9, A" a. a=mms))
s)).
(24)
In the original paper, these terms are thresholded and used as filters to create train datasets. In our application as score, we add them up and multiply by (-1) to ensure that lower is better. This gives the InfoSumm baseline:
InfoSumm InfoSurn_score:: score == -saliesscs__sccooo + faithfulness__cooe
(25)
We integrate this over the masked-out tasks of all M answers as we do for SelfReflect.
E. LICENSING INFORMATION
Table X contains licensing information for models used in this paper.
'There is no differentiation between answers for the conditioning and for masked-out tasks b in their setup. hecause they condition one the other.
35

---

Published as a conference paper at ICLR 2026
Table Licencing information for models used in this work
LLM
License
Reference
DeepSock RI Distil Qwen 2.5 32B
MIT
DoepSeck-At Al et
2025
Gemma family
hctpss/aal.poooeeedde Lerns
Gemma Team ot al
2025
Gemini 2.0 Flash
Apache
Ministral 8B Instruct
Jiang al. 2024
1conses/MRI-0-
Llama 3.1 70B Instract
hctpat/ /www ama..omm com/ 11ama a 1/
Meta AT 20746
icense/
Llama 3.3 70B Instruct
7w0 1wa.a. cwm///c.com////// 3mm
Meta Al 20245
License/
Llama Scout Instruct
hitps:
1/
lccanse/
Ahdin
Qwen 2 7B Instruct Qwen family
Apache-2.0
G Ai
Apache-2.0
Yang 21
2024a
Owen family
Apache-2.0
QwQ 32B
Apache-2.0
Qwen Team 20250
F RATING GOOD AND BAD SUMMARIES WRITTEN BY HUMANS
Table Mirroring Table we compare how often our SelfReflect metric, and other possible metrics, discriminates good from bad summaries of answer distributions. In this version, the summaries are written by humans rather than by Gemini, on disjoint set of questions. Mean 95% interval. Confidence intervals are larger than in Table because have less manually written summaries of answer distributions than the automated ones in Table
Good summarics vs bad summarics
Good vs almost-good
Detailed vs truncated
Verhulized uncertainty only majority answer
Verbalized vs or-ooncatenated
Percentage or-poncatenated
Metrie
Summarization
98. 20% 45%%
60.18% 201
70.00% 200
14%
17% 10%
51.72%
LM Judge
98. 98.50%
$4. 19.199 ser
55.00%
14. 44.48%
7..939-g17.40
24. 14.1145-2:
Opt. Transport
78.14%-4444mm
57.19%
10.00%
79.31% 114.70
72.41%
Emmeediing
74.85% 4055
44.01 96.01 amaa
60.00% 413
20.69% 9638
41.38% sen
13.79% cor
SR-PMI
85.33%
60.18%
75.00% 2
3.445
24. 24.14%
31.07%:s..4.60
SR-sampling-fres
12.22%-x2.am
80.24% -4.77
75.00%
62.07% -:7 6:000
62.07% 47..00
58.62%%
SR S2.PYTR PyTree)
47.90%
58.38% owra
35.00%
96.55
9....oo 6438
82. 76.
79.31%-a.4450 44
SeltRellect
In Table in the main paper, summaries from sampled answers. We choose an automated LLM approach because it is more scalable (with accordingly preciser 95% intervals) and reproducible than manual annotation. For reproducibility, we also report the prompts below. To ensure the quality of the results, we have, however, also replicated the experiments where we wrote the summaries manually for 334 questions of the Natural Questions dataset, on a disjoint split from those in Table [ Table / reports the results on how often SelfReflect, and other metries, rated good summaries better than their worse counterparts. The results are analogous to those in the main paper in that SelfReflect scores the highest on all metrics except on one where the P(True) ablation achieves a slightly better result.
Gemini 2.0 Flash generate various types of good and bad
Gemini 2.0 Flash prompt to generate 'good' summaries in Table 1
Below, you are given (n_answers) individual answers to the question [questtonn'.
Your geal is to summarize the (b_answers) answers into one answer.
The summarized. answer should mention the main possibilities mentioned by the (n answers) answers. a possibility is mentioned only once, it can be skipped so that the summary zemains concise.
If some possibilities are mentioned much more often than others, delineate which possibilities are more often found in the others by using worda like "most likely" and "could also be".
36

---

Published as a conference paper at ICLR 2026
The format of the sumnarized answer should be the same each individual answez. Provide only the answer, if it were part of the (n answess) answers, without statements like "The answers include...".
similarly, the summarized answer should the same wording as the original answers. If the original answer always uses "is situated", then use "is situated" and not "is located".
The summarized answer should reflect what the (n_answers) answers deem possible. They can contain factually wrong options. Do not correct those, just report the possibilities as they are given in the answers.
Here are the (n_answers) answers:
x_1 (answer)'
{_in_answess) [answer]'
Please provide the summarized answer.
Gemini 2.0 Flash prompt to generate an 'almost-good' summary from a 'good' summary in Table Also used generate 'truncated" from 'detailed' summaries.
Below, you given
to the question (question)',
Your goal to shorten the answer.
If the answer mentions multiple possibilities, only return the main possibility.
If the answer includes a main answer and details, remove the details.
The shortened answer should have the same format as the original answer. If the original answer uses full sentences, the shortoned answez should also use full sentence.
The shortenea answer should use the same wording as the original answers. If the original answer always uses "is situated", then use "is situated" and not "is located".
answer can contain factually wrong options. Do not correct those, just shorten what the answer says, even is factually wrong.
Original answer: (good_sunmary)
Please provide the shortened answer.
Gemini 2.0 Flash prompt to generate 'bad' summary from 'good' summary in Table 1
Below, you are given a response to the question "qqesticn]''
Your goal is change the answer.
The answer should generally stay clase to the original anawer, with only some key factual terms changed.
The answer might already be factually wrong, But the goal is still to change the key facts, so that the changed answer is different from the original one.
The changed answer should have the same format as the original answer. The structure should remain the same, only keywords should be exchanged.
The changed answer should also use the same wording as the original answers for any non-factual words. IS the original
37

---

Published as a conference paper at ICLR 2026
answer always uses "is situated", then use "is situated" and "is located". Original answer: (good_summary]
Please provide the changed answer.
For verbalized, percentage, or-concatenated and majority answers, first use prompt cluster the answer distribution into clusters statements and which answers belong to which cluster statement:
Gemini 2.0 Flash prompt cluster samples from answer distribution into list of cluster representatives and cluster memberships.
Below, you are given (n_answers) individual answers to the question (qquestion'" These [n_answers} answers can be seen as samples from an answer distribution. Your goal is to cluster the distribution in two steps:
First step: Find the clusters and their representatives.
Each cluster contains set of answers that are essentially the same. This means they may vary
level of detail, but their primary answer should be the same.
fiferent clusters should be mutually exclusive answers.
There are at least two clusters.
The answer contain factually wreng options. Do not correct them, just cluster answers they are.
Output a json file with each entry giving the "cluster_id" id (cluster_1, cluster_2, ...), and a "representati ve_nswerr", copy-pasted from the
below.
Second step: Match the answers to their clusters.
to Match each of the [n_answers) individual cluster representative.
Output json file with each entry giving the "cluster_id" (eluster_1, cluster ...) and the "cluster members", list [x_1, x_2 ...l.
Here are the (n_answers) answers:
x_1 (answer)
(_n_aeeerrs [answer]'
Please output the two json files, one after another. Each json file should start with "ijson
We then count how many member each cluster has (manually in code opposed asking the LLM since this increases accuracy), and provide lists of the representative answers of each cluster and their relative frequencies build the percentage and or-concatenated summaries. We sort the resulting list frequency. For the majority answer, we directly return the representative answer of the highest-likely cluster. The verbalized uncertainty summary built by removing the percentages in their brackets from the percentage summary.
Gemini 2.0 Flash prompt to generate pprrentagg--ucceraiinyy summaries in Table |
Below, you are given list the question "qquestion'".
with their arobabilities to
Your goal is to stitch these answers together into one sentence. most likely
The sentence should have the structure that <Answer A> (<pzobability of Answer A>8 sure), , but it
38

---

Published as a conference paper at ICLR 2026
could also be <nnser B> (<probability of Answer B>$ sure) <Answer C> (<probability of Answer C>$ sure)
Stick to the original wording of the answers as much as possible, but you can add small words so that the sentence becomes gramnatically coherent sentence.
The answer can contain factually wreng options. Do not correct those, just stitch together the answer options, even if it is factually wrong.
List of answers:
prob' 0.72, 'statement':
prob' 0,22, statement/;
Please provide the coherent sentence.
Gemini 2.0 Flash prompt to generate or-concatenated" summaries in Table [
Below, you are given 1ist of answers with their probabilities to the question (question)".
Your geal is to stitch these answers together into one sentence.
The sentence should have the structure 'Either <Answer A> <Answer B> <Answer or ...'
The sentence should be grammatically ccherent.
Stick to the original wording of the answers as much as possible.
The answer contain factually wreng options. Do not correct those, just stitch together the answer options, it is factually wrong,
List of answers:
'prob': 0.72, 'statement':
prob' 0.22, statement':
Please provide the coherent sentence.
39

---

Published as a conference paper at ICLR 2026
G HOW WELL DOES SELFREFLECT DISTINGUISH GOOD FROM BAD SUMMARIES
Table 10: How well does SelfReflect, and other metrics, discriminate between good and bad sum- maries of answer distributions. For each column, the second summary lacks textual details or misrep- resents relative probabilities. Mean 95% confidence interval. Results per dataset, where Table T shows results averaged across datasets.
Good summaries bad summariis
Good almost good
Detailed vs truncated
Verbalized uncertainty vs only majority answer
Verbalized vs or.concalenated
Percentage vs or-concutemated
Metric
Natural Questions
Summarization
98.70%=2mm0
40.67%
53.55%
11. 11.57%ba
57 77.02%ka
65. 19..9%
LM Judge
99.10%==....
50.21% 135
65. 16% dons
24. 24.99% 27.00%
33. 8858 45.45%
38.02% es%
Ope. Transport
91.89% 1988
56.50% 25
45. 16
48 76%
47. 11%
70.25%
Embedding
94.49% 41.7
31.34% gg ware
47. 47.10
5.79% 4015
46.28%-gg..o woss
38.02% 6038
SR SR-logl
99.10% =0.0a
90.04% oor
90.97%
66. 16.94%% u. 3..3
39.67% s..72s
49.59% s..9
SR. PMI
90.89%
47.90%
69 09.668F22..00
23.97%
9.09%-a5
16.53%%
SR sampling- free
96.30% 1.13
74.42% mor
82. 22.588--33.0m so0m
39.67 7osm
30.58 2113
SR- SR-P(True)
55.86%
74.95%
62.58% s7.00s
92.56%
69.42% s5.215
85.95% 1..19
SelfReflocr
90.90%:=..002
98. ao
95.04%
74.38%ee..m6
43.47%
SimpleQA
Summarization
15.40%:
36.36% 1iw
50.00%k
26, 26.32%%2
63. 16. tes
18.42%
LM Judge
96.30% 1.17%
50,3 16 1.19
1169% 46.67%
10.53% 999
31,58% 20099
31.58%+
30006
Opt. Transport
68.70% 1.24
16.67% 13349
22.999
21,93
Embeedling
97. 77.80% caes
81.61 16-: 1475
83.33% 13330
10.53% 11.99
42. 11::1 25.395
84% 21.00
SR-Jogl
91.50%
84.09%
100.00%
36, 84.8%%%..
31,58%
31.58%
SR-PMI
84.00% 2279
25.21 20.211 2.we
33.33% 1ea7s
26, 26.32%% 19899
21.05%: 18.398
26.32% muse
SR- SR-sampling--ree
79. 20.20R%
36.47% 1:
60.00%
15, 75.9%
31.58% 1 m0999
52. 77.6366
SR-PYTrus)
72.10%
87.50% 2.0%
83.33% 13.44
841.21%
21,05
78.95% 1532%
SelfRefleer
96.60% 1339
91.63% 1340
100.00%
68. 18.42%% 188.42%%
68.
42.4%% 20:99
73.68% PAAE
TriviuQA
Summanizttion
95, 25.900% 1.799
43.02% 1::90
53.25% 1966
37.25% 11:37
58.82% 1/9
62.75 22.75% 11:
LM Judge
99.60%
39.35%
54.55%
9.80%
37.25%
29.41%
Opt. Transport
75.10% 2400
55.71% 1.99
37. 37.66%% 10825
50.98% 13:725
62.75% 12.7%%
66.67% 12/4%
Embeediig
97.20%
89 89,42%
96, 10%
23.53%
39. 39.22%%
33.33% 33%
SR-logl
98.50% 6179
22.79%
72.73%
45.10%
47.06% 199
SR-PMI
90.30%
25 93.95%%
28. 28.57% 10399
29.41% 12311
23.53% 11.45
27.45% 12.2%
SR samplling-free
89.30%
53.60%
59. 74.%
45.10%
49.02%
50.98%
SR. P(True)
67.90% 3.ses
83.64% 3.706
77.92% 4.10
78.43% 11:396
80.39S 1049s
90.28% 5.19%
SelfRellect
99.80%
87.87%
80.52%
68.63%
70.59%
74.51%
H MMLU TESTS OF THE SELFREFLECT METRIC PER DATASET
Table 11: Rank Correlations between how SelfReflect, and others, rank good/overonfidee//rrndoo summaries of MMLU multiple-choice summaries versus how metric specialized for this task does. Mean + 95% CI. Per Q means we check the rank correlation on each individual question and then average the rank correlations globally, Avg means let the approaches calculate the average score across all good/ccoffident/andom summaries of the 1000 questions and then rank them based on the average (like in a benchmark).
Gemma 12B
Qwen 2.5 7B Instruct
Metric
Per Q
Avg
Per Q
Avg
Summarization
0.44+
0.80
0.54+ ous
0.80= 0.00
LM Judge
0.80 umz
1.00= ase
0.56 om
0.72_non
Opt. Transport
0.69 ozz
0.88 uoi
0.60 am
0.81 us0
Embedding
0.29+ ama
0.18= o.2
0.02+ one
0.40= 0.00
SR-logl
0m581
1.00 noo
0.64 am
1.00 nco
SR-PMI
-0.03
-0.20 0.0
0.29 o
0.47=no no1
SR-sampling-free
0.57
0.83
-0.161 am
-0.42.
SR-P(True)
0,66+
1.00
0.08 our
0.20=
SelfReflect
0.
1.00_
0.56 om
0.93
40

---

Published as a conference paper at ICLR 2026
I USER STUDY DETAILS
User studies were carried out using TryRating. with five raters per task. Raters were allowed to rate as many tasks they wanted. All raters were US-based English speakers, and were paid $18/hr.
The users were presented with the instructions shown in Fig. 16 which included two examples with hand-crafted summaries (for space reasons, we include only one summary here).
Task Guidelines
You shown question, and
that question.
two possible summartes survey responses. Your tack the summary that best summanzes individual responses.
Some consider:
summary viewpoints responses? which viewpoints are less prevalant?? great summary reflects viewpoints, and makes clear which
newpoinns. good reflects sieapoints, but might not indicate which more or less common. OK summary only reflects same viewpoints, A bad summary does not reflect any of the siewpoints.
acourately excludes key information or edds information thai is not mentioned in the survey resporises. equaiily
the
siewpoints?
information
summary one olther
summary easy
survey answers,
to read and understand.
knnelldgge
question!
assess which summary y best representi the sempled responiess (which might not recilyy sowrr the question).
Here few examples:
Question: where do peaches come from in the us?
Peeches come from several states the US, primeriy Georgia, Califorii, and South Carolina.
Peaches U.S.
Califamia, Peaches primarily come from Calfernia, followed by South Carolina and Georgia.
South Carolina.
Peaches
Georgia, Peaches are primartly grown in states like Georgla, California, and South Carolina the US.
primarly
Peaches primarily come Calffonii Georgis the
from Georgia, California, and South Corolina. Paaches the US primarily come from California, Georgia, and South Carolina. leading producer Poaches primarily come from California, which is the leading producer in the U.S.
Poaches U.S.
Peaches primaniyy produced
Summary
Summary 2
Peaches primarily come from Califoria, followed by Georgia and South Carolins.
Peaches primarily come from Calffonia.
Summary
Summary
summmary. identiifie choice. In contrest, Summary 2 does not mention South Carolina Georgia.
example Summary
responses, California primary
Figure 16: Instructions for user study (truncated; actual instructions contained second example, which we have here for space).
To ensure quality responses, constructed twenty "golden answer" tasks, where the summaries were manually constructed to either the definition of "good", "nearly good", or "bad" summaries, as described in Section4.1 Ten of these questions answer the golden answer in 80% of tasks to proceed. The remaining ten questions were periodically included as verification checks. A total of 215 raters passed the entrance exam and contributed ratings.
given entrance exam, with raters required to
Confidence intervals were calculated using 100 bootstrapped samples.
41

---

Published as a conference paper at ICLR 2026
J AUTOMATIC SUMMARY GENERATION
J.1 EXPERIMENTAL DETAILS
In this section, we denote sampling parameters as (T=1, topp=1, topk=None, mi minp=0).
Non-reasoning/RL.F models For the models in Table 4, the same model is used for both gen- erating the answers, and generating the summaries. We sample answers with (T-1, topp=1, topk=None, minp-0;. For summaries generation, we use greedy decoding, i.c. (T=0, topp=1, topk=None, minp=0).
we also want sample the answers with- out reasoning and make use of the reasoning only for the summaries generation. For Qwen3, it is possible to suppress the reasoning with tokenizer awwly_ppppllaaeee wwwpchhttttttteeeeeeeeeememm emplate (..., For QwQ-32B and
Reasoning/RLVR models For the models Table 5
enable_st enable_thinkil see_hhinninggFFaa not find a way to suppress reasoning and hence we sample the answers from Qwen2.5533B-IInsrruct which is the RLHF model which served base model for the RLVR training. We sample answers with (T-1, topp=1, topk=None, minp=0)..
Seeiii-----Diiiil----ie- --Dissill-QQeen25--333 we did
For the Greedy summary generation, we non-reasoning greedy decoding with the same model as for the answers generation, i.e., for rows QwQ-32B and Qwen2.5-32B Instrrcct.
221-Distill-Qwen25-22222 we use
For the Basic and Sample Summarize summaries generation, we use the reasoning mode of each respective model. Unlike for RLHH models, we stray away from using greedy decoding for summary generation, because the creators of the reasoning models we use warn that the use of greedy decoding with reasoning "as can lead to performance degradation and endless repetitions". Hence, for each model we use the respective recommended sampling parameters available on their respective HuggingFace model card.
J.2
PROMPTS USED
Prompt for the basic summary generation method in Section 5
Please respond to the following question (question)".
Your goal to summarize all possible answers to this question:
If there are multiple possible answers, the summarized answer should mention. the main possible answers. However, you do not have 1ist possibilities that are too unlikely.
It some possibilities are more likely than others, delineate which possibilities are more more likely by using words like "most likely" and "could also be".
The format of the summarized answer should be the same as normal answer.
If there only clear answer to the question, just provide that answer, without hedging across possibilities.
Please provide the summarized answer.
Prompt for the CoT summary generation method Section 5
Please respond the following question (questton)".
Your geal is to first reason about all possible answers to this question and then summarize them into final answer:
Reflect on whether there are multiple possible answers to this question.
42

---

Published as a conference paper at ICLR 2026
If there are multiple possible answers, the sumnarized answer should mention the main possible answers. However, you do not have to 1ist possibilities that are too unlikely.
If some possibilities are more likely than others, delineate which possibilities "most likely" and "could also be".
more more likely by using words like
The format of the summarized answer should be the same as typical answer and be stand-alone.
If there is only clear answer to the question, just provide that answer, without hedging across possibilities.
The output should in the following format: Reasoning: [REASON NG ABOUT WHICH POSSIBLITIES THERE ARE AND HOW LIKELY THEY ARE]
Summary: [SUMMARIZED ANSWER]
Please provide the reasoning and then the summarized answer.
J.3 RESULTS PER DATASET
Table 12: SelfReflect score & (x10 rounded for readability) for the SimpleQA dataset, averaged across 1000 questions. The results in small font relative to Greedy.
Model
Po (4 ④)
Single-decoding methods
Sample & summarize
unimodal
Greedy
Basic
CoT
N 10
N 20
Qwen2.5 0.5B Instruct
Yang et al. 2024a
1%
98
97
95
99
99
Qwen2.5 1,5B Instruct
Yang el al. 2024g
2%
98
97
93-s
90
89
Qwen2.5 3B Instruct
Yang el al. 2024a
5%
101
101
99
93
91-s
Qwen2.5 5 7B Instruct
Yang ci al. 2024a
15%
102+4
102+a
93-s
92-c
Qwen2.5 14B Instruct
Yang ct al.
20244
101
102
Qwen2.5 32B Instruct
Yang al.
20224
17%
103
108
110+s
94-a
Qwen2.5 72B Instruct
Yang el
20244
18%
99
100
88
87
Phi 4 Abdin al. [2244]
3%
99
99
97
89
87
Ministral 8B Instruct 2410 (Tiang et al.
2024
15
117
116
114-
109
107 -9
Llama 3.1 70B Instruct (Mela ATl 2024a
16%
97
97
97
90-6
90
Llama 70B Instruct (Meta A1 2023b
29%
100
103
113+15
92-a
91-s
Llama Scout
Instruct (Mela
2025
100
Gemma 1B Instruct
(Gemma Teamt ct
17%
123
134-11
135+11
124
118-
Gemma 3 4B Instruct
(Gemmaa
2025
25%
118
135
138
108
106
Gemma 3 12B Instruct
(Gemma Team ct
2025
35%
117
129
135
112
112
Gemma 3 27B Instruct
(Gemma Team et al
2025
49%
109
124
134+a5
103-6
101-7
43

---

Published as a conference paper at ICLR 2026
Table 13: SelfReflect score + (x10- rounded for readability) for the Natural Questions dataset, averaged across 1000 questions. The results in small font are relative to Greedy.
Model
Po (A
Single-decoding methods
Sample & summarize
unimodal
Greedy
Basic
CoT
N
10
N 20
Qwen2.5 0.5B Instruct
Yang et al. 2024a
5%
92
90
90
92
92
Qwen2.5 1.5B Instruct
Yang ci a.. 2024a
13%
90
91
89
85
84
Qwen2.5 3B Instruct
Yang el al. 2024a
30%
95
96-1
97+a
88
87-
Qwen2.5 7B Instruct
Yang ct al.
2024a
32%
98-4
101+7
90-s
89-s
Qwen2.5 14B Instruct
Tang et al.
217244
56%
97
100+a
85-0
Qwen2.5 32B Instruct
Yang
50%
100ce
104+10
89
89
Qwen2.5 72B Instruct (Yang ct al.
20244
48%
89
94
98+9
84
83-c
Phi 4 (Abiin al. 22244]
36%
89
88
92
83
82
Ministral 8B Instruct 2410 Ujang ct al.
2024
17%
101
99
99
95
94-1
Llama 3.3 70B Instruct (Mela ATl 2024b
68%
91
96
104,18
86-6
85
Llama Scout 17B Instruct (Meta A1
2025
59%
96+c
104+14
88-2
87-4
Gemma IB Instruct (Gemma
22%
113
126-1a
127+14
113+1
108-6
Gemma 4B Instruct
56%
106
123-17
128
100-a
Gemma 3 12B Instruct (Gemma Team ct
2125
57%
103
118-14
121417
99-s
99-6
Gemma 3 27B Instruct
(Gemma Team ct al.
2025
72%
100
116
121
98
97
Table 14: SelfReflect score 1x10-3, rounded for readability) for the TriviaQA dataset, averaged across 1000 questions. The results in small font are relative to Greedy.
Model
Pe (Al
Single-decoding methods
Sample & summarize
unimodal
Greedy
Basic
CoT
N
10
N
20
Qwen2.5 0.5B Instruct
Yang et al.
2024a
96-1
96
98.4
Qwen2.5 1.5B Instruct
Yang
36%
94+1
87-n
87->
Qwen2.5 3B Instruct
Yang
45%
99
100
91
Qwen2.5 7B Instruct
Yang al.
Z0248
60%
95
98 :a
100
91
91
Qwen2.5 14B Instruct
Yang el al
2024a
76%
88
93
94
85
84
Qwen2.5 32B Instruct
Yang ct al. 21244
79%
92
98
101 +8
89
89
Qwen2.5 72B Instruct (Yng et a1.
211244
85%
89 +a
90++
84
83
Phi Abdin et al.] 20244
69%
89
84
83
Ministrar Instruct 2410 [Jiang al..
2024
56%
104
103
103-1
Llama 3.1 70B Instruct (Mela AIl 2024a
82%
88
89
85
85
Llama 3.3 70B Instruct (Meta AI, 202416
92%
91
93
95.4
89
88
Llama Scout 17B 16e Insttruc (Mela Al
2025
81%
89
92.
96
87-
86
Gemma 3 IB Instruct (Gemma Team et al.
2025
40%
112
127
125
113
108
Gemma 4B Instruct
(Gemaa Team el 2125
76%
101
114-1a
118+17
96-s
94-c
Gemma 12B Instruct (Gemma Team ct al.
2025
84%
103 Pm
107+15
94
93
Gemma 27B Instruct
(Gemma
104+13
91
44

---

Published as a conference paper at ICLR 2026
Table 15; Runtime of generating different summaries in seconds prompt per GPU, averaged across all three datasets with 1000 questions each. Note that some models are sharded across multiple GPUs in this case, represent their total computational requirements, we report the summed runtime of all GPUs, ie., although a prompt may run through one second using four GPUs, we will count as four seconds total. In small font, relative comparisons w.r.t. Greedy.
Model
Single-decodie methods
Sample & summarize
Greedy
Basic
CoT
N 10
Qwen2.5 0.5B Instruct
Yang ct al. 2024a
0.93
1.26x1.35
1.26 1.36
1.79 1.98
Qwen2.5 1.5B Instruct
Yang ct al 2024a
0.94
0.91 xo.96
x.29
1.89 x2.01
Qwen2.5 5 3B Instruct
Yang et al. 2024a
0.84
x.85
1.00x1.20
1.82
Qwen2.5 7B Instruct
Yang èt al. 2024a
0.80
0.84
x.09
x2.35
Qwen2.5 14B Instruct
Yang et al
2024a
0,96
x1.00
x.33
2.44x2253
Qwen2.5 32B Instruct
Yang c1 a1
2024g
1.11
1.22xx..0
1.72
3,42x3.07
Qwen2.5 5 72B Instruct
Yang cl a1.
2024g
1.68
1.63
3.48x2.07
x.41x2.62
Phi 4 14B (Abdin et al. 2024
1.09
1.02
1,43
2.96x2.71
Ministral 8B Instruct 24TU (Jrang et al.
2024
0,91
x1.00 x1.00
1.04x1.15
1,82
Llama 3.1 70B Instruct Meta Al 20243
4.16
x,45x1.07
10,58
10,59 x2.35
Llama 3.3 70B Instruct (Meta Al 2024b
3,54
3.17x0.89
x,25x1120
x.23
Llama 4 Scout 17B 16e
Instruct (Meta Al
2025
3.45
3.92x1.18
5.67
9,05x2.62
Gemma 3 IB Instruct (Gemma Team
4025
0.89
0,86x0.97
0.93
1.74x1.00
Gemma 3 4B Instruct (Gemma Ieam
2025
0.95
0.93xo.96
1.09
1.96x2.06
Gemma 3 12B Instruct (Gemma Team
2025
1.12
1.14x1.02
1.54x1.38
Gemma 3 27B Instruct (Gemma Team
2025
1.32
1.36x1.03
2.26x1.71
3.09x2.34
Table 16: Character length of different summaries in seconds per prompt per GPU, averaged across all three datasets with 1000 questions each. small font, relative comparisons w.r.t Greedy.
Model
Single-decoding methods
Sample summarize
Greedy
Basic
CoT
Qwen2.5 0,5B Instruct
Yang ct al.
2024a
130.10
699.90 (09,93xxa. xx..ss
896.85x..a
155.39
Qwen2.5 1.5B Instruct
Yang et a
2024a
152.44
151 x4
528 x..4
310. 16x2.03
Qwen2.5 Instruct
Yang ci al.
2024a
81.55
168.60 (6..6
254.03
274.03 x3336
Qwen2.5 Instruct
Yang el al.
2024a
93.16
149.61 x1.61
198.33
178.72 1.92
Qwen2.5 Instruct
Yang ct m 2024a
92.93
170.49 1..s
245.97
203.53
Qwen2.5 Instruct
Yang et al 2024a
64.53
145,30
18..80x..88
138.33
Qwen2.5 72B Instruct
Yang et al
2024a
157.04 1.61
247.99 x..58
17..61
14B Abdii al.
124.85
203.35
281.96xa..0
273.05
Ministral 8B Instruct
Uning et al.
2024
49.15
100.54
179.57xx.e5
130.08
Llama 3.1 70B Instruct
Mcta All 2024a
168.64
267.6 067.63.. L..0
499.54
225. 03.
Llama 3.3 70B Instruct
(Meta
20246
113.47
229, 019,455xm x2.0.02
277.31
152.70
Ilama 4 Scout 17B 16c
Instrucr (Meta A1
2025
132.81
275.57 x2.07
220.68
214.80
Gemma IB Instruct
(Gemma Team et 20
4125
22.66
38.91 x1.772
183.15
61.7 x
Gemma 4B Instruct
(Gemma Team et al 4125
26.45
80.66xa.0s
113.46xx4.22
101.16x...
Gemma 12B Instruct (Gemma leam dl
2025
28.51
102.76 x.x..o
174.70x6.1a
70.67 x
Gemma 27B Instruct
(Gemma Team al
2025
37.77
137.6 x3..62xx.64
192.39x6.08
70.97
45

---

Published as a conference paper at ICLR 2026
K
EXPERIMENT DETAILS OF COT DEEP DIVE
K.1
RESULTS PER DATASET
TriviaQA
Natural Questions
SimpleQA
Answer Distribution Uhcertain
Answer Distribution Uncertain Certain
Answer Distribution Uncortain Cortain
36%
49%
16%
40%
2 %%
20%
3%
13%
9%
35%
9%
69%
Cortain
Uncertain
Cortain
Uncertain
Certain
Uncertain
CoT Summary
Summary
Summary
Figure 17: Confusion matrices between certainty of CoT summaries vs. actual answer distributions for Qwen2.5 72B Instruct. Judged by Gemini 2.0 Flash for 500 random questions per dataset.
We show the results per dataset in Fig. 17 By just looking at the marginals of the answer distribution, can infer that the question difficulty increases from TriviaQA to Natural Questions to SimpleQA, as the number of questions with uncertain answer distributions increases from 16% to 44% to 78% of questions. However, the majority of the CoT summaries uncertain for all three datasets (for 62%, 75%. and 89% of questions respectively). meaning that CoT is underconfident and Natural Quesions-i.e distribution. A balance is clearly needed. The Greedy method, by contrast, is overconfident. While it outperforms Co7' on average, it underperforms Sample & Summarize since will only present single option. In similar way, simply making CoT more confident would likely lead to detrimental effects on harder questions. Already as-is, there are relevant shares of questions SimpleQA and Natural Questions where CoT is certain while the actual answer distribution is not (9% each). Generating self- reflective summaries that are truthful to the model's 's internal uncertainty thus remains a challenging task.
TriviaQA suggests answers that do have high probability under the true
K.2 PROMPTS USED TO CLASSIFY CERTAINTY VS. UNCERTAINTY
We used Gemini 2.0 Flash to judge a summary mentions multiple semantically different answers not, and likewise if samples from the distributions do In particular, we provide it with the corresponding context, and ask it to classify into three classes: A) Fully certain, B) not fully certain, C) Completely uncertain. We then aggregate B and C form the general "uncertain" category (we found this to give better results than to directly give a binary task to Gemini, because it allows us explicitly tell how handle the "not fully certain" edge case). We manually verified that the classification is qualitatively correct. We show the prompts below, along with qualitative examples of how Gemini classified each summary and each answer distribution.
Gemini 2.0 Flash prompt to classify CoT summary in Fig. We call A. 'Certain' and group B. and C. into 'Uncertain'.
Below, you are given an answer to the question [questionl'.
Your goal is to classify which type of answer this is:
A. The answer is certain, it only mentions one answer option.
might mention one or two further answer options but judges them as less likely.
B. The answer not fully certain.
C. The answer is very uncertain. It mentions many mutually exclusive answer options, without clear single most likely answer.
Ignore differences in form and style. You are only supposed to judge the answer semantically
46

---

Published as a conference paper at ICLR 2026
Here is the answer: ICoT summary)
Please respond with the category what type of this answer this is. Respond only with A, B, or C.
Gemini 2.0 Flash prompt to classify answer distribution samples in Fig. We call A. 'Certain" and group B. and C. into "Uncertain"
Below, you are given (n_answers) individual answers to the question [question)''. These (n_answers) answers can be seen as samples from an answer distribution.
Your goal is to classify which type of distribution this is: A. The answers all do not contradict each other, to one two that differ from the majority answer.
B. The answers give multiple mutually exclusive answer epticns, but there is one answer option that is given in the majority of cases. C. The answers give multiple mutually exclusive answer opticns, and they are almost all different, without a clear majority answer.
The answers will have some natural variability. Ignore differences in form and style. You are only supposed to judge it answer options are semantically different.
Here are the (n_answers) answers: x_1 '(answer)'
[___answers "answer]'
Please respond with the category what type of this distribution this is. Respond only with A, B, or C.
Examples for bottom right quadrant: CoT classified as uncertain, answer distribution uncertain
Question: What day, month, and year was the municipality of Valdivia, Antioquia, Colombia, founded?
"The most likely founding date of the municipality of Valdivia, Antioquia, Colombia, July 16, 1679. However, some sources suggest could also August 15, 1679, or there might be earlier records of a settlement in the area."
CoT summary:
Answer distribution:
"Valdivia, Antioquia, Colombia, was founded on July 29, 1854.",
"Valdivia, Antioquia, Colombia was founded August 5, 1836.",
"Valdivia, Antioquia, Colombia, was founded on July 7, 1861.".
"Valdivia, Antioquia, Colombia, was founded on July 3, 1616.".
"Valdivia, Antioquia, Colombia was founded on July 12, 1854.",
"Valdivia, Antioquia, Colombia, was founded on January 27, 1616.",
"Valdivia, Antioquia. Colombia was founded on August 5, 1645.".
"Valdivia, Antioquia, Colombia, was founded on July 1, 1886.".
"Valdivia, Antioquia, Colombia, was founded on May 19, 1849.",
"Valdivia, Antioquia, Colombia, was founded on January 19, 1886."
Question: After the War 1812, Edward William McBride (1791-1834) worked as what for the king's printer, John Cameron, on the York Gazette until April 1815?
CoT summary: "Edward William McBride most likely worked as printer's assistant for John Cameron at the York Gazette, but he could also have been typesetter proofreader."
47

---

Published as a conference paper at ICLR 2026
Answer distribution:
"Edward William McBride worked as a printer for John Cameron on the York Gazette until April 1815.",
"Edward William McBride worked as a printer for John Cameron on the York Gazette until April 1815.",
"Edward William McBride worked as a printer for John Cameron on the York Gazette until April 1815.",
"Edward William McBride worked as printer for the king's printer, John Cameron, on the York Gazette until April 1815.".
"Edward William McBride worked as printer for John Cameron on the York Gazette until April 1815.",
"Edward William McBride worked as printer for John Cameron on the York Gazette until April 1815.",
"Edward William McBride worked a compositor for John Cameron the York Gazette until April 1815.",
"Edward William McBride worked as a compositor for John Cameron on the York Gazette until April 1815.",
"Edward William McBride worked as printer for John Cameron on the York Gazette until April 1815.",
"Edward William McBride worked as a printer for John Cameron on the York Gazette until April 1815."
Examples for top left quadrant: CoT classified as certain, answer distribution certain
Question: who hosted and won the inagural world cup?
CoT summary: "Uruguay hosted and won the inaugural FIFA World Cup in 1930."
Answer distribution:
"The inaugural World Cup was hosted by Uruguay in 1930, and Uruguay also the tournament.",
"The inaugural World Cup was hosted by Uruguay in 1930, and Uruguay also the tournament.",
"The inaugural World Cup was hosted by Uruguay in 1930, and Uruguay also won the tournament.".
"The inaugural World Cup was hosted by Uruguay in 1930, and Uruguay also won the tournament.",
"The inaugural World Cup was hosted by Uruguay in 1930, and Uruguay also won the tournament.".
"The inaugural World Cup was hosted by Uruguay in 1930, and Uruguay also won the tournament.".
"The inaugural World Cup was hosted by Uruguay in 1930, and Uruguay also won the tournament.",
"The inaugural World Cup was hosted by Uruguay in 1930, and Uruguay also the tournament.".
"The inaugural World Cup was hosted by Uruguay in 1930, and Uruguay also the tournament.",
"The inaugural FIFA World Cup was hosted by Uruguay in 1930, and Uruguay also won the tournament."
Question: Who received the IEEE Frank Rosenblatt Award in 2019?
CoT summary: "Yann LeCun received the IEBE Frank Rosenblatt Award in 2019."
Answer distribution:
48

---

Published as a conference paper at ICLR 2026
Yann LeCun received the IEEE Frank Rosenblatt Award in 2019.",
Yann LeCun received the IEEE Frank Rosenblatt Award in 2019.",
"Ynn LeCun received the IEEE Frank Rosenblatt Award in 2019.",
Yann LeCun received the IEEE Frank Rosenblatt Award in 2019.".
Yann LeCun received the IEEE Frank Rosenblatt Award in 2019.".
Yann LeCun received the IEEE Frank Rosenblatt Award in 2019.",
Yann LeCun received the IEEE Frank Rosenblatt Award in 2019.",
Yann LeCun received the IEEE Frank Rosenblatt Award in 2019.",
"Ynnn LeCun received the IEEE Frank Rosenblatt Award in 2019.",
"Yann LeCun received the IEEE Frank Rosenblatt Award in 2019."
Examples for top right quadrant: CoT classified as uncertain, answer distribution certain
Question: During what year did Tatsuo Miyajima have his first solo exhibition?
CoT summary: "The most likely year for Tatsuo Miyajima's 's first solo exhibition is 1988, this is the year most commonly cited in reliable sources. However, there a possibility that it could have been as early as 1986, though this is less well-documented."
Answer distribution:
"1988",
"Tatsuo Miyajima had his first solo exhibition in 1988.".
"Tatsuo Miyajima had his first solo exhibition in 1988.",
"1988".
"1988",
"1988",
"1988",
"1988",
"Tatsuo Miyajima had his first solo exhibition in 1988.",
"1988"
Question: At which University of Puerto Rico campus did New York State Representative Nydia Velázquez work as a professor from 1976 1981?
CoT summary: "The most likely campus where Nydia Velázquez worked as professor from 1976 1981 is the University of Puerto Rico, Río Piedras, given its prominence and alignment with her academic and political background. However, it could also be the University of Puerto Rico, Mayaglez or Cayey, though these are less likely."
Answer distribution:
"University of Puerto Rico, Río Piedras Campus.".
"University of Puerto Rico, Río Piedras Campus.",
"University of Puerto Rico, Río Piedras Campus.". ".
"University of Puerto Rico, Rio Piedras Campus.".
"University of Puerto Rico, Río Piedras Campus.", "University of Puerto Rico, Río Piedras Campus.", "University of Puerto Rico, Rio Piedras Campus.". "University of Puerto Rico, Rio Piedras Campus.", "University of Puerto Rico, Río Piedras Campus.". ".
"University of Puerto Rico, Río Piedras Campus."
49

---

Published as a conference paper at ICLR 2026
Examples for bottom left quadrant: CoT classified certain, answer distribution uncertain
Question: Danger Island, The Micro Ventures, The Arabian Knights and The Three Musketeers were short segments in which classic children's ' TV programme?
CoT summary: The most likely answer is The Banana Splits Adventure Hour, as it featured a mix of different adventure segments that could include titles like Danger Island, The Miero Ventures, The Arabian Knights, and The Three Musketeers.
Answer distribution:
These segments part of the classic children's TV programme "The Banana Splits Adventure Hour."
The programme you'te re referring to is "The Banana Splits Adventure Hour."
The classic children's TV programme that featured these short segments is "The Banana Splits Adventure Hour."
These segments were part of the classic children's TV programme "The Banana Splits Adventure Hour."
The programme you're referring to is "The Quatermass Experiment," " but it seems there might be mix-up. Those segments actually from the classic children's TV show "Vision On," which aired in the UK from 1964 to 1976.
These were segments the children's TV show "The Storyteller."
These segments were part of the classic children's TV programme "The Banana Splits Adventure Hour."
These segments were part of the classic children's TV programme "The Dame's House," also known "Dame's House" "The hastily assembled Dame's house for children." which aired in the UK in the 1940s and 1950s. However, the most well- incaraatio featuring these segments was likely "The Goon Show." " although it's important to note that "Danger Island" and "The Mícro Ventures" were specific to "The Dame's House."
The segments you mentioned were part of the classic children's TV program "The Banana Splits Adventure Hour."
These segments were part of the classic children's TV programme "The Banana Splits Adventure Hour."
Question: Which 'A-road' passes through Preston and Harrogate?
CoT summary: There is no single A-road that passes through both Preston and Harrogate. The closest options are the A65, which passes through Harrogate, and the A683, which passes through Preston.
Answer distribution:
The A65 passes through both Preston and Harrogate.
The A59 road passes through both Preston and Harrogate.
The A65 road passes through both Preston and Harrogate,
The A59 road passes through Preston and Harrogate.
The A65 road passes through both Preston and Harrogate,
The A65 passes through both Preston and Harrogate.
The A65 road passes through both Preston and Harrogate,
The A.59 road passes through Preston and Harrogate.
The A65 passes through Preston and Harrogate.
The A65 passes through both Preston and Harrogate.
50

---

Published as a conference paper at ICLR 2026
L FINETUNING TO GENERATE SELF-REFLECTIVE SUMMARIES
Table 17 includes experiments with post-training to generate summarized uncertainties. Despite testing multiple setups, we have not found a strategy that trains the model to output summaries of its uncertainties. It seems to capture the style, and the results for certain train samples, but fails to find generalizable mechanism. We see this as potential for future studies.
Table 17: SelfReflect scores of models that have been supervised finetuned (SFT) and/or direct preference optimized (DPO) on sample-and-summarizz summaries with LoRA adapters. Each epoch includes 10,000 example summaries. No approach consistently improves both in- and out-of- distribution over sampling greedily from Qwen 3 8B Instruct.
Finetuning
Train Natural
Eval Natural
OOD TriviaQA
Train TriviQQ
Eval TriviQQA
OOD
Natural
Qwen Instract
94
SFT (5 opochs), Ir 1c-4
90-3
92-2
96-1
96-2
96+1
92-2
SFT epoclss, Ir 1o-4
97
+ SFT (15 epochs), lr le-4
88 55-5
94-0
97-22
92-6
97
94-1
SFT (20 epochs), Ir le.4
87--6
98
92-6
98
94
+ DPO (5 epochs).
le-5
95
97--2
99
97
94
+ DPO (10 epochs). Ir le.5
95--2
95--1
97--2
100+22
974-2
9541
epochs), le.5
97-12
95
+ DPO (20 spochs), Ir le-5
96-3
96-22
97+22
99:11
97+2
95+1
ep.J, 1e-4 DPO ep.),
97+3
* INPO, Ir 1c-4 (20 epochss
99.66
98:11
102-7
102.44
100-5
98
DPO, Ir 10-4.3 0 cpoccc
93-0
93
95
98-0
95-0
93-1
epochs)
96+11
+ SPT epochs), Ir le-4 DPO, 1e-4 epochs)
97:-4
97--3
99:41
9:.-
99
9743
SFT (5 epochs), Ir Ie-4 DPO, le-4,
epochs)
90-15
96 -2
96
92
SFT epochs), 1e-4
(20 epochs)
9641
+ SFT (5 spochss, lt 1o-4 DPO, It Io-5. 0.5 (20 spochs)
92-1
94-0
96-1
96-2
96+1
93-1
M RESULTS ON RETRIEVAL--AUGMENTE GENERATION
The datasets we have used in Section 5 are closed-book question-answering datasets. This to have experimental setup in which there sufficient questions in which the LLMs have non- unimodal output distributions and must actually summarize multiple options. However, SelfReflect measure whether an LLM can summarize its current answer state no matter which previous context is responding to. To show this, we run an additional experiment on HotpotQA (Yang et al. 2018), retrieval- augmeeed generation (RAG) dataset in which each question is given along with background information from an according Wikipedia page.
Table 18: SelfReflect score 4 1x100- 3 for readability) on HotpotQA, RAG dataset. The results in small font are relative to Greedy.
Model
Greedy
Basic
CoTT
Sample & Summarize (N = 20)
Qwen 2.5 1.5B
76
76
76
73
Qwen 2.5 3B
80
81
81
79
Qwen 2.5 7B
79
79-0
80+1
78
Qwen 2.5 14B
78
80+2
80+2
76
Qwen 2.5 32B
79
80+1
80+1
78
Qwen 2.5 72B
78
79+1
80+2
76
Microsoft Phi
79
78-1
78
76
Ministral 8B 2410
86
86
85_.
84
Llama 3.1 70B
79
78
The results are similar to the main paper results: Sample & Summarize is the only summary generator that consistently produces better summaries than just giving the greedy answer. Note that the gap to greedy tighter here than in the main paper. This is because in RAG questions, the answer less uncertain, that distributions are more often unimodal and no hedging required.
51

---

Published as a conference paper at ICLR 2026
Table 19: Sample & Summarize summaries that summarize an increasing number of sampled answers. Generated by Qwen 2.5 72B Instruct on TriviaQA questions.
Answers used Sample & Summarize SeltReflect score 1. (x10 ?) Length (characters)
10
20
30
40
50
60
70
80
90
100
82
82
82
93.33
104.37
113.22
120.46
123.97
127.27
133.82
135.28
136.42
137.58
N BEHAVIOR OF SAMPLE & SUMMARIZE
To get better understanding the best-working approach, sample & summarize, we extend its analysis here. First, provide qualitative examples understand how summary strings generated by look like. We conduct this analysis on Qwen 2.5 72B Instruct on TriviaQA questions.
In 50% of the questions, the LLM is certain about its answer, thus the 50 samples from its answer distribution are (virtually) the same. In this case, the Sample & Summarize string just looks like a normal answer, for example:
Question:
The naval battle of Coronel in World War I took place in which ocean? Sample & Summarize (N=50): The naval battle of Coronel took place in the Pacific Ocean.
In the large majority (#45% of other cases, the LLM is uncertain about its answer, but usually just split between or three answer options. In this case, the answer remains relatively brief, for example:
When used by John Major in a speech of October 1993, which three word term was misinterpreted as re-assertion of traditional moral values? Sample & Summarize (N=50); "Back to Basics" is the most likely term, though it could also be "classless society.".
Question:
Then there are the high- vighanariance cases where the LLM answers (or hallucinates) many different options. This happens for about 5% of the questions, and we provide two examples below:
Question: What is the highest award in the Boys Brigade?
Sample Summarize (N=50): The highest award the Boys Brigade is most likely the Queen's Badge, but it could also be the King's Scout Award or the President's ' Badge. Other possibilities include the Medal of the Order of St. George, the Diamond Efficiency Medal, the Duke of Edinburgh's Gold Award, the King's Badge, the Jubilee Badge, the Diamond Award, the Chief's Medal, and the BB Company Award.
Which company announced in early 2002 that it would be transferring production from its factory in Malmesbury, Wiltshire to Malaysia?
Question:
Sample& & Summarize (N=50): The company that announced the transfer of production from its factory in Malmesbury, Wiltshire to Malaysia in early 2002 was most likely Smiths Aerospace. It could also be British Aerospace, which later became part of BAE Systems, Dyson. Other possibilities mentioned include CORGI, Swindon Powertrains, Psion, and GKN.
Sample & Summarize method leads naive listing behavior in those high-variance cases. We also see this effect when we increase the answers that Sample & Summarize samples up to N = 100 in Table 19 The length of the average summary string grows, driven by this subset of questions, but seems to converge. Likewise, the SelfReflect score does not reward longer and longer lists. Instead, it flattens out since the additional examples carry no more information.
It is an interesting question what other answers would faithfully reflect the uncertainty especially in these high- high-variance listing cases. One could for example give an answer that roughly outlines the distribution ("It was some larger British technology firm, but I am not sure which.") rather than giving exact options. We do not want to impose this way over another our paper, we take a technical perspective: Whatever the answer may look like, it is supposed to give the same information to the user (or a judge LLM) that we would get if we sampled the LLM multiple times. This is the criterion that the SelfReflect score measures (by this, it also penalizes if there is a list of unrelated options, because they would lower the likelihood of the actually relevant options, and thus move away from faithful representation of the distribution, see the examples in the next question). We propose SelfReflect to give this measurement tool to the community in the future search for different ways to communicate uncertainties to the user.
52

---

Published as a conference paper at ICLR 2026
Table 20: SelfReflect score of self-critique approach based on initial summary from Sample summarize (N=10) (x10-3 for readability). The results in subscript font are relative to Sample & summarize (N=10).
Model
Sample & summarize N = =0
Self-Critique N 10
Qwen2.5 0.5B Instruct
Yang al.
2024a
96
97
Qwen2.5 1.5B Instruct
Yang elal
2124a
87
87
Qwen2.5 3B Instruct
Yang el al.
2024a
91
92
Qwen2.5 7B Instruct
Tang al.
20248
Qwen2.5 14B Instruct
Yang
85-1
Qwen2.5 32B Instruct
Yang
91
91
Qwen2.5 72B Instruct Yang al.
20244
85
84
Phi [Ahdin et all..22000 2024
85
84-1
Ministral 8B Instruct 2410 (Tinn al.
2024
101
100-1
Llama 70B Instruct (Mcia A1 2024
87-u
Llama 70B Instruct (Meta 2024h
88-1
Llama Scout 17B 166 Instrucl [Mcla A1
2025
88
86
Gemma 3 IB Instruct
(Gemma Team
2025
117
117
Gemma 3 4B Instruct
(Gemma Team or al. 2025
101
102-1
Gemma 3 12B Instruct Cremma Team et al
2025
102
99-3
Gemma 3 Instruct (Gemma Team ct al
2025
95-2
O RESULTS WITH SELF-CRITIQUE
One additional approach tried to improve the performance of summaries self-critique, where the model first generates a summary based on one of the other strategies and is then given the chance critique and improve its summary. In Table 20 we show the results for applying it the Sample & summarize (N=10) strategy, where we give the model access to 10 further samples in the second step of critiquing its original summary. As see, the self-critique approach does not consistently improve the SelfReflect score. However, some improvements can be seen for the larger Gemma models, making this potentially interesting approach to pursue further in future work, e.g. in an iterative fashion.
53


Reviews:
[{"review_id": "qu5Q5IWZhJ", "reviewer": "Reviewer_j1CR", "summary": "**Summary**: This paper investigates the problem of communicating uncertainty of the distribution over answers to a question, advocating for the generation of all possible answers and their likelihood when generating an answer to a question (instead of generating the most likely answer and a confidence score). It then proceeds with (1) proposing and validating a metric–**SelfReflect**– to quantify the faithfulness of a summary to a LLMs internal distribution over answer, (2) empirically investigating whether through various prompting techniques LLMs are able to produce such descriptive summaries of their internal distribution.\n\n**Main contributions**:\n\n- Novel paradigm for uncertainty quantification, requiring an answer that is descriptive of the LLM’s internal uncertainty.\n- Proposal of a theoretically grounded metric, SelfReflect, that measures faithfulness between a descriptive answer (dubbed *summary*) and the LLM’s internal distribution over answers. \n- Extensive experimentation with 16 popular LLMs across 3 datasets to assess their abilities in generating descriptive summaries of their internal uncertainty either through prompting, finding them to be inapt.", "questions": "**Questions**: \n\n1. If I understand correctly, according to _Section 3.1 Summaries as Predictive Sufficient Statistics_, an ideal summary should capture the relative frequency and diversity of the answers. This aligns well with the example in Figure 1 for which the answer distribution concerns confident-sounding answers. However, I’m curious to understand how the definition of an _ideal summary_ would account for cases  where some of the answers have uncertainty in themselves, e.g., “It’s likely to be Paris” or “It’s possible that the answer is Toulouse”?\n2. The paper mentions the computational requirements of the SelfReflect metric in lines 203-208 and convergence of the SelfReflect score under various computational budgets in Appendix B (Figures 6-10). This convergence analysis is based on the SelfReflect score itself. However, I was interested in knowing how exactly Self-Reflect compares to other baselines such as LLM-as-a-judge in some controlled experiment similar to study 1 in Section 4.1. Can the authors comment on whether there’s a minimum number of samples to ensure the faithfulness of SelfReflect (i.e., if we have less than N sampling budget, we’re better off using other baseline metric?)\n3. Concerning the methodology, I’d like to clarify how is the b sentence selected? For example, in Figure 3, is b selected as the most frequent generation, or does SelfReflect eventually iterate over different sampled sequences $a^{(i)}$?\n4. Line 293 mentions “fine-grained quality differences” when discussing the superior behavior of the SelfReflect metric on the Good vs Almost Good setting. I could not find the details of how almost good summaries are generated. How do you ensure that these summaries differ by only fine-grained differences and what kind of differences are these? Was there any human validation performed to ensure this is the case?\n5. Line 298 states “matched only twice by its own SR-P(True) ablation”. Should it be “three times” instead of “twice”, since performance values between the SelfReflect and SR-P(True) ablation are also overlapping in the “Verbalized vs or-concatenated” setting?\n6. Could the authors motivate the selection of different models across controlled experiments in Section 4?\n7. Section 5 includes experiments with numerous prompting and fine-tuning strategies. I was wondering if an iterative self-critiquing approach would improve upon simple prompting or “sample and summarize”?\n8. In Section 5.2 the authors suggest that despite fine-tuning and providing examples of descriptive summaries to the models, they seem to be unable to perform the task and that “model memorizes individual summaries rather than learning a general mechanism”. Could this be due to data quality issues (since the data is generated synthetically) and/or hyperparameter tuning?\n\n\n\n\n**Writing style**:\n\n- Line 40: More of a subjective preference but expressions “We believe we can do better than this.” seem to be unnecessary and a bit informal for a conference style paper. I’d suggest replacing it with something more formal.\n- Line 351: Please provide a citation for the Krippendorf’s alpha. This can be for a particular implementation or a relevant paper.\n\n\n** Clarity**:\n- The experimental design for study 2 is unclear for me. Here are a few terms and expressions that were ambiguous: \n     - line 314: “true ratio of answers”: what is the true ratio of answers computed over? Is it over the whole test set or based on the normalized logits for each question?\n     - line 319: “compute the true Wasserstein distance between the distribution described in the summary and that of the test-set answers” → Could you clarify what the test-set answers distribution is in the context of a MMLU and how that is computed in a per question vs whole dataset basis? Why is the reference-metric a reference metric? What’s the intuition behind it?\n\n**Typo**:\n- Line 374: “decoding” → “decoding pass”\n- Line 448: “as example case” → “As an example”\n- Line 378: “Developing” → “developing”\n- Table 2 caption: “Mean $\\pm 95\\%$.” → “Mean $\\pm 95\\%$ CI.”", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- Novel contributions for uncertainty quantification in LLMs, offering a fresh perspective on uncertainty quantification and designing new set of metrics and evaluations;\n- Proposal of a new metric that is grounded in information theory concepts, such as mutual information.\n- Experimental design allows for the controlled assessment of the capabilities of the proposed metric, comparing it against numerous baselines including LLM-as-a-judge approaches.\n- Benchmark 10+ models in 3+ datasets (TriviaQA, NaturalQuestions, SimpleQA, MMLU) and 5 different prompting strategies and 2 fine-tuning approaches to assess model’s abilities in generating descriptive summaries.\n- Various appendices supporting various analyses and arguments in the paper – e.g., providing a convergence analysis of SelfReflect metric.\n- Interesting findings, including the model’s inability (even after SFT or DPO) to provide descriptive summaries of its own internal uncertainty.\n- Well-written and well-organized.", "weaknesses": "W1. **Controlled experiments in section 4 concern different models**, raising questions about the generalization of the results (see Questions). \nW2. Some sections are a bit confusing or not clearly explained (see Questions for details)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "hrT5Qxogun", "reviewer": "Reviewer_2DwF", "summary": "The paper introduces a new setting of uncertainty estimation where the model is required to provide a comprehensive summary of all of its possible answers. \nIn order to measure the quality of the output with respect to the summary, a SelfReflect metric is introduced, which uses sufficient statistics arguments to measure the summarisation quality of the provided answer given a variety of answers sampled from a model.\nThe authors further investigate the quality of the resulting metric on syntetic data.\nThe metric is then used to determine the extent to which the model's 'internal knowledge' is expressed in its outputs for multiple models and decoding strategies.\nThe authors anticipate that their findings would improve the ability to communicate the model's uncertainty to the end user as well as help several other uncertainty related tasks in NLG.", "questions": "1. Would masking on sentence level be a valid approximation of SelfReflect?\n2. Could there be an alternative to using Wasserstein distance?\n3. As this work largely addresses on fact recollection by the model, what impact could introducing RAG have on the performances presented?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is clearly written, experiments are reasonably documented in the appendix. \n2. Theoretical Justficiation of SelfReflect metric appears to be sound, the proofs with sufficient statistics provided in the Appendix A are convincing.\n3. Visible good effort to connect the theoretical propositions to experimental evaluation. Authors provide ablations of the SelfReflect metric as well as consider a broad range of LM decoding paradigms, including reasoning. \n4. Human feedback studies improve the 'value' of the paper.", "weaknesses": "1. Minor (little to no impact on my score):\n    1. Antropomorphising the langauge models: while subjective and stylistic, I view it as a minor negative aspect. I.e. line 077: \"its internal beliefs\", line 82: \"making LLMs aware of their internal uncertainties\", Line 478: \"make LLMs honestly describe\", etc.\n    2. Line 15: \"all options it deems possible\" - for a language model all options (i.e. every combinatoric token sequence) are technically \"possible\" unless -inf is allowed in the logits somehow. A strange way to put it unless some thresholding is in place.\n    3. Line 483: \"Extracting all output possibilities is also a core necessity for conformal approaches\", not sure if this is factually accurate considering that what is meant by all possibilities here is extracting them as a single output string.\n2. Major:\n    1. Emprical benefits are unclear. The Tab. 1 presents comparison between several metrics as well as ablations of the SelfReflect. It can be seen that in Good vs Bad summaries the LM Judge is only second to the unablated SelfReflect, and even then 0.01% away from not achieving 95% significance. Tab.1 could be presented as a heatmap as well, depicting the performance for all combinations of summaries incorporating *all pairs* for a more wholesome picture. \n    2. Some experiments may have unanticipated caveats:\n        1. The data for the Tab. 1 was reportedly created by using Qwen2.5 7B for answer generation and Gemini 2.0 Flash for summary creation. Given the tiny margins for the main Good vs Bad summaries experiment, one may want to ablate this by answers and summaries coming from different models / generated by other means. The almost-good summary generation appears to be even less robust, since it is created starting from an already model generated summary. The other pairings also seem somewhat arbitrary. \n        2. Fig 4: The certainty / uncertainty of the answers / summaries is ill defined. It therefore hard to pass judgement based on that.\n        3. Tables 4 and 5: Reasoning model output lengths appears to be strange: it is hardly twice the length of normal outputs. RLVR tuned models from Tab.5 are not trained for summarization and fact retrieval, but for facing math and logic problems. \n    3. Although I do not have an in-depth familiarity with state of the art summarization literature, the paper fails to compare SelfReflect to summarization specific metrics. For example something on the lines of [1] (Which is also a PMI of sorts). \n    4. The second part of the contribution, the analysis on the expression of internal knowledge via different decoding schemes lacks clear placement: it is unclear how exactly we can benefit from this knowledge. The guidance in the Outlook is inspecific. \n\n### References:\n1. Jung, J. et al. Information-Theoretic Distillation for Reference-less Summarization. Preprint at https://doi.org/10.48550/arXiv.2403.13780 (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZuKLSD9pAG", "reviewer": "Reviewer_2owx", "summary": "The authors introduce SelfReflect, an information-theoretic metric that measures how well a summary string matches an LLM’s true internal answer distribution. The SelfReflect metric measures how faithfully such a summary represents the model’s true internal answer distribution using masked-token prediction and Wasserstein distance. Extensive experiments show that current LLMs cannot genuinely articulate their internal uncertainties.", "questions": "Please check the Weaknesses. I'm not quite familiar with the background knowledge of this paper, so I'm willing to modify my rating based on the comments from other reviewers and ACs.", "limitations": "", "rating": 4, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The experimental results in Section 4 including Distinguishing Good, Bad, and Almost-Good Summaries, Multiple-choice QA and Alignment with Human Judgments are solid and sufficient.\n\n2. The explanation of why CoT fail on reflecting the LLM’s internal confidence is very interesting.", "weaknesses": "1. To be honest, I am not quite familiar with this area but would like to ask a very general question. Why do you think the topic, whether a LLM can accurately express its own confidence through natural language, is important?\n\n2. Based on 1, do we even have a promising metric to reflect LLM’s internal probability distribution yet? If not, the topic of investigating whether a LLM can accurately express its internal probability distribution may not be reliable.\n\n3. For Eq2, I am quite confused why the left hand side is equivalent to the right hand side? Why is the conditional probability measured by mutual information?\n\n4. Could you please explain why we should use 1-Wasserstein distance to measure the difference between distributions instead of JS/KL divergence?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eHT3YLshPh", "reviewer": "Reviewer_zDzS", "summary": "The authors introduce SelfReflect, a novel metric to measure the faithfulness of LLMs expressing their uncertainty about answers. Specifically, they examine a certain type of expression of uncertainty, which entails an explicit natural language \"description\" of a distribution. The SelfReflect metric is a method to measure the consistency between such a natural language \"distribution summary\" and the real output distribution of the LLM. They do this by alternating putting the distribution summary or some real samples of the distribution in the context and seeing how the distribution of a masked word changes in these two scenarios. The logic: if conditioning on the distribution summary and the \"real distribution\" (represented by a set of samples) leads to similar masked word probabilities, then they contain the same information. **This point initially confused me: there is the judge LLM (which is giving the masked word probabilities) and then there is the LLM being measured (which gives the distribution summary and real distribution). These are not the same LLM.**\n\nThe findings are that the metric is generally more effective than the baselines they show, especially when it comes to nuanced judgements (judging distribution summaries that are subtly wrong in small ways). They also validate that it correlates well with human judgements.\n\nHaving established the effectiveness of the metric, they then proceed to measure various LLMs via this metric to see if they can generate effective distribution summaries in a single inference call, and conclude that *they cannot*. Neither simple prompting, chain-of-thought reasoning, reasoning models (like DeepSeek-R1), nor finetuning successfully get the LLMs to generate faithful summaries. They state that LLMs are effective at writing good distribution summary strings given samples from the distribution (the \"Sample & Summarize\" approach), but not at actually \"introspecting\" and generating the distribution summaries based on their own internal state.", "questions": "I would want to see an Appendix with some qualitative analysis of some examples from the N=50 case. What do these summaries look like?\n\nI would want to see some discussion of if these summaries are actually helpful for an end-user. There is no rigorous discussion of whether distribution summaries in natural language are actually useful, and what the benefit is compared to other ways to summarize the distribution of answers from an LLM.\n\nCan an analysis of summary length vs N be done?\n\nTo be honest it feels like especially the sample&summarize method is effectively a form of implicit clustering. Like what I must infer is that the summary is essentially grouping different samples together and assigning some probabilities based on the frequency of certain samples over others. This is explicitly what you're doing in Appendix G it seems? Why is the connection to clustering not made explicit in the paper?\n\n\"The negative result shows LLMs can't introspect\" why do you think this might be? Is it a training data issue, architectural limitation, or something about how RLHF works? Do you have hypotheses for what would enable introspection?\"\n\nHow does your Sample & Summarize approach compare to existing methods like Semantic Entropy (Farquhar et al. 2024), which also clusters sampled answers? What's the advantage of natural language output over numerical entropy?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "I think the problem being presented (existing methods like providing a single number representing the certainty of one answer not being rich enough to understand the LLMs internal state) is interesting and well-motivated. \n\nThe metric is validated extensively. I especially appreciate the human study.\n\nThe theoretical connection to predictive sufficiency is compelling, the proofs look rigorous.\n\nThe authors do seem to have tested the negative result extensively, trying many different approaches to have the models be able to introspect. They establish that \"model introspection\" is an open and very difficult problem.\n\nResults are comprehensive and reproducible, writing quality in the paper is clear and comprehensive.", "weaknesses": "I am not sure that the authors effectively justify why a natural language summary of the answer distribution is better than just teaching the model to generate structured output representing is answer distribution: something like [(\"Paris\", 0.75), (\"Marseille\", 0.15), (\"Toulouse\", 0.10)].\n\nThe example the authors provide in their diagram (Figure 3) is interesting, but it's not at all clear to me what happens when you extrapolate to N=50. What do the summary strings actually look like? Is it an enumeration of 30 different answers in e.g. high variance cases? Is this a useful thing to look at as an end-user? It would be interesting to incorporate some notion of \"utility\" for an end-user.\n\nWhat is the \"compression strategy\" the LLMs are using to generate these distribution summaries? I'm assuming some answers get tossed. Which ones, and why, and are we losing important distributional information this way? It's not clear to me. Like I understand your metric hypothetically should avoid losing important distributional information. But I still really would like to see what the final distribution summaries look like at this level. Appendix E is somewhat informative but not sufficient: it demonstrates the metric, but not what the actual distribution summaries generated from the LLMs look like. \n\nI think a qualitative analysis of some examples of the summaries generated would be extremely useful.\n\nThe paper is extremely dense and long, with the appendices. Some important details are buried in the appendices, it's so long I may have missed other details, not sure.\n\nI think the paper may be overreaching in its claims, specifically regarding the second contribution: examining if LLMs are capable of expressing distributional summaries without access to samples from the distribution, and if they can be trained to do so. \"With SelfReflect, we make a resounding negative observation: modern LLMs are, across the board, incapable of revealing what they are uncertain about, neither through reasoning, nor chains-of-thoughts, nor explicit finetuning.\" -- I think this is slight overclaiming, they are not able to estimate their answer distribution, which is one specific way they can make the user aware of their uncertainty.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)