How Stable is the Next Token? A Geometric View of LLM Prediction Stability
Authors: Deyuan Liu
Zecheng Wang
Zhanyue Qin
Zhiying Tu
Dianhui Chu
Dianbo Sui
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Model
Post Training
Primary area: foundation or frontier models, including LLMs

Abstract: Large Language Models (LLMs) exhibit impressive capabilities yet suffer from sensitivity to slight input context variations, hampering reliability. Conventional metrics like accuracy and perplexity fail to assess local prediction robustness, as normalized output probabilities can obscure the underlying resilience of an LLM's internal state to perturbations. We introduce the Token Constraint Bound ($\delta_{\mathrm{TCB}}$), a novel metric that quantifies the maximum internal state perturbation an LLM can withstand before its dominant next-token prediction significantly changes. Intrinsically linked to output embedding space geometry, $\delta_{\mathrm{TCB}}$ provides insights into the stability of the model's internal predictive commitment. Our experiments show $\delta_{\mathrm{TCB}}$ correlates with effective prompt engineering and uncovers critical prediction instabilities missed by perplexity during in-context learning and text generation. $\delta_{\mathrm{TCB}}$ offers a principled, complementary approach to analyze and potentially improve the contextual stability of LLM predictions.

Published as a conference paper at ICLR 2026
How STABLE IS THE NEXT TOKEN? A GEOMETRIC VIEW OF LLM PREDICTION STABILITY
Deyuan Liu' Zecheng Wang!. Zhanyue Qin' Harbin Institute of Technology
Zhiying Tu'
Dianhui Chu
Dianbo Sui!
Wechat AI
ABSTRACT
Large Language Models (LLMs) exhibit impressive capabilities yet suffer from sensitivity to slight input context variations, hampering reliability. Conventional metrics like accuracy and perplexity fail to assess local prediction robustness, as normalized output probabilities can obscure the underlying resilience of an LLM's internal state to perturbations. We introduce the Token Constraint Bound (ŌTCB), a novel metric that quantifies the maximum internal state perturbation LLM can withstand before its dominant next-token prediction significantly changes. Intrinsically linked to output embedding space geometry, STCB provides insights into the stability of the model's s internal predictive commitment. Our ex- periments show êTCB correlates with effective prompt engineering and uncovers critical prediction instabilities missed by perplexity during in-context learning and text generation. θTCB offers principled, complementary approach to analyze and potentially improve the contextual stability LLM predictions.
INTRODUCTION
Large Language Models (LLMs), such as GPT-4 (OpenAl et al., 2023), LLaMA (Touvron et al., 2023: Dubey et al., 2024) and Gemini (Team et al., 2023), demonstrate remarkable capabilities, yet paradoxically exhibit striking sensitivity to contextual nuances. This brittleness manifests as substan- tial performance variations due subtle modifications: accuracy can fluctuate by up to 76% from minor formatting changes (Sclar et al., 2023) or range from 54% to 93% based on example order (Zhao et al., 2021). Such variations stem from alterations in prompt phrasing (Razavi al., 2025), example selection and ordering (Lu et al., 2021), or even basic formatting. Despite established scal- ing laws (Kaplan et al., 2020: Hoffmann et al., 2022) fueling impressive in-context learning (Brown et al., 2020; Dong et al., 2022; Wei et al., 2023), evidence indicates that increased model scale does not inherently confer enhanced robustness; larger models may even exhibit new sensitivities (Lu et al., 2021; Wei et al., 2023). This underscores the urgent need for robust stability metrics in mod- emn AI evaluation, particularly for reliable deployment in mission-critical applications demanding consistent performance (Weidinger al., 2021; Herrera-Poyaton et al., 2025).
Appraising contextual influence with precision is imperative, yet existing evaluation frameworks prove inadequate. Task accuracy yields only an aggregate performance view, overlooking the stabil- ity of individual predictions amid contextual shifts. Perplexity (Jelinek et al., 1977), though standard for sequence likelihood (Liang et al., 2022; Holtzman et al., 2021), conflates probabilities, thereby obscuring local dynamics essential for robustness. Moreover, it often neglects internal state geome- try and fails to ensure internal stability even for high-probability tokens (Cohen-Inge) et al., 2025). Crucially, the softmax normalization applied derive output probabilities can mask prediction's underlying stability; high probability can arise from relative normalization, not necessarily from robust internal state. This implies that a high token probability offers no guarantee that the originat- ing internal state h is itself resilient to minor variations. Even as emerging metrics (Zhang et al., 2024: Tian et al., 2023; Geng et al., 2023) address confidence and calibrationchiefly by aligning probabilities with correctness likelihood (Tian et al., 2023)they do not directly gauge the robustness of a specific next-token prediction's 's dominant rank to perturbations in the internal representation h. A well-calibrated, high-confidence prediction may therefore belie unstable equilibrium within the internal state (Liu et al., 2025). This gap in assessing the immediate predictive mechanism's stability against internal perturbations is the direct impetus for central research question:
Corresponding author.

---

Published as a conference paper at ICLR 2026
Prompt: "Answer Yes or No: Is 9.11 greater than 9.97
Prompt: Is 9.11 > 9.9
Dutput Ve
Output V
0.9
0.6
0.3
bren(h)
PC Component 2
0.0
0.0
LLM
Predicts: "No"
[|óva]| 0.089 (Stable)
Predicts:
Final State
Predicts: "Yes"
lbvull
1.403 (Changed!)
Output Vg
"unsafe"
dycB
Boundary Dist (5 (.444)
0.6
(Original)
[inzicie)
ht (Outside)
0.0
Yes
X No
PCA Component
Figure 1: The Token Constraint Bound (ŌTCB) mechanism. ôTCB quantifies the maximum perturbation
model's internal can withstand before the next-token prediction changes. Left panel illustrates how hidden state perturhation Ah impacts the next token prediction. Small perturhations (Ah1, implicitly within δTCB radius) may preserve the output, while larger ones (Ah2 >> òTCB) can flip it (from "No" to "Yes"). ). бTCB bounds the perturbation size for stable output. (b) Right panel shows that the original hidden state h and a perturbed state h' inside stability region predict "No". Another perturbation h" outside the region flips the prediction to "Yes". demonstrating the practical consequence of exceeding the stability boundary.
Q: How can quantify the stability of LLM's immediate prediction state, induced by a specific prompt or context, against small internal variations?
Addressing this question necessitates transcending aggregate performance metrics to develop mea- sures specifically targeting the local robustness of prediction mechanisms. We must quantify how susceptible the next-token output distribution is perturbations in the internal representation gener- ated from the input contexta challenge at the intersection of representation stability and prediction reliability.
Our approach. We propose the Token Constraint Bound (ŌTCB), measure of this critical lo- cal stability. θTCB quantifies a "safety margin" around the internal state h resulting from context processing: larger orcn means the models next-token prediction (particularly its top choice) with- stands greater internal perturbations Ah without significant change. It gauges the model's commit- ment to its current output ranking, given h. As explicated in Section and depicted in Figure 1, ôTCB offers direct measure of the output layer's robustness to hidden state variations. Therefore, a high бTGB signals stably confident prediction state engendered by effective context.
We hypothesize that effective context, such as well-crafted prompts or informative ICL examples, not only guides models to correct answers but also induces more stable internal state h, as reflected by higher бTCB values. This stability, signifying robust internal commitment to a predictive path, yields more reliable predictions. Consequently, бTCB offers quantitative measure for context effec- tiveness beyond accuracy, serving as a proxy for the robustness of context-derived decision-making and complementing uncertainty metrics focused on "knowledge strength" (Ma et al., 2025).
Our experiments corroborate this. We show δTCB distinguishes prompt quality and exhibits distinct behaviors across confidence regimes, correlating with distributional flatness in uncertain cases and logit margins in high- high-confidence scenarios. Results confirm òTCB 's sensitivity output embed- ding geometry, its link to semantic content, and its ability to flag incipient instability during text generationdynamica perplexity overlooks.
Our contributions are threefold:
(a) We introduce and theoretically ground the Token Constraint Bound (ŌTCn), a novel metric that measures the local robustness of an LLM's next-token prediction to internal state perturbations, and detail its practical computation (Section 2).
(b) We derive an expression that intrinsically links êTCB to the geometric dispersion of output em- beddings, thus identifying geometric underpinnings of prediction stability ( (Section 3). (c) Through empirical evaluation, we demonstrate êTCB's 's capacity to assess prompt effectiveness and showcase its application in refining both prompt engineering and ICL (Section 4).
2

---

Published as a conference paper at ICLR 2026
High conkieeees
Laver FOREER
Arr--hop Reaporo Yau No.
He FINE
32 6. does 9.13 230 bor Wix
ppeficentygmee proate than 5.07 Arswer
ALL numaroon city grewer du
Veen 2.53
3.938
CuApo Disc Bustion
1.086
1.998
LLowe
65.1.89
0.680
Cont)
Outpurt Dietmiuttoo
Outpun Diemibution
3.48
Outp vs
Output is
Webasalty
1.0
0.8
0.97
Probassity
0.8
0.73
Rg IIC (Tess)
0.15
0.5
0.27
3- no dareeee weeeet hel hos
by iZoro-hott
0.2
0.03
All t.
the Uoweeet scrnect Respare ths
sumoroall yrunce
0.0
Effect of prompt confidence ÓTCB-
(b) Effect of In-Context Learnnng on &TCB-
Figure 2: êTCB reflects context-induced prediction stability. (a) Illustrates how prompts inducing higher prediction confidence (lower Ven, state h2)) lead to significantly larger бTCR compared to prompts yielding lower confidence (higher V.ar. state fuz). (b) Shows how In-Context Learning examples modify the hidden state and consequently the prediction and its stability. Adding examples can initially decrease stability while flipping the prediction, but consistent examples can increase stability for the target output.
2 PRELIMINARIES: UNDERSTANDING LLM PREDICTIONS AND STABILITY
This section establishes the foundational concepts for analyzing local output stability in LLMs. We begin by outlining the LLM output mechanism, then explore what it means for a prediction to be stable against internal variations. This leads the introduction of the Jacobian matrix as a tool for quantifying sensitivity, and finally culminates in the formal definition of our ÒTCB-
2.1 LANGUAGE MODEL OUTPUT AND DISTRIBUTION CONCENTRATION
Consider LLM whose final layer computes hidden state h E Rd. This state is linearly trans- formed by an output weight matrix W E RVxd to produce logits x = Wh. where V is the vocab- ulary size. Each row wi of W corresponds to the output embedding for token i. The probability distribution over the next token, e RR, is obtained via the softmax function:
cxp(2:)
softmax(z). where o, Cy=1exp(e
This distribution satisfies Σ 0 = and 0; 2 0. The model's prediction is typically the token i* maximizing Og. A useful measure of the concentration of this distribution is the effective vocabulary size Vest:
1
Vott (o)
E1 of
Van ranges from I to V. inversely relating to the L2 norm squared of the probability vector. Our analysis hinges on understanding how the characteristics of this output vector o, including its con- centration, relate to its stability when the context-derived hidden state h undergoes small changes.
2.2 DEFINING STABILITY: WHAT DOES IT MEAN FOR A PREDICTION TO BE STABLE?
Our core objective is understand the robustness of an LLM's next-token prediction. Specifically, we want to know: if the LLM's 's internal summary of the context (represented by the final hidden state h) changes slightly, how much does its next-token probability distribution change? Let Ah E R represent a small internal "wobble" or perturbation to the hidden state h. Such a perturbation to the model's 's internal representation of the context could arise from minor input variations, noise in the computation, other subtle disturbances. Let o' softmax(W(h- Ah)) be the perturbed output distribution. The resulting change in the prediction is Ao = o' - o. The central question motivating "safety margin" for h: how large can the perturbation Ah be before the change in the output Åo becomes unacceptably large? Answering this requires way to relate the magnitude of the internal
work, reiterated from the Introduction Q, is how to quantify
3

---

Published as a conference paper at ICLR 2026
perturbation Ah the magnitude of the resulting output change Åo. This safety margin offers insights distinct from interpreting output probabilities o as direct measures of absolute confidence; instead, ÔTCB focuses the local integrity and resilience the current predictive mechanism itself.
2.3 QUANTIFYING THE IMPACT OF PERTURBATIONS: THE ROLE OF THE JACOBIAN
To precisely relate changes in h to changes in o, utilize the concept of the Jacobian matrix. To first order, for small Ah, the change in the output distribution Ao can be approximated linearly:
Ao = (h)ahh
where Jw whh E RRd is the Jacobian matrix of the output probabilities o with respect to the hidden state h. is given by:
do
do
Oz
Jw (h) 0h
(diag(o) - -oo) W.
(4)
0z
Jh
diag(o)-oo'
W
The Jacobian Jw(h) (h) essentially captures the sensitivity of each output probability o, to infinitesimal changes in each dimension of the hidden state hg- Its entries de, Note that the Jacobian depends both the current output distribution and the output weight matrix W. To relate the
overall magnitude of the state perturbation ||A/||2 to the overall magnitude of the output change
||ol|2, use matrix norms. A standard inequality bounds the output change using the Jacobian's Frobenius norm:
||oo|22 S
J[Jw (h)l( Ah|l2-
The Frobenius norm [Jw(h))lr (Ei (E'
provides comprehensive, aggregate measure of the sensitivity of all output probabilities all hidden state dimensions. A larger Frobe- nius norm indicates that, the output probabilities are more sensitive to changes the hidden state.
1/2
2.4 THE TOKEN CONSTRAINT BOUND (δTCB): OUR MEASURE OF STABILITY
We are interested finding the maximum allowable perturbation radius ||hh|2 such that the re- sulting change in the output distribution, measured by its L2 norm | remains below
predefined small tolerance > 0. That is, we impose the condition || la S e. Using the bound from Eq. (5), we require [|Jw(h)]l ||
Rearranging for || l2 gives
e
A 1|2 <
(6)
[Jw(h)]l
This naturally motivates bound on the perturbation norm:
metric for local output stability, which define this upper
Definition 1 (Token Constraint Bound εTCB). Given the output weight matrix W. hidden state resulting output distribution o softmax( Wh), and a tolerance >> 0 for the maximum L2 change allowed in the Token Constraint Bound 8TCB at state h is defined as:
STCB(h) = (h
(7)
Here, Jw(h) is the Jacobian given by Eq. (4) and I| || denotes the Frobenius norm.
The parameter dimensionless scale factor chosen by the user, representing the desired tolerance for output distribution change. Consequently, θTCB (h) quantifies the L2-norm radius of the largest hyper-sphere of perturbations Ah around the current hidden state that, to a first-order approxima- tion, guarantees the change in the output probability vector remains within e. A larger êTCB (h) significs that the model's current prediction state o, as induced by the context leading to h, is in- trinsically more robust to small internal variations ("wobbles") in this hidden state. Conversely, smaller 8TcB (h) indicates that the prediction mechanism is more sensitive to such internal pertur- bations at this specific point. The erucial term AJw(h)]F in the denominator has fundamental connection the geometry of the output embeddings, which explore in detail in Section 3.
4

---

Published as a conference paper at ICLR 2026
3 ÔTCB VIA OUTPUT EMBEDDING GEOMETRY
Def. 1 introduced the Token Constraint Bound (ÓTCB) as a measure of local output stability. To unlock its full diagnostic power and understand its nuanced behavior, we now dissect its core compo- nent: the Frobenius norm of the output Jacobian, [[J (h)llr. This section reveals that J[Jw (h)llr, and consequently STCB, is deeply intertwined with the geometric arrangement of the model's output token embeddings wo relative to the current prediction probabilities o. Intuitively, a prediction is expected to be more stable if the leading token's embedding is well-isolated from competitors, or the model is highly certain (peaked o).
3.1 THE GEOMETRY OF OUTPUT: TOKEN EMBEDDINGS AND THEIR MEAN
Recall that the output weight matrix W E R contains the output embedding vector wp for each token i as its rows. A key concept in understanding the geometric influence on stability is the probability mean embedding vector:
Ho (h)
Σ 0,0 - TT j=1
W
Here, (h) E RR represents the current bebbability-eeighhee locus within the embedding space. This mean vector, (h), reflects the current grobability-wiighhe locus within the embedding space, effectively representing the "center of mass" or the resultant directional influence of the entire output distribution on the embedding geometry.
3.2 DERIVING THE JACOBIAN NORM: CONNECTING SENSITIVITY TO EMBEDDING SPREAD
We now derive an exact analytical expression for the squared Frobenius norm of the output Jacobian, [ (h) (h)ll liè which is the crucial term determining STCB. Let {w, be the output embedding vectors (rows of W) and ww (h) be the prohahelity-weightedd mean embedding as defined in Eq. (8).
Proposition (Exact Squared Jacobian Norm Appendix I). For a given output weight matrix W and hidden state let o softmax( Wh) b) be the output probability vector. The squared Frobenius norm of the output Jacobian Jw (h) = (diag(o) oo W is exactly:
JJw(h)l Σ of l|w, Mm (h)
(9)
This sum represents the squared Euclidean distances between each embedding and the mean embedding Ma (h),, weighted by the corresponding squared probability o².
Interpretation of the Formula. Eq. (9) is pivotal. It states that the overall sensitivity of the output distribution to hidden state perturbations (as captured by [|J (h)]l) is determined by how "spread out" the token embeddings w; are from their grobability-weighte mean pm, (h), with each squared distance lww Mw (h) being amplified or diminished by the square of its token's probability of
The of weighting is crucial:
Embeddings for tokens with very low probability o; (and thus low current ""evidence" or ""belief" from the model) contribute minimally to the sum, even geometrically distant from pur. The model effectively de- de-weights their geometric influence on stability at this state.
Embeddings for gigh--robability tokens (carrying significant ""evidence") contribute substantially, particularly if they are far from Mww Their geometric influence on the Jacobian norm is quadrati- cally emphasized by oz.
This of weighting distinguishes Eq. (9) from measures like the trace of the standard probability- weighted covariance matrix of embeddings. This distinction arises directly from the definition of the softmax Jacobian and is fundamental for correctly interpreting òTCB (see Appendix G).
3.3 THE FULL FORM OF δTcB AND ITS GEOMETRIC MEANING
Substituting the exact squared Jacobian norm from Prop. I Eq. (9) into the definition of ÔTCB Eq. (7) yields its complete form:
θTCB (h)
(10)
V EV_ of l|w. phw (h) 2
5

---

Published as a conference paper at ICLR 2026
Table 1: Pearson Correlations validating regime-dependent stability. Shows strong positive Corr(rreB, Vea) (r=0.95) in diverse prompts (DPD, N = 309, hroad regime), indicating stability driven by flatness. In contrast, strong positive Corr(rCB, 2x 2y-) (r=0.62) emerges in high-confidence cases (Low- Van Targeted, LVD, .N = 360). where Com((TCH, Ven) is negligible
= 0.08), confirming stability relies on lop-token separation when confidence is high. Metrics are the Token Constraint Bound (ÓTCB), effective vocabulary size (Var), and the logit margin between the top two tokens (zx 2z-).
Dataset Samples)
Com(8reB, Veet)
Con(ôreB; - 2y*))
Corr(z+ - -Ay", Vatt)
Diverse Prompts (DPD, N 309)
0.95 (Strong +)
-0.40 (Modcrate -)
-0.41 (Moderate
Low-Vet Vett Targeted (LVD, 360)
0.08 (Near Zero)
0.62 (Strong +)
-0.60 (Strong -)
This equation provides clear geometric interpretation: êTCB is inversely proportional to the square root of the oe-weihtted sum of squared Euclidean distances between each token embedding w, and the probability weeghtte mean embedding M (h). A larger δTcB indicates higher local robustness of the prediction generated from h. This geometric dispersion, weighted by of. directly dictates the "safety radius" around h, within which the output distribution changes by most e. Understand- ing this relationship is key to interpreting how context shapes êTCB and, by extension, prediction stability, as conceptually illustrated in Figure and Figure 3.
3.4 INTERPRETING STABILITY ACROSS PREDICTION REGIMES
The exact expression for [Jw (h)ll? in Eq. (9) and the visualization in Figure elucidate how òTcB behaves under different prediction certainties.
High-Confidence
Uncertain
High-Confidence Regime (Low Vatt Peaked o). When o is highly peaked on token & (og + 1, Vea + 1), Pun (h) + wh. causing [[Jwhh]l? + and
10; H yc
W
ÒTCB + 0c ( Figure a). Here, extreme cer- tainty implies extreme stability. The sum approxi- mates to Σfft of l|w, wxll2 (Appendix
Cru- cially, competitor probabilities 0j (and thus 0²) are super- exponentially sensitive to the logit margin between the top-two candidates, 2 2z- = Zoopl zop- Larger
бTCB Large
8yCB Small
Figure 3: Output Distribution Determines Geometric Stability. The Token Constraint Bound (δTCB) is function of the geometric arrangement of embeddings. (a) High Con- fidence: Peaked distribution concentrates phw (h) near the dominant embedding was minimizing the sum and maximizing 8ren- (b) Uncertainty: Flatter distribution spreads the (h) among active embeddings, increas- ing the sum and reducing òTcB. Mw (h), to- ken embeddings w.
Fs - 2y- values drastically reduce wlJw (I) boosting бTCB- This underpins the empirical positive correlation between êTCB and žµ - 2z- when Ven is low Table 1, Low-VeR). Distances ||w, w further modulate this: more distant competitors require even smaller of for the same stability.
Uncertain Regime (Higher Vett, Flatter o). When probability is spread over multiple tokens (larger Vett.
Figure 3 b), many o; are non-negligible. If these probable tokens' embeddings w, are distant from ww (h), their l|w, - tw (1))l2 terms in- crease (h) I|; reducing δTcB. Crucially, however, a high Vor does not guarantee low òTCB: high-probability embeddings {w:} are geometrically clustered (and thus all near fw (h)), the I|w; Mw (h) ||3 terms could be small despite significant of values, potentially resulting in larger ÔTCB. This highlights geometry's primacy, validated by experiments where clustering embeddings (fixed o) increases ÒTCB. In this uncertain regime, under simplifying assumptions ( Appendix F,
Appendix )), approximations can suggest [h) x 1/Vett This implies ÒTCB & VVeit. aligning with empirical correlations over diverse prompts ( (Table 1, Diverse Prompts), where over- all distribution shape often dominates individual logit margins.
EXPERIMENTS
This section empirically substantiates the thcoretical framework for ÒTCB, focusing on its connection to output embedding geometry and its utility in LLM analysis. We address:
ôTCB's 's intrinsic properties, including sensitivity to output embedding geometry and correlations
with standard metrics (Ven = 2z-) ) across confidence regimes (Section 4.2).
The role in diagnosing accuracy-stabiliiy confficts and robust prompt engineering (Section 4.3) ). How êTCB complements Perplexity (PPL.) by assessing local prediction robustness (Section D.3

---

Published as a conference paper at ICLR 2026
4.1 EXPERIMENTAL SETUP
Models. Primary experiments utilize the LLAMA-3.I-8D model (Touvron et al., 2023). All computations were performed on NVIDIA RTX 4090 GPUs.
Datasets and Rationale. MMLU (Hendrycks et al., 2020): Employed for broad validation and initial characterization due to its diverse subject matter. We typically sampled Nimi = 100 ques- tions from "test" splits of 3-5 reasoning-heavy subjects (e.g., formal logi phi (hilosophy to assess general trends and robustness under varied content. GSM8K (Cobbe et al., 2021): Utilized for detailed intervention analysis and prompt optimization case studies. Its multi-step reasoning na- ture provides a fertile ground for examining how nuanced prompt changes affect both accuracy and internal stability. An initial pool of Ninit_ pool 100 questions from the "test" set used for broader studies, with specific problems selected for deep dives. DPD and LVD Datasets: We synthesized two datasets for correlation analysis. The Diverse Prompts Dataset (DPD) contains prompts from a range of tasks, designed to elicit varied model confidence levels. The Low-Veff Targeted Dataset (LVD) was created by modifying DPD prompts to generate high-confidence predictions.
Prompting Strategies. Shro-Shot: Minimalist prompts for baseline correlation studies and initial conflict identification. Few-Shot/ / Interventions: range of k-shot prompts (k: 5). ICL variations
(e.g., algebraic vs. arithmetic focus, hyper-specific examples), and instructional prefixes were used in diagnostic analyses and optimization experiments.
Core Metrics. At critical prediction points (e.g., just before generating the answer token for MMLU multiple-choice before the first token of the final numerical answer in GSM8K): Token Constraint Bound (6rсв(()) Computed via Eq. (10). For all experiments, we set the tolerance parameter e = 1.0, which normalizes the metric. Since our analysis focuses on relative changes in stability, the specific value of is less critical than its consisteney. Higher STCB indicates greater internal state robustness. Effective Vocabulary Size (Var(o)): From Eq. (2). Lower VeH indicates more peaked, confident distribution. Logit Margin (=k 2z-): Defined Zupp Aup2- Larger positive values suggest stronger discrimination for the top choice. Task-Specits Accuracy (Acc): Binary score based on ground truth. Perplexity (PPL): For local analysis in relation ÒTCB, we often refer to the negative log probability of the predicted token log Opredicued). Further details on experimental configurations are in Appendix C.
4.2 INTRINSIC PROPERTIES AND VALIDATION OF 8TCB
Validating Sensitivity to Output Embedding Geometry Objective: To empirically confirm ôTCB's 's direct dependence on output embedding ge- ometry (W), independent of the probability distribu- tion (o). Method Summary: We synthetically ma- nipulated W (clustering/dippersinng competitor em- beddings) while holding h and o (thus local PPL) constant for diverse MMLU prompts. θTCB was re- calculated. Results: Table shows the hypothesis śTCB W ) > STCB W ong) > STCB [ disperse) held for 90% of prompts overall, directly substanti- ating the geometric term in Eq. (10). This highlights
Table Simulation confirms ôTCB's 's geomet- ric sensitivity. Percentage of prompts validating daweerr > bong > Rassense when manipulating W (K 10 competitors) while fixing o. The effect robustly held (90% overall), confirming geomet- ric influence distinct from probability shape.
Prompt Category
Hypothesis Held
Low Vee (< 20) Medium Var (20-100)
95%
92%
High Vanr (> 100)
80%
Overall
90%
that ÒTCB captures a dimension of stability tied to the embedding space that probability-only metrics would miss.
Correlations Across Different Confidence Regimes Objective: To validate predicted shifts in òTCB'S 's correlations with VeS and 2 - 23- based on prediction confidence. Method Summary: Two MMLU zero-shot datasets: Diverse Prompts (DPD, N = 309) and Low-Vear Targeted (LVD, N = 360). Results: Table 1 confirms the theorized regime dependence. In the DPD (broad regime), Corr(ôreB, Ven) is 0.95, indicating stability is largely driven by overall distribution flatness. In stark contrast, for the LVD (high-confidencee, CoNI(STCB, Venr drops negligible 0.08, while Corr(òrcB, 2 2z-) becomes a strong 0.62. This shift empirically validates that when the model confident, 8TCB reflects the separation of the top token from its competitors rather than just the general peakedness of the distribution.
4.3 8TCB AS DIAGNOSTIC TOOL. FOR PROMPT ENGINEERING
ôTCB uniquely identifies stability issues that accuracy or conventional confidence metrics (Vett 24: zz-) may miss. Common accuracy-stability conflict scenarios include: (1) Accurate but Unstable:
7

---

Published as a conference paper at ICLR 2026
Table 3: Combined Impact of Fren-Enhancemmet on Mean Metries for Unperturbed and Perturbed Prompts
(MMLU & GSM8K), Metrics are Accuracy (Acc), Token Constraint Bound (STCB), Effective Vocabulary Size (Ver) and Logit Margin (2k 2z+ ). Perturbed metrics include Accuracy Variance (AccVarpen). Performance
Drop Rate (PDR), and Worst-Case Accuracy (ACCases).
Unperturbed Metrics
Perturbed Metrics
Benchmark Prompt Type Acc
Avg. 8TCB
Avg. Ven Avg. 2
2y-
Acc Varpen
PDR (%)
AcCwors
Very Confident Questions (VCQ Set)
MMLU
Baseline
0.90
771.5
1.08
4.5
0.05
0.10
0.80
Enhanced 0.95
1025.2
1.03
6.0
0.02
0.03
0.90
GSM8K
Baseline 0.85
2407.0
1.01
4.2
0.06
0.12
0.75
Enhanced 0.92
4410.8
1.05
5.8
0.03
0.05
0.85
Ambiguous
Questions (AQ Set)
MMLU
Baseline
0.40
1983.0
1.01
1.5
0.15
30
0.15
Enhanced 0.70
2734.0
1.00
4.0
0.07
10
0.30
GSM8K
Baseline 0.35
3412,8
1.04
1.2
0.18
35
0.10
Enhanced 0.65
6625,5
1.02
3.8
0.08
12
0.45
correct yet brittle predictions; (2) Inaccurate but Stable: robustly wrong predictions; (3) Confident but Unstable: high confidence indicators (e.g., P(top1) or F - =2z²) but low δTCB: (4) Uncertain but Stable: flatter distribution but resilient underlying state. These elaborated with examples in Appendix D.I.
Case Study:
Systematic Prompt Optimization Guided by ÔTCB Objective: To demonstrate
using ÔTCB for guiding prompt engineering toward more robust solutions. Method Summary: We followed an iterative enhancement process on MMLU & GSM8K, detailed in Appendix C.4.2. First, we ran baseline prompts over 3-5 random seeds identify "Very Confident Questions" (VCQ: high and stable accuracy) and "Ambiguous Questions" (AQ; low/unstable accuracy or low δTCB). Second, for AQ sets, we performed targeted prompt engineering, systematically refining components like ICL examples and instructional phrasing to co-optimize for both accuracy and STCB. Finally, we evaluated the enhanced prompts on unperturbed and perturbed data to measure gains in performance and robustness.
Results: Table 3 (illustrative of observed trends) shows that STCB ggided enhanced prompts achieve higher Acc and significantly higher mean δTCB (c.g., for the MMLU AQ set, from 1983.0 to 2734.0). More critically, they exhibit superior robustness to perturbations, exemplified by lower Performance Drop Rate (PDR) (e.g., MMLU AQ PDR: 30% 10%) and higher worst- case accuracy (Accwars)) (e.g., MMLU AQ set AcC 15% - 30%). ). This underscores that co- optimizing for ôTCB yields more dependable LLM performance, particularly under minor contextual shifts. It is noteworthy that even for the Ambiguous Questions (AQ) set, the average Vofr remains low (cf. Table 3) suggesting that ambiguity in correctness does not necessarily correspond low model confidence; the model can be confidently wrong.
To benchmark frcn-guides optimization, we compared against a baseline strategy of perplexity- guided selection, where prompts are chosen to minimize the negative log- prg-prbbability of the target answer. shown in Table 4, while êTCB yields more robust solutions with higher stability and better worst-case performance under perturbation.
improves accuracy, co-optimizing for
Table 4: Comparison of Prompt Optimization Strategies. Co-optimizing for ÒTCB leads to more robust prompts compared to solely optimizing for perplexity (PPL., showing higher worst-case accuracy (AcCwees) under perturbation.
Optimization Strategy
Avg. Ace
Avg. ÒTCB f
Avg. PPL
Accwest
Baseline Prompt
0.55
15.4
3.2
0.25
PPL-Guided
0.70
18.2
1.9
0.40
5rcB-Guided (Ours)
0.72
35.8
2.4
0.65
4.3.1 DEEP DIVE:
IMPACT OF PROMPT COMPONENT INTERACTIONS ON GSM8K
Objective: To showcase 8TCB' 's fine- -rainga diagnostic capabilities in dissecting prompt component effects. Method Summary: On GSM8K problem gsm8k_ 811, systematically varied ICLs, instructions. and question phrasing from an accurate bascline. Results: Table 5 highlights counter- intuitive accuracy-stabiliy trade-offs. For instance, simply adding clarifying phrase ("7 days".
8

---

Published as a conference paper at ICLR 2026
Table 5: Impact of Prompt Interventions Accuracy and Stability Metrics for GSM8K question. Striking trade-offs between Acc and êTCB emerge. For instance, adding "(7 days)" (Row 2) tanks to 0% but boosts śrcB significantly (8.20 - 46.97), indicating stable but incorrect state. Row shows an extreme case: Zero-Shot with strong instruction results in 0% Ace but astronomical ôrcB ~ 49k, epitomizing "confidently and extremely stably wrong" prediction. Metrics shown are Accuracy (Acc), Token Constraint Bound (ÓTCB), ) Effective Vocabulary Size (Var), top-2 logit margin (Zx 25*).
Index
Intervention Description (gsm8k_811) __1))
Ace (%)
òTCB f
Ven 4
Zy"
Baseline (New Algebraic ICLs, Original Question) Clarified ("7 days") + New Alg. ICLs
100.0
8.20
1.54
3.25
2
0.00
46,97
1.04
5.23
3
Zero-shot CoT Instr. Clarified Q New Alg. ICLS
0.00
10.95
1.44
2.09
4
Role-Playing Instr. Clarified Q New Alg. ICLs
0.00
62.14
1.03
5.98
Algebraic Decomposition Instr. + Clarified Q New
0.00
10.38
1.33
3.62
Alg. ICI.s
6
Hyper- Slyper-Spccffic ICL Alg. Decomp. Instr. + Clarified
0.00
103.87
1.02
5.55
Q
7
Zero-Shot (No ICLs) + Alg. Decomp. Instr. +
0.00
49450.23
1.00
11.29
Clarified Q
8
Formal Language Instr. Clarified Q New Alg. ICLS
0.00
58.28
1.04
5.32
10a
P Par1 Bo)
10a
Orcs (Stability)
10a
10a
exp explllisscttes
exp eeplattessiees
ser
explo
son 10
(UaN
10
1 sidesso
20
30
60
70
Generation Step
ÕтCB log P(Predicted)
Instability Region
Figure 4: 6TCB dynamics P(2nd best) during potentially repetitive generation. Plot shows fTCB (blue, left y-axis) and P(2nd best) (green, right y-axis) versus generation step for I.LAMA-311-3 Sharp dips in SreB (e.g., around steps 5-10, 20-25) often correlate with spikes in P(2nd best), indicating transient local instability not captured by average sequence PPL. Later, high, stable ÒTCR (e.g., steps 30+) can characterize degenerate loop, showing robust commitment to the repetitive patter..
Row 2) decimated accuracy (100% - 0%) but boosted θTCB substantially (from 8.20 to 46.97), inducing stable yet incorrect state. An with a strong algebraic instruction yielded 0% accuracy but an astronomical ÒTen RV 49k and perfect confidence (Verr 1.00, zą - 2n- 11.29). This epitomizes an extremely "confidently and stably wrong" prediction. These findings underscore ôTCB² 's capacity to uncover complex failure modes where models are robustly committed to erroneous reasoning pathsinsights that accuracy or standard confidence scores alone cannot provide.
more extreme case is Row 7, where zero-shot setup
CONCLUSION AND FUTURE WORK
To address Large Language Model sensitivity to input context variations, this paper introduces the Token Constraint Bound (δTCH), novel metric quantifying the local stability of next-token predic- tions against internal state perturbations. Intrinsically linked to the of-weighted geometric disper- sion of output embeddings. ÒTCB offers a principled measure of an LLM's predictive commitment resilience. Our experiments demonstrate ôTCB's 's utility in assessing prompt effectiveness and its ability to uncover critical prediction instabílities missed by perplexity, thus providing a valuable complementary tool for analyzing and potentially enhancing LLM contextual robustness. While these findings are promising, our current investigation primarily uses a specific LLM and a focused set of scenarios. Future work should expand this research across broader spectrum of models, vary- ing scales, and diverse application contexts to validate and generalize the utility of θTCB. Exploring its application understanding perturbations intermediate layers could also yield deeper insights into representation robustness. Further investigation into òTCB's 's role in model editing, fine-tunings and robustness against more structured attacks remains important avenue.

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This work is supported by the National Natural Science Foundation of China (Grant No. 62306087), the Key Research and Development Program of Shandong Provinc (Grant No. 2025CXPT077) the Natural Science Foundation of Shandong Province (Grant No. ZR2023QF154), the Research on Cognitive Processing Technologies for Multimodal Big Data in Policing Information Project (Grant No. 2024DXZD0004), and CCF-Tencen Rhino-Biin Bird Open Research Fund (Grant No. CCF-Tencent RAGR20250105).
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Aryan Agrawal, Lisa Alazraki, Shahin Honarvar. and Marek Rei. Enhancing 1lm robustness to perturbed instructions: An empirical study. arXiv preprint arXiv:2504.02733, 2025.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek. Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021.
Nurit Cohen-Inger, Yehonatan Elisha, Bracha Shapira, Lior Rokach, and Seffi Cohen. Forget what you know about Ilms evaluations- are like chameleon. arXiv preprint 222v::2502.004455 2025.
Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, et al. A survey on in-context learning, arXiv preprint arXiv:2301.002. 2022.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshor,, Aobo Yang. Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song. Danielle Pintz, Danny Livshits, David Esiobu, Dhruv Choudhary. Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Ander- son, Graeme Nail, Gregoire Mialon, Guan Pang. Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jade Copet. Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Ma- hadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock. Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Al- wala, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Lauren Rantala-Yean Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Man- nat Singh, Manohar Paluri, Marcin Kardas, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Torabi, NoraesToraab, Nikolay Bashlykov, Nikolay Bogoychev, Niladri Chatterji, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prajjwal Bhar- gava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sum- baly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana Chennabasappa, Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra. Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarck Sheasha, Thomas Georgiou. Thomas Scialom, Tobias Speckbacher, Todor Mihaylov, Tong Xiao, Ujjwal Karn, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do, Vish Vogeti, Vladan Petro- vic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang, Xiaoqing Ellen Tan, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Goldschlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen. Zoe Papakipos, Aaditya Singh, Aaron Grattafiori, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alex Vaughan, Alexei Baevski, Allie Feinstein, Amanda
11

---

Published as a conference paper at ICLR 2026
Kallet, Amit Sangani, Anam Yumus, Andrei Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, Annie Franco, Aparajita Saraf. Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Bran- don Spence, Brani Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Mejia, Changhan Wang, Changkyu Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, Danny Wyatt, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Duc Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Erik Brinkman, Esteban Ar- caute, Evan Dunbar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Firat Ozgenel, Francesco Caggioni, Francisco Guzmán, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriclla Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Govind Thattai, Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayanan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Aspegren, Hunter Gold- man, Ibrahim Damlaj, Igor Molybog, Igor Tufanov, Irina-Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Japhet Asher, Jean-Baptiste Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan. Jenny Zhen. Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang. Joe Cummings. Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres Josh Ginsburg, Junjie Wang. Kai Wu, Kam Hou U, Karan Saxena, Karthik Prasad, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kun Huang. Kunal Chawla, Kushal Lakhotia, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang. Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Maria Tsimpoukelli, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Ke- neally, Michael L. Seltzer, Michal Valko, Michelle Restrepo. Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Moham- mad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikolay Pavlovich Laptev, Ning Dong, Ning Zhang, Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager, Pierre Roux, Pi- otr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj, Qian Liang, Rachad Alao, Rachel Rodriguez, Raf Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Rohan Maheswari, Russ Howes, Ruty Rinott, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Sa- tadru Pan, Saurabh Verma, Seiji Yamamoto, Sharadh Ramaswamy, Shaun Lindsay, Shaun Lind- say, Sheng Feng. Shenghao Lin, Shengxin Cindy Zha, Shiva Shankar, Shugiang Zhang. Shuqiang Zhang, Sinong Wang, Sncha Agarwal, Soji Sajuyigbe. Soumith Chintala, Stephanie Max. Stephen Chen, Steve Kehoe, Steve Satterfield, Sudarshan Govindaprasad, Sumit Gupta, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury. Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Kohler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Tim- othy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Kumar. Vishal Mangla, Vítor Albiero, Vlad Ionescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li. Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Con- stable, Xiaocheng Tang. Xiaofang Wang, Xiaojian Wu, Xiaolan Wang, Xide Xia, Xilun Wu, Xinbo Gao, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang. Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang, Yuchen Hao, Yundi Qian, Yuzi He, Zach Rait, Zachary Vito,, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, and Zhiwei Zhao. The Ilama 3 herd of models, 2024. org/ abs 220772000000 21783.
URL
Jiahui Geng, Zongxiong Chen, Yuandou Wang, Herbert Woisetschlaeger, Sonja Schimmler, Ruben Mayer, Zhiming Zhao, and Chunming Rong. A survey on dataset distillation: Approaches, appli- cations and future directions. arXiv preprint arXiv:2305.01975 2023.
Bahareh Harandizadeh, Abel Salinas, and Fred Morstatter. Risk and response in large language models: Evaluating key threat categories. arXiv preprint arXiv:2403. 14988, 2024.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint
12

---

Published as a conference paper at ICLR 2026
arXi::2009.03300, 2020.
David Verrera-Poyatoss Carlos Peláez-Gonzālez, Cristina Zuheros, Andrés Herrera-Poyatos Virilo Tejedor, Francisco Herrera, and Rosana Montes, An overview of model uncertainty and variability in Ilm-based sentiment analysis. challenges, mitigation strategies and the role of explainability, arXiv preprint arXiv:2504.044:: 2025.
Judy Hoffman, Daniel A Roberts, and Sho Yaida. Robust learning with jacobian regularization. arXiv preprint 1908.
2019.
Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Train- ing compute-optimal large language models. arXiv preprint arXivv2203.15555, 2022.
Ari Holtzman, Peter West, Vered Shwartz, Yejin Choi, and Luke Zettlemoyer Surface form compe- tition: Why the highest probability answer isn't always right. arXiv preprint arXiv:2104.08315, 2021.
Fred Jelinek, Robert L Mercer, Lalit R Bahl, and James K Baker. Perplexitya measure of the dif- ficulty of speech recognition tasks. The Journal of the Acoustical Society of America, 62(S1): S63-S63, 1977.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. arXiv preprint arXiv:2001..8361, 2020.
Percy Liang, Rishi Bommasani, Tony Lee, Dimitris Tsipras, Dilara Soylu, Michihiro Yasunaga, Yian Zhang, Deepak Narayanan, Yuhuai Wu, Ananya Kumar, et al. Holistic evaluation of language models. arXiv preprint arXiv:2211. 122111.09 1.09/ 10, 2022.
Xiaoou Liu, Tiejin Chen, Longchao Da Chacha Chen, Zhen Lin, and Hua Wei. Uncertainty quantification and confidence calibration in large language models: A survey. arXiv preprint arXiv:2503. 15850, 2025.
Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. arXiv preprint vrXiv:2104.0878k 2021.
Huan Ma, Jingdong Chen, Guangyu Wang, and Changqing Zhang. Estimating Ilm uncertainty with logits. arXiv preprint arXiv:2502.00290. 2025.
Gary Mareus. The next decade in ai: four steps towards robust artificial intelligence. arXiv preprint arXiv:2002.06177, 2020.
Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. Rethinking the role of demonstrations: What makes in-context learning work? arXiv preprint wwXiv:2202.128877 2022.
Fuseini Mumuni and Alhassan Mumuni. Explainable artificial intelligence (xai): from inherent explainability to large language models. arXiv preprint 1eXiv:2501.09960 2025.
Roman Novak, Yasaman Bahri, Daniel A Abolafia, Jeffrey Pennington, and Jascha Sohl- Dickstein. Sensitivity and generalization in neural networks: an empirical study. arXiv preprint arXiv:1802.08760. 2018
OpenAl, :, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mo Bavar- ian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadett-Shapirg Christopher Bemer, Lenny Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-Luisa Brakman, Greg Brockman, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosie Campbell. Andrew Cann, Brittany Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang. Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu,
13

---

Published as a conference paper at ICLR 2026
Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix, Simón Posada Fishman, Juston Forte, Isabella Fulford, Leo Gao, Elie Georges, Christian Gibson, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Gontijo-Lopes,, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufei Guo, Chris Hallacy, Jesse Han, Jeff Harris, Yuchen He, Mike Heaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey, Peter Hoeschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang, Haozhun Jin, Denny Jin, Shino Jo- moto, Billie Jonn, Heewoo Jun, Tomer Kaftan, ukasz Kaiser, Ali Kamali, Ingmar Kanitscheider, Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim, Hendrik Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, ukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis. Kyle Kosic, Gretchen Krueger, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy Lec, Jan Leike, Jade Leung. Daniel Levy, Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz Litwin, Theresa Lopez, Ryan Lowe, Patricia Lue, Anna Makanju, Kim Malfacini, Sam Manning, Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, An- drew Mayne, Bob McGrew, Scott Mayer McKinney, Christine McLeavey, Paul McMillan, Jake McNeil, David Medina, Aalok Mehta, Jacob Menick, Luke Metz. Andrey Mishchenko, Pamela Mishkin, Vinnie Monaco, Evan Morikawa, Daniel Mossing, Tong Mu, Mira Murati. Oleg Murk, David Mély. Ashvin Nair, Reiichiro Nakano, Rajeev Nayak, Arvind Neelakantan, Richard Ngo, Hyeonwoo Noh, Long Ouyang, Cullen O'Keefe, Jakub Pachocki, Alex Paino, Joe Palermo. Ash- ley Pantuliano, Giambattista Parascandolo, Joel Parish, Emy Parparita, Alex Passos, Mikhail Pavlov, Andrew Peng, Adam Perelman, Filipe de Avila Belbute Peres, Michael Petrov, Hen- rique Ponde de Oliveira Pinto, Michael, Pokorny, Michelle Pokrass, Vitchyr Pong, Tolly Pow- ell, Alethea Power, Boris Power, Elizabeth Prochl, Raul Puri, Alec Radford, Jack Rae, Aditya Ramesh, Cameron Raymond, Francis Real, Kendra Rimbach, Carl Ross, Bob Rotsted, Henri Roussez, Nick Ryder, Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman, Daniel Selsam, Kyla Sheppard, Toki Sherbakov, Jessica Shieh, Sarah Shoker, Pranav Shyam, Szymon Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Slama, lan Sohl, Benjamin Sokolowsky. Yang Song, Natalie Staudacher, Felipe Pet- roski Such, Natalie Summers, Ilya Sutskever, Jie Tang, Nikolas Tezak, Madeleine Thompson, Phil Tillet, Amin Tootoonchian, Elizabeth Tseng, Preston Tuggle, Nick Turley, Jerry Tworek, Juan Felipe Cerón Uribe, Andrea Vallone, Arun Vijayvergiya, Chelsea Voss, Carroll Wainwright, Justin Jay Wang. Alvin Wang, Ben Wang, Jonathan Ward. Jason Wei, CJ Weinmann, Akila Weli- hinda, Peter Welinder, Jiayi Weng, Lilian Weng, Mau Wiethoff, Dave Willner, Clemens Winter, Samuel Wolrich, Hannah Wong, Lauren Workman, Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan, Wojciech Zaremba, Rowan Zellers, Chong Zhang, Marvin Zhang. Shengjia Zhao, Tianhao Zheng, Juntang Zhuang, William Zhuk, and Barret Zoph. Gpt-4 technical report, 2023.
Yahao Pang, Xingyuan Wu, Xiaojin Zhang, Wei Chen, and Hai Jin. Fedeat: A robustness optimiza- tion framework for federated llms. arXiv preprint 2rXiv:2502./1866 2025.
Amirhossein Razavi, Mina Soltangheis, Negar Arabzadeh, Sara Salamat, Morteza Zihayat, and Ebrahim Bagheri. Benchmarking prompt sensitivity in large language models. In European Conference on Information Retrieval, pp. 303-313. Springer, 2025.
Melanie Selar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. Quantifying language models' sensi- tivity to spurious features in prompt design or: How learned to start worrying about prompt formatting. arXiv preprint arXiv:2310.11 13244, 2023.
Ayush Singh, Navpreet Singh, and Shubham Vatsal. Robustness of llms to perturbations in text. arXiv preprint arXiv:2407.08988) 2024.
Petr Sychev, Andrey Goncharov, Daniil Vyazhev, Edvard Khalafyan, and Alexey Zaytsev. When an 11m is apprehensive about its answers-and when its uncertainty is justified. arXiv preprint arXiv:2503.01668, 2025.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Slay Petrov, Melvin Johnson, Joannis Antonoglou, Julian Schritwieseer Amelia Glaese, Jilin Chen,
14

---

Published as a conference paper at ICLR 2026
Emily Pitler, Timothy Lillicrap, Angeliki Lazaridou, Orhan Firat, James Molloy, Michael Isard, Paul R. Barham, Tom Hennigan, Benjamin Lee, Fabio Viola, Malcolm Reynolds, Yuanzhong Xu, Ryan Doherty, Eli Collins, Clemens Meyer, Eliza Rutherford, Erica Moreira, Kareem Ay- oub, Megha Goel, George Tucker, Enrique Piqueras, Maxim Krikun, lain Barr, Nikolay Savi- nov, Ivo Danihelka, Becca Roelofs, Anaïs White, Anders Andreassen, Tamara von Glehn, Lak- shman Yagati, Mehran Kazemi, Lucas Gonzalez, Misha Khalman, Jakub Sygnowski, Alexandre Frechette, Charlotte Smith, Laura Culp, Lev Proleev, Yi Luan, Xi Chen, James Lottes, Nathan Schucher, Federico Lebron, Alban Rrustemi, Natalie Clay, Phil Crone, Tomas Kocisky, Jeffrey Zhao, Bartek Perz, Dian Yu, Heidi Howard, Adam Bloniarz, Jack W. Rae, Han Lu, Laurent Sifre, Marcello Maggioni, Fred Alcober, Dan Garrette, Megan Barnes, Shantanu Thakoor, Jacob Austin, Gabriel Barth-Maron, William Wong, Rishabh Joshi, Rahma Chaabouni, Deeni Fatiha, Arun Ahuja, Ruibo Liu, Yunxuan Li, Sarah Cogan, Jeremy Chen, Chao Jia, Chenjie Gu, Qiao Zhang, Jordan Grimstad, Ale Jakse Hartman, Martin Chadwick, Gaurav Singh Tomar, Xavier Gar- cia. Evan Senter, Emanuel Taropa, Thanumalayan Sankaranarayana Pillai, Jacob Devlin, Michael Laskin, Diego de Las Casas, Dasha Valter, Connie Tao, Lorenzo Blanco, Adrià Puigdomènech Badia, David Reitter, Mianna Chen, Jenny Brennan, Clara Rivera, Sergey Brin, Shariq Iqbal, Gabriela Surita, Jane Labanowski, Abhi Rao, Stephanie Winkler, Emilio Parisotto, Yiming Gu, Kate Olszewska, Yujing Zhang, Ravi Addanki, Antoine Miech, Annie Louis, Laurent EI Shafey, Denis Teplyashin, Geoff Brown, Elliot Catt, Nithya Attaluri. Jan Balaguer, Jackie Xiang, Pidong Wang. Zoe Ashwood, Anton Briukhov, Albert Webson, Sanjay Ganapathy, Smit Sanghavi. Ajay Kannan, Ming-Wes Chang, Axel Stjerngren, Josip Djolonga, Yuting Sun, Ankur Bapna. Matthew Aitchison, Pedram Pejman, Henryk Michalewski, Tianhe Yu, Cindy Wang, Juliette Love, Jun- whan Ahn, Dawn Bloxwich, Kehang Han, Peter Humphreys, Thibault Sellam, James Bradbury, Varun Godbole, Sina Samangooei, Bogdan Damoc, Alex Kaskasoli, Sébastien M. R. Arnold, Vijay Vasudevan, Shubham Agrawal, Jason Riesa, Dmitry Lepikhin, Richard Tanburn, Srivat- san Srinivasan, Hyeontaek Lim, Sarah Hodkinson, Pranav Shyam, Johan Ferret, Steven Hand, Ankush Garg, Tom Le Paine, Jian Li, Yujia Li, Minh Giang, Alexander Neitz, Zaheer Abbas, Sarah York, Machel Reid, Elizabeth Cole, Aakanksha Chowdhery, Dipanjan Das, Dominika Ro- goziska, Vitaly Nikolaev, Pablo Sprechmann, Zachary Nado, Lukas Zilka, Flavien Prost, Luheng He, Marianne Monteiro, Gaurav Mishra, Chris Welty, Josh Newlan, Dawei Jia, Miltiadis Allama- nis, Clara Huiyi Hu, Raoul de Liedekerke, Justin Gilmer, Carl Saroufim, Shruti Rijhwani, Shaobo Hou, Disha Shrivastava, Anirudh Baddepudi, Alex Goldin, Adnan Ozturel, Albin Cassirer, Yun- han Xu, Daniel Sohn, Devendra Sachan, Reinald Kim Amplayo, Craig Swanson, Dessie Petrova, Shashi Narayan, Arthur Guez, Siddhartha Brahma, Jessica Landon, Miteyan Patel, Ruizhe Zhao, Kevin Villela, Luyu Wang. Wenhao Jia, Matthew Rahtz, Mai Giménez, Legg Yeung. Hanzhao Lin, James Keeling, Petko Georgiev, Diana Mincu, Boxi Wu, Salem Haykal, Rachel Saputro, Ki- ran Vodrahalli, James Qin, Zeynep Cankara, Abhanshu Sharma, Nick Fernando, Will Hawkins, Behnam Neyshabur, Solomon Kim, Adrian Hutter, Priyanka Agrawal, Alex Castro-Ros, George van den Driessche, Tao Wang. Fan Yang, Shuo yüin Chang, Paul Komarek, Ross Mcllroy, Mario Lui, Guodong Zhang, Wael Farhan, Michael Sharman, Paul Natsev, Paul Michel, Yong Cheng, Yamini Bansal, Siyuan Qiao, Kris Cao, Siamak Shakeri, Christina Butterfield, Justin Chung, Paul Kishan Rubenstein, Shivani Agrawal, Arthur Mensch, Kedar Soparkar. Karel Lene, Timothy Chung, Aedan Pope. Loren Maggiore, Jackie Kay, Priya Jhakra, Shibo Wang, Joshua Maynez, Mary Phuong, Taylor Tobin, Andrea Tacchetti, Maja Trebacz, Kevin Robinson, Yash Katariya, Sebastian Riedel, Paige Bailey, Kefan Xiao, Nimesh Ghelani, Lora Aroyo, Ambrose Slone, Neil Houlsby, Xuchan Xiong. Zhen Yang, Elena Gribovskaya, Jonas Adler. Mateo Wirth, Lisa Lee, Music Li, Thais Kagohara, Jay Pavagadhi, Sophie Bridgers, Anna Bortsova, Sanjay Ghemawat, Zafarali Ahmed, Tianqi Liu, Richard Powell, Vijay Bolina, Mariko linuma, Polina Zablotskaia, James Besley, Da-Woon Chung, Timothy Dozat, Ramona Comanescu, Xiance Si, Jeremy Greer, Guolong Su, Martin Polacek, Raphaël Lopez Kaufman, Simon Tokumine, Hexiang Hu, Elena Buchatskaya, Yingjie Miao, Mohamed Elhawaty, Aditya Siddhant, Nenad Tomasev, Jinwei Xing, Christina Greer, Helen Miller, Shereen Ashraf, Aurko Roy, Zizhao Zhang, Ada Ma, Angelos Fi- los, Milos Besta, Rory Blevins, Ted Klimenko, Chih-Kuan Yeh, Soravit Changpinyo, Jiaqi Mu, Oscar Chang, Mantas Pajarskas, Carrie Muir, Vered Cohen, Charline Le Lan, Krishna Haridasan, Amit Marathe, Steven Hansen, Sholto Douglas, Rajkumar Samuel, Mingqiu Wang. Sophia Austin, Chang Lan, Jicpu Jiang, Justin Chiu, Jaime Alonso Lorenzo. Lars Lowe Sjösund. Sébastien Cevey, Zach Gleicher, Thi Avrahami, Anudhyan Boral, Hansa Srinivasan, Vittorio Selo, Rhys May, Konstantinos Aisopos, Léonard Hussenot, Livio Baldini Soares, Kate Baumli, Michael B. Chang, Adrià Recasens, Ben Caine, Alexander Pritzel, Filip Pavetic, Fabio Pardo, Anita Gergely,
15

---

Published as a conference paper at ICLR 2026
Justin Frye, Vinay Ramasesh, Dan Horgan, Kartikeya Badola, Nora Kassner, Subhrajit Roy, Ethan Dyer, Victor Campos, Alex Tomala, Yunhao Tang, Dalia El Badawy, Elspeth White, Basil Mustafa, Oran Lang, Abhishek Jindal, Sharad Vikram, Zhitao Gong, Sergi Caelles, Ross Hem- sley. Gregory Thornton, Fangxiaoyu Feng, Wojeiech Stokowiec, Ce 7.eegg Phoebe Thacker, Çalar Ünlü, Zhishuai Zhang, Mohammad Saleh, James Svensson, Max Bileschi, Piyush Patil, Ankesh Anand, Roman Ring, Katerina Tsihlas, Arpi Vezer, Marco Selvi, Toby Shevlane, Mikel Rodriguez, Tom Kwiatkowski, Samira Daruki, Keran Rong, Allan Dafoe, Nicholas FitzGerald, Keren Gu-Lemberg. Mina Khan, Lisa Anne Hendricks, Marie Pellat, Vladimir Feinberg, James Cobon-Kerr, Tara Sainath, Maribeth Rauh, Sayed Hadi Hashemi, Richard Ives, Yana Hasson, YaGuang Li, Eric Noland, Yuan Cao, Nathan Byrd, Le Hou, Qingze Wang, Thibault Sottiaux, Michela Pagainin, Jean-Baptiste Lespiau, Alexandre Moufarek, Samer Hassan, Kaushik Shiv- akumar, Joost van Amersfoort, Amol Mandhane, Pratik Joshi, Anirudh Goyal, Matthew Tung, Andrew Brock, Hannah Sheahan, Vedant Misra, Cheng Li. Nemanja Rakievi, Mostafa Dehghani, Fangyu Liu, Sid Mittal, Junhyuk Oh, Seb Noury, Eren Sezener, Fantine Huot, Matthew Lamm, Nicola De Cao, Charlie Chen, Gamaleldin Elsayed. Ed Chi, Mahdis Mahdieh, Ian Tenney, Nan Hua, Ivan Petrychenko, Patrick Kane, Dylan Scandinaro, Rishub Jain, Jonathan Uesato, Romina Datta, Adam Sadovsky, Oskar Bunyan, Dominik Rabiej, Shimu Wu, John Zhang, Gautam Va- sudevan, Edouard Leurent, Mahmoud Alnahlawi, Ionut Georgeseu, Nan Wei, Ivy Zheng, Betty Chan. Pam G Rabinovitch, Piotr Stanczyk, Ye Zhang, David Steiner, Subhajit Naskar. Michael Azzam, Matthew Johnson, Adam Paszke, Bhbung-Cheng Chiu. Jaume Sanchez Elias, Afroz Mo- hiuddin. Faizan Muhammad. Jin Miao, Andrew Lee, Nino Vieillard, Sahitya Potluri, Jane Park, Elnaz Davoodi, Jiageng Zhang, Jeff Stanway, Drew Garmon, Abhijit Karmarkar, Zhe Dong, Jong Lee, Aviral Kumar, Luowei Zhou, Jonathan Evens, William Isaac, Zhe Chen, Johnson Jia, Anselm Levskaya, Zhenkai Zhu, Chris Gorgolewski, Peter Grabowski, Yu Mao, Alberto Magni, Kaisheng Yao, Javier Snaider, Norman Casagrande, Paul Suganthan, Evan Palmer, Geoffrey Irving, Ed- ward Loper, Manaal Faruqui, Isha Arkatkar, Nanxin Chen, Izhak Shafran, Michael Fink, Alfonso Castaño, Irene Giannoumis, Wooyeol Kim, Mikoaj Rybiski, Ashwin Sreevatsa, Jennifer Prendki, David Soergel, Adrian Goedeckemeyer, Willi Gierke, Mohsen Jafari, Meenu Gaba, Jeremy Wies- ner, Diana Gage Wright, Yawen Wei, Harsha Vashisht, Yana Kulizhskaya, Jay Hoover, Maigo Le, Lu Li, Chimezie Iwuanyanwu, Lu Liu, Kevin Ramirez, Andrey Khorlin, Albert Cui, Tian LIN, Marin Georgiev, Marcus Wu, Ricardo Aguilar, Keith Pallo, Abhishek Chakladar, Alena Repina, Xihui Wu, Tom van der Weide, Priya Ponnapalli, Caroline Kaplan, Jiri Simsa, Shuangfeng Li, Olivier Dousse, Fan Yang, Jeff Piper, Nathan Ie, Minnie Lui, Rama Pasumarthi, Nathan Lintz, Anitha Vijayakumar, Lam Nguyen Thiet, Daniel Andor, Pedro Valenzuela, Cosmin Paduraru, Daiyi Peng, Katherine Lee, Shuyuan Zhang, Somer Greene, Duc Dung Nguyen, Paula Kury- lowicz, Sarmishta Velury, Sebastian Krause, Cassidy Hardin, Lucas Dixon, Lili Janzer, Kiam Choo, Ziqiang Feng, Biao Zhang, Achintya Singhal, Tejasi Latkar, Mingyang Zhang, Quoe Le, Elena Allica Abellan, Dayou Du, Dan McKinnon, Natasha Antropova, Tolga Bolukbasi, Orgad Keller, David Reid, Daniel Finchelstein, Maria Abi Raad, Remi Crocker, Peter Hawkins, Robert Leblond,
Dadashi, Colin Gaffney, Sid Lall, Ken Franko, Egor Filonov, Anna Bulanova, Rémi Vikas Yadav, Shirley Chung, Harry Askham, Luis C. Cobo, Kelvin Xu, Felix Fischer, Jun Xu, Christina Sorokin, Chris Alberti, Chu-Cheng Lin, Colin Evans, Hao Zhou, Alek Dimitriev, Han- nah Forbes, Dylan Banarse, Zora Tung, Jeremiah Liu, Mark Omernick, Colton Bishop, Chintu Kumar, Rachel Sterneck. Ryan Foley, Rohan Jain, Swaroop Mishra, Jiawei Xia, Taylor Bos, Ge- offrey Cideron, Ehsan Amid, Francesco Piccinno, Xingyu Wang, Praseem Banzal, Petru Gurita, Hila Noga, Premal Shah, Daniel J. Mankowitz, Alex Polozov, Nate Kushman, Victoria Krakovna, Sasha Brown, MohanmadHossein Bateni, Dennis Duan, Vlad Firoiu, Meghana Thotakuri, Tom Natan, Anhad Mohananey, Matthieu Geist, Sidharth Mudgal, Sertan Girgin, Hui Li, Jiayu Ye, Ofir Roval, Reiko Tojo, Michael Kwong, James Lee-Thorp, Christopher Yew, Quan Yuan, Sumit Bagri, Danila Sinopalnikov. Sabela Ramos, John Mellor, Abhishek Sharma, Aliaksei Severyn, Jonathan Lai, Kathy Wu, Heng-Tze Cheng, David Miller, Nicolas Somnerat, Denis Vnukov, Rory Greig, Jennifer Beattie, Emily Caveness. Libin Bai, Julian Eisenschlos, Alex Korchemniy, Tomy Tsai, Mimi Jasarevic, Weize Kong, Phuong Dao, Zeyu Zheng, Frederick Liu, Fan Yang, Rui Zhu, Mark Geller, Tian Huey Teh, Jason Sanmiya, Evgeny Gladchenko, Neje Trdin, Andrei Sozan- schi, Daniel Toyama, Evan Rosen, Sasan Tavakkol, Linting Xue, Chen Elkind, Oliver Woodman, John Carpenter. George Papamakarios, Rupert Kemp, Sushant Kafle, Tanya Grunina, Rishika Sinha, Alice Talbert, Abhimanyu Goyal, Diane Wu, Denese Afusu-Afriyie, Cosmo Du, Chloe Thornton, Jordi Pont- Tuset, Pradyumna Narayana, Jing Li, Sabaer Fatehi, John Wieting, Omar Ajmeri, Benigno Uria, Tao Zhu, Yeongil Ko, Laura Knight, Amélie Héliou, Ning Niu, Shane
16

---

Published as a conference paper at ICLR 2026
Gu, Chenxi Pang, Dustin Tran, Yeqing Li, Nir Levine, Ariel Stolovich, Norbert Kalb, Rebeca Sonam Goenka, Wenny Yustalim, Robin Strudel, Ali Elqursh, Balaji Lak- shminarayanan, Charlie Deck, Shyam Upadhyay, Hyo Lee, Mike Dusenberry, Zonglin Li, Xuezhi Wang, Kyle Levin, Raphael Hoffmann, Dan Holtmann-Riee, Olivier Bachem, Summer Yue, Sho Arora, Eric Malmi, Daniil Mirylenka, Qijun Tan, Christy Koh, Soheil Hassas Yeganeh, Siim Pöder, Steven Zheng. Francesco Pongetti, Mukarram Tariq, Yanhua Sun, Lucian Ionita, Mojtaba Seyedhosseini, Pouya Tafti, Ragha Kotikalapudi, Zhiyu Liu, Anmol Gulati, Jasmine Liu, Xinyu Ye, Bart Chrzaszez. Lily Wang, Nikhil Sethi, Tianrun Li, Ben Brown, Shreya Singh, Wei Fan, Aaron Parisi, Joe Stanton, Chenkai Kuang, Vinod Koverkathu, Christopher A. Choquette- Choo, Yunjie Li, TJ Lu, Abe Ittycheriah, Prakash Shroff, Pei Sun, Mani Varadarajan, Sanaz Bahargam, Rob Willoughby, David Gaddy, Ishita Dasgupta, Guillaume Desjardins, Marco Cornero, Brona Robenek, Bhavishya Mittal, Ben Albrecht, Ashish Shenoy, Fedor Moiseev, Henrik Jacobsson, Alireza Ghaffarkhah. Morgane Rivière, Alanna Walton. Clément Crepy, Alicia Parrish, Yuan Liu, Zongwei Zhou, Clement Farabet, Carey Radebaugh, Praveen Srinivasan, Claudia van der Salm, Andreas Fidjeland, Salvatore Scellato, Eri Latorre-Chimoto, Hanna Klimezak-Pluciske, David Bridson, Dario de Cesare, Tom Hudson, Piermaria Mendolicchio, Lexi Walker, Alex Mor- ris, Ivo Penchev, Matthew Mauger, Alexey Guseynov, Alison Reid, Seth Odoom, Lucia Loher, Victor Cotruta, Madhavi Yenugula, Dominik Grewe, Anastasia Petrushkina, Tom Duerig, Anto- nio Sanchez, Steve Yadlowsky, Amy Shen, Amir Globerson, Adam Kurzrok, Lynette Webb, Sahil Dua, Dong Li, Preethi Lahoti, Surya Bhupatiraju, Dan Hurt, Haroon Qureshi, Ananth Agarwal, Tomer Shani, Matan Eyal, Anuj Khare, Shreyas Rammohan Belle, Lei Wang, Chetan Tekur, Mi- hir Sanjay Kale, Jinliang Wei, Ruoxin Sang, Brennan Saeta, Tyler Liechty, Yi Sun, Yao Zhao, Stephan Lee, Pandu Nayak, Doug Fritz, Manish Reddy Vuyyuru, John Aslanides, Nidhi Vyas, Martin Wicke, Xiao Ma, Taylan Bilal, Evgenii Eltyshev, Daniel Balle, Nina Martin, Hardie Cate, James Manyika, Keyvan Amiri, Yelin Kim, Xi Xiong, Kai Kang, Florian Luisier, Nilesh Tripuraneni, David Madras, Mandy Guo, Austin Waters, Oliver Wang, Joshua Ainslie, Jason Baldridge, Han Zhang, Garima Pruthi, Jakob Bauer, Feng Yang. Riham Mansour, Jason Gelman, Yang Xu, George Polovets, Ji Liu, Honglong Cai, Warren Chen, XiangHai Sheng, Emily Xue, Sherjil Ozair, Adams Yu, Christof Angermueller, Xiaowei Li, Weiren Wang. Julia Wiesinger, Em- manouil Koukoumidis, Yuan Tian, Anand lyer, Madhu Gurumurthy, Mark Goldenson, Parashar Shah, MK Blake, Hongkun Yu, Anthony Urbanowicz, Jennimaria Palomaki, Chrisantha Fernando, Kevin Brooks, Ken Durden, Harsh Mehta, Nikola Momchev, Elahe Rahimtoroghi, Maria Geor- gaki, Amit Raul, Sebastian Ruder, Morgan Redshaw, Jinhyuk Lee, Komal Jalan, Dinghua Li, Ginger Perng, Blake Hechtman, Parker Schuh, Milad Nasr, Mia Chen, Kieran Milan, Vladimir Mikulik, Trevor Strohman, Juliana Franco, Tim Green, Demis Hassabis, Koray Kavukeuoglu, Jeffrey Dean, and Oriol Vinyals. Gemini: A family of highly capable multimodal models, 2023.
Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D Manning. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback. arXiv preprint arXiv:2305. 14975, 2023.
Hugo Touvron, Louis Martin, Kevin Stone. Peter Albert, Amjad Almahairi, Yasmine Babaci, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, al. Llama Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.09288 2023.
Yuqing Wang and Yun Zhao. Rupbench: Benchmarking reasoning under perturbations for robust- ness evaluation in large language models. arXiv preprint 24X::24066.11200 2024.
Jerry Wei, Jason Wei, Yi Tay, Dustin Tran, Albert Webson, Yifeng Lu, Xinyun Chen, Hanxiao Liu, Da Huang, Denny Zhou, al. Larger language models do in-context learning differently. arXiv preprint arXiv:2303.00 03846, 2023.
Laura Weidinger, John Mellor, Maribeth Rauh, Conor Griffin, Jonathan Uesato, Po-Sen Huang. Myra Cheng, Mia Glaese, Borja Balle, Atoosa Kasirzadeh, et al. Ethical and social risks of harm from language models. arXiv preprini urXiv:2 2021.
Mozhi Zhang, Mianqiu Huang, Rundong Shi, Linsen Guo, Chong Peng, Peng Yan, Yagian Zhou, and Xipeng Qiu. Calibrating the confidence of large language models by eliciting fidelity. arXiv preprint ar 2il:2440402655. 2024.
17

---

Published as a conference paper at ICLR 2026
Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. Calibrate before use: Improving few-shot performance of language models. In International conference on machine learning, pp- 12697-12706. PMLR, 2021.
18

---

Published as a conference paper at ICLR 2026
CONTENTS
1
Introduction
2
Preliminaries:
Understanding LLM Predictions and Stability
2.1
Language Model Output and Distribution Concentration
2.2
Defining Stability: What Does It Mean for a Prediction to Be Stable?
2.3
Quantifying the Impact of Perturbations: The Role of the Jacobian
2.4
The Token Constraint Bound (δTCB): Our Mcasure of Stability
3
ÒTCB via Output Embedding Geometry
3.1
The Geometry of Output: Token Embeddings and Their Mean
5
3.2
Deriving the Jacobian Norm: Connecting Sensitivity to Embedding Spread
3.3
The Full Form of 8TcB and Its Geometric Meaning
3.4
Interpreting Stability Across Prediction Regimes
Experiments
4.1
Experimental Setup
4.2
Intrinsic Properties and Validation of бTCB
4.3
δTCB as Diagnostic Tool for Prompt Engincering 4.3.1 Deep Dive: Impact of Prompt Component Interactions on GSM8K
5 Conclusion and Future Work
A LLM Usage Statement
20
B Related Work
20
Experimental Details
C.I
Datasets and Task-Specific Setup
21
C.2
Prompting Strategies and Examples
21
C.2.1 MMLU Prompts
................................
C.2.2 GSM8K Prompts (Baseline and Interventions)
21
C.3
òTCB Calculation and Parameters
22
C.4
Details of Specific Experiments..
23
C.4.1 Geometry Sensitivity Simulation
23
C.4.2
Systematic Prompt Optimization Protocol
23
C.4.3
Perturbation Strategies for Robustness Evaluation
23
C.4.4 Text Generation Dynamics Setup
23
D Further Discussion and Examples
24
D.I Elaboration on Eccuracy-Stabilit Conflict Scenarios
................................
24
D.2 Detailed Interpretation of GSM8K Intervention Analysis
24
D.3 Differentiating ÒTCB from Perplexity (PPI.)
E Quantifying Semantic Context Influence.
26
F Statistical Derivation of òTCB Approximation
26
Exact, Non-Statistical Expression for [[Jw|lr and Its Relation to Weighted Variance
29
H Relating the Statistical Approximation and the Exact Expression
I Derivation of the Exact Jacobian Norm Formula
32
J Derivation of Regime-Dependent TCB Behavior
35
J.I Regime I: High Flatness (Large Vett >> 1)
35
J.2 Regime 2: Low Flatness (Small Van & 1)
36
19

---

Published as a conference paper at ICLR 2026
A LLM USAGE STATEMENT
LLMs were used solely as auxiliary tools for paper polishing. They did not contribute to the genera- tion of research ideas, the design of experiments, the development of methodologies, data analysis, or any substantive aspects of the research. All scientific content, conceptual contributions, and ex- perimental results are entirely the work of the authors. The authors take full responsibility for the contents of this paper.
B RELATED WORK
Our work on the Token Constraint Bound (δTCB) builds upon and differentiates itself from several lines of research LLM evaluation, robustness, and interpretability.
LLM Sensitivity and the Need for Robustness Metrics The pronounced sensitivity of LLMs to subtle imput variations is well-documented. Studies have demonstrated substantial performance fluctuations arising from minor alterations in prompt phrasing (Razavi et al., 2025), example order- ing and selection (Zhao et al., 2021; Lu et al., 2021), or even formatting details (Sclar et al., 2023). urgent need for evaluation methods that go beyond ag-
This brittleness (Marcus, 2020) highlights gregate task performance to assess the inherent stability of LLM predictions. Importantly, increased model scale, despite adherence to scaling laws (Kaplan al., 2020; Hoffmann et al., 2022), does not invariably confer enhanced robustness to these nuanced changes (Lu et al., 2021; Wei et al., 2023), further motivating metries like бTCB that focus on local stability. The imperative for reliable deploy- ment in critical applications (Weidinger et al., 2021; Herrera-Poyater et al., 2025; Harandizadeh et al., 2024) makes understanding and quantifying such instabilities paramount.
Limitations of Conventional Evaluation Metrics Conventional metrics offer limited insight into local predictive robustness, a gap STCB aims to address. Task accuracy, while a primary indicator, provides an aggregate view that mask underlying prediction fragility due to contextual shifts (Zhao et al., 2021). Perplexity (Jelinek et al., 1977), though standard for assessing sequence like- lihood (Liang al., 2022; Holtzman et al., 2021), aggregates probabilities and can obscure local dynamics and the true stability of the internal state underpinning a prediction (Cohen-Inger et al., 2025). may not reflect genuine model conviction, especially in cases of "surface form competition" (Holtzman et al., 2021) or when the softmax normalizes over poorly supported options. Emerging metries targeting confidence and calibration (Zhang et al., 2024; Tian et al., 2023; Geng et al., 2023; Zhao et al., 2021) focus on aligning output probabilities with the likelihood of correctness (Tian et al., 2023). While valuable, these do not directly quantify the resilience of a specific prediction's 's dominant rank against perturbations in the model's internal representation h (Liu et al., 2025). A well-calibrated, high-confidence prediction might still stem from an internally fragile state. òTCB measures this internal representational stability directly.
Internal State Dynamics and Representational Stability Our work intersects with research ex- ploring LLM internal representations and their stability. Efforts enhance robustness by, for ex- ample, aligning hidden states of perturbed instructions with original ones (Agrawal et al., 2025), implicitly underscore the importance of stable internal configurations. òTCB provides a direct, quan- titative measure of this local stability specifically for the next-tok pext-token prediction mechanism, assessing the "safety margin" or integrity of the current predictive commitment arising from h. This connects to broader goals in Explainable AI (XAI) that seek to understand internal model workings (Mumuni & Mumuni, 2025), where ÊTCB can pinpoint internal decision points of high or low resilience.
Prompt Engineering, In-Context Learning, and Stable State Induction δTCB is particularly relevant to analyzing the effectiveness of prompt engineering and In-Context Learning (Brown et al., 2020; Dong al., 2022). Our central hypothesis posits that effective contextualization, such as through well-designed prompts or informative ICL examples, guides LLMS not only to correct an- swers but also to more stable internal states h. reflected by higher òTCB values. This aligns with findings suggesting ICL's efficacy often stems from its ability to clarify task structure (Min et al., 2022). ôTCB can thus serve as a quantitative tool to assess how effectively different contextual in- puts induce robust internal commitments to predictive path, complementing metrics focused on epistemic "knowledge strength" (Ma et al., 2025).
Output Embedding Geometry and Predictive Stability A key theoretical underpinning of STCB- detailed in Section 3, is its intrinsic link to the geometric dispersion of output token embeddings. While the general importance of embedding space properties for model robustness is acknowledged
20

---

Published as a conference paper at ICLR 2026
(Pang et al., 2025), δTCB establishes specific, analytical relationship between the geometry of the output embeddings (weighted by current prediction probabilities of) and the local stability of the next-token prediction against perturbations in h. This provides mechanistic, geometric interpreta- tion of local prediction stability, moving beyond probability- based analyses alone.
Complementarity with Uncertainty Quantification (UQ) ÔTCB is positioned as complemen- tary to, rather than a replacement for, existing Uncertainty Quantification (UQ) methods (Liu et al., 2025; Sychev et al., 2025). While UQ approaches typically aim to gauge a model's confidence or "knowledge strength" often based on output probabilities (e.g., entropy. prediction margins), θTCB specifically assesses the robustness of the predictive mechanism itself to internal state fluctuations. prediction can exhibit high output probability (high confidence by UQ standards) yet originate from an unstable internal state (low δTCB), indicating "confidently unstable" scenario. Conversely, a moderately confident prediction might be highly stable. êTCB thus offers a distinct perspective on reliability, focusing the local integrity and resilience of the model''s current predictive commit- ment rather than solely its expressed certainty.
Perturbation- -Baee Robustness in Neural Networks and LLMS Prior work on NN robustness has leveraged Jacobian norms and perturbations to quantify sensitivity and improve generalization. (Novak et al., 2018) empirically study sensitivity via the Frobenius norm of the input Jacobian, finding trained NNs more robust near training data and linking it to generalization gaps. Subsequent methods regularize this norm for adversarial robustness, e.g., (Hoffman et al., 2019) for classification margins. In LLMs, recent studies evaluate robustness to input perturbations like typos rephrasing (Singh et al., 2024; Mumuni & Mumumi, 2025), with benchmarks like RUPBench (Wang & Zhao, 2024) showing larger models' resilience. бTCB extends these by deriving an exact closed-form for the softmax-linear Jacobian norm, linking it to output embedding geometry, and repurposing it for contextual stability in ICL and prompt engineering- -oreusiinnnno on hidden state perturbations rather than inputs.
C EXPERIMENTAL DETAILS
C.I DATASETS AND
SETUP
MMLU (Hendrycks al., 2020): For general validation (e.g., correlations in Table 1), ques- tions were sampled from the 'test" splits of subjects like lommalloogic.. 'philosophy', 'ab- stract algebra', 'moral scenarios', and 'professional law'. Laressionalllw'..... The standard multiple-choie format (A, B, C, D) was used. The critical prediction point for ôTCB calculation was immediately before the model generated the single token corresponding to its chosen letter (e.g., 'A', 'B", 'C', or 'D'). GSM8K (Cobbe et al., 2021): For detailed intervention and prompt optimization studies, ques- tions were sampled from the 'test' set. These are grade-school math word problems requiring multi-step reasoning. The critical prediction point for ÒTCB was typically before the model gener- ated the first token of the final numerical answer, after producing its chain-of-thought (CoT) rea- soning. The final answer usually identified by a pattern like "The final answer is \boxed (X).".
C.2 PROMPTING STRATEGIES AND EXAMPLES
C.2.1 MMLU PROMPTS
Zero-Shot Multiple Choice:
Question: (question_text)
Options:
A) (option_A_text:
B) (opticn_B_texx
C) (option_C_textt
D) Sottoo_D_ttexe
Answer:
The model is expected to complete with 'A', 'B', 'C'. 'D.
C.2.2 GSM8K PROMPTS (BASELINE AND INTERVENTIONS)
Zero-Shot Chain-of-Thought (CoT):
Question: [neestion_text] Let's 8 think step by step.
21

---

Published as a conference paper at ICLR 2026
The model generates the reasoning steps and concludes with "The final answer is \boxed (X)."
Few-Shot Chain-of-Thought (CoT) (k = 5 baseline for GSM8K):
Question: cesmppaar_quusttion )
Let's s think step by step.
The final answer is \boxed [exempaarl_answeeeee
###
Question: Let's s think step by step. fexemplar2_Cos _er___o_____ _oT_ssoution]
The final answer is \boxed (exeeplaralal answer)) ###
(3 more exemplars)
###
Question:
courrent_question_tt Let's s think step by step.
Exemplars were typically drawn from the GSM8K 'train' set. Interventions for GSM8K problem gsm8k 811 (as in Table 5): The base question gsmak_ _11 is: "Felix earns $0.25 for each branch he trims from tree. He trimmed branches from trees. If he earned $60 in total, what is the average number of branches he trimmed per tree?" (Correct Answer: 20)
Clarified Q ("7 :: The original question text was appended with: "(Felix works 7 days week)." " This clarification irrelevant and misleading for this specific problem.
ICI. Variations:
New Algebraic ICLs (Baseline ICLs for Table 12):
Exemplars were selected/written to em- phasize setting up and solving algebraic equations (e.g., using variables like x, y).
Hyper-Specific ICL: An ICL example was crafted to be structurally almost identical to gsm8k_8l 811 (e.g.. "John earns SX per item. He processed Y items from Z batches. If he earned STotal, what the average items per batch?"), but with different numbers and context.
Instructional Prefixes: These were typically inserted directly before "Let's think step by step." in few-shot setup, the main instruction in zero-shot setup.
Zero-shot CoT Instr. (implied for Col 3 of Table 12 in context): Simply "Let's think step by step." as the primary instruction for the new question.
Role-Playing Instr: "You are brilliant mathematician. Solve the following problem by showing your detailed work." "
Algebraic Decomposition Instr: "Decompose this problem algebraically, Define variables, up equations, and solve them step-by-step to find the final answer."
Formal Language Instr: "Use precise mathematical language and formal notation in your solution. Ensure each step clearly justified."
For Table 5, "New Alg. ICLs" were used unless "Hypef-Specific ICL." or "No ICLs" (Zero-Shot) is specified.
C.3 êTCB CALCULATION AND PARAMETERS
The Token Constraint Bound STCB(h was computed using Eq. (10) from the main text:
(TCB(h)
EL of lw,, p (h)ll2
The tolerance parameter was set to 1.0 for all experiments. This choice normalizes ôTCB such that it represents the inverse the Jacobian's 's Frobenius norm (scaled by we-whittted embedding variance). As stated in the main text, relative changes and comparative values of śTCB generally more informative than its absolute magnitude, which depends on e. The hidden state was taken from the output of the final transformer layer, just before the unembedding layer.
22

---

Published as a conference paper at ICLR 2026
C.4 DETAILS OF SPECIFIC EXPERIMENTS
C.4.1 GEOMETRY SENSITIVITY SIMULATION
For each input prompt: 1. The original hidden state hoig and output probability distribution Oocig
were obtained. ÔTCB (horig: Worig was calculated. 2. The top K 10 competitor tokens (tokens j * topl with the highest 0ÿ) were identified. 3. To create W ciuster: For each competitor embedding w (j E K competitors),
new position was unjtuster (moving it halfway towards the top-1 token's s embedding). Embeddings of other tokens remained unchanged. δTCB (orig Weluste) was calculated using the original horig and Ourig- 4. To create
g + ar (opl
00 uppi
w ( where a = 0.5
W disperse: For each competitor embedding wy, its new position was wj willaperse w ( topl
) where 3 0.5 (moving it away from the top-1 token's embedding along the same line, by half the original distance). ÔTCB (Norig. W disperse) was calculated. The hypothesis ÔTCB ( cluster) STCB (CCB(Wen))) orig) > STCB ( ddspersee ) was then checked.
C.4.2 SYSTEMATIC PROMPT OPTIMIZATION PROTOCOL
1. Baseline Characterization: A set of questions (c.g.. 50-100 from MMLU/GSM8K) was run with baseline prompt (e.g.. zero-shot for MMLU, 5-shot CoT for GSM8K) across multiple random seeds (e.g., S = 3 or S 5) for ICL selection or minor phrasing variants to get Acc statistics. Metrics collected per question: Mean Accuracy, Accuracy Variance (across seeds), Mean êTCB (at critical token, averaged over seeds if multiple correct paths), θTCB Variance. VCQ (Very Confident Questions) were defined as those with, e.g., Mean Ace 2 0.9, Acc Var & 0.05, Mean ỘTсB > Th, ÒTCB Var < Vz. AQ (Ambiguous Questions) were defined those with, e.g., Mean Acc < 0.6, or Ace Var > 0.15, or Mean ÒTCB < Tr, or òTCB Var > VH. Thresholds (TH, Tr, VL, VH) were set empirically based observed distributions.
2. Targeted Prompt Enhancement: For AQ questions, prompt engineering efforts focused on in- creasing both Ace and 8TCB. Techniques included: Refining ICL examples (e.g., ensuring CoT steps are clearer, more analogous to the target problem structure, varying reasoning styles). Modify- ing instructional phrases (e.g., adding "Be very careful with calculations," " "Explain your reasoning clearly"). Switching prompting strategy (e.g., from 5-shot to 2-shot with very high-quality examples, or adding a self-reflection step). For MMLU, this could involve adding a directive like "Choose the best option and explain why." For VCQ questions, efforts might focus further increasing δTcB if wasn't i already maximal, or ensuring robustness (see below).
Enhanced prompts were evaluated on the same metrics. Robustness was tested using perturbed inputs (see Appendix C.4.3), Metrics like Accuracy Variance on perturbed inputs (AceVarpent) Performance Drop Rate (PDR), and Worst-Case Accuracy (Accworst) under perturbation were key.
2.Evaluation (Unperturbed and Perturbed):
C.4.3 PERTURBATION STRATEGIES FOR ROBUSTNESS EVALUATION
Perturbations were designed to be plausible minor variations:
Syntactic Perturbations (applied to the question text):
Using pre-trained paraphrasing model (e.g., T5-based) to generate 2-3 vari- ants of the question.
Paraphrasing:
Reordering: For questions with multiple clauses/condittins, reordering them semantically permissible.
Synonym Replacement: Replacing 1-2 keywords with close synonyms.
Semantic Perturbations (primarily for ICL/ Few-Shot setups):
ICL Example Reordering: Changing the order of the few-shot examples in the prompt.
ICI. Example Replacement: Replacing one of the k examples with another valid but perhaps slightly less similar or slightly lower-quality example from the training set.
Adding a minor distractor sentence to the prompt context.
The goal was not to make the task unsolvable but to test sensitivity to typical input variations.
C.4.4 TEXT GENERATION DYNAMICS SETUP
The prompt used to induce potentially repetitive behavior was:
System: Repeat the following word exactly five times: 'banana'. After repeating it five times, say TTask finished.'.
23

---

Published as a conference paper at ICLR 2026
User: Okay, I understand. will now repeat the word 'banana' five times and then say Task finished.'
Assistant:
The model (LLAMA-3.I-8B)
allowed to generate tokens greedily. STCB, P(top1), P(2nd best), and VeH were calculated at each token generation step. The figure typically shows dynamics if the model continues beyond the explicit instruction, e.g-, by getting stuck in loop of "banana" or "Task finished." " or exhibiting other non-ideal behaviors. The "dips" in бTCB often occur points where the model is less certain about continuing pattern versus breaking out or switching to another token.
D FURTHER DISCUSSION AND EXAMPLES
D.I ELABORATION ON ACCURACY-STABILLTY CONFLICT SCENARIOS
The main text Section 4.3) mentioned four key conflict scenarios. Here are more detailed concep- tual examples:
1.Accurate but Unstable (High Acc, Low STCB): Scenario: The model correctly answers a com- plex reasoning question, but its internal state is near a decision boundary where a slight internal "wobble" could have led to a different (incorrect) reasoning path and answer. Example (MMLU- Q: "Which of these legal principles is most directly violated by ex post facto laws?" Correct Answer: "Nulla poena sine lege". Model predicts "Nulla poena sine lege" (Acc=1). However, its δTCB is low (e.g., 2). This might be because the embedding for another plausible but incorrect principle like "Stare decisis" is geometrically positioned such that small perturbation h could shift the logits sufficiently to make "Stare decisis" the top choice. The correct prediction is thus brittle.
2. Inaccurate but Stable (Low Ace, High ôTCB): Scenario: The model makes clear error, often due a misinterpretation or flawed reasoning pattern, but it is very robustly committed to this error. Example (GSM8K-like, from Table 5, Row 2): Felix problem with the misleading "7 days" clarification. The model incorrectly calculates the answer (Acc=0) but does so with high ÒTCB (46.97). It has latched onto stable, but flawed, interpreaation/proceduree
3. Connfident but Unstable (High P(top1)/2g 11:---2p 2y-, but Low òTCB): Scenario: The output prob- ability for the top token is high, and/or the logit margin to the next token is large, suggesting strong confidence. However, the internal state supporting this is not robust. Example: Prompt: "What is the primary ingredient in concrete?" Model predicts "Cement" with P(Cement) = 0.95 and large logit margin 24 2j- 5.0. Superficially, this looks very confident. However, δTCB is low (e.g.. 1.8). This could occur if: The embedding WCement is relatively far from the mean embedding p (h) (making the ccmmet WCement nm (h) 112 term large despite & being large too, if the distance is very large). Or, many other low- low-probability competitor embeddings {we } are clustered very tightly around mm (h), leading to many small of [|, p (h) terms whose sum significant, contribut- ing to large Jacobian norm. The high probability P(Cement) might hide an underlying geometric configuration that is sensitive to perturbation.
4. Anneertain but Stable (Low P(top1)/zz -y. High Veits but High ôTCB): Scenario: The model's 's output distribution is relatively flat, indicating uncertainty among several top choices. Yet, the internal state representing this uncertainty is stable. Example: Prompt: "Which of these is a common pet: A) Dog, B) Tiger, C) Whale, D) Ant". Model output: P(Dog) = 0.5, P(Ant) = 0.3 (perhaps due to "common"), "), P(Tiger) = 0.1, P(Whale) = 0.1. Varr is relatively high. However, êTCB could be high if the embeddings WDng and WAm are very close to each other (and thus to (h) ) they dominate), while Wriger and WWhale are far away. The model is stably "stuck" deciding between "Dog" and "Ant", but it's not about to suddenly jump to "Tiger". The uncertainty itself has a stable geometric basis.
D.2 DRTAILED INTERPRETATION OF GSM8K INTERVENTION ANALYSIS
This provides row-by-row interpretation of the results for GSM8K question gsm8k_811 811 shown in Table 5 of the main paper.
Row 1: Baseline (New Algebraic ICLs, Original Question) Metrics: Acc=100% 8TCB
8.20, Ven = 1.54, 2g 2j. 3.25. Interpretation: The baseline prompt with algebraic ICLS successfully solves the problem. The stability (ŌTCB 8.20) is moderate, indicating reasonably
24

---

Published as a conference paper at ICLR 2026
robust correct prediction but with potential for improvement. Veil and 24 2j- reflect good confidence.
Row 2: Claritied Q ("7 days") New Alg, ICLs Metrics: Acc=0% бTCB = 46.97, Vetr = 1.04, Fk: 2zz 5.23. Interpretation: Adding the misleading "7 days" clarification breaks accuracy completely. Crucially, òTCB increases dramatically. This indicates the model has latched onto an incorrect interpretation or reasoning path due to the "7 days" phrase, and this incorrect path is highly stable. The high confidence metrics (Vett R 1, 2g Zj- high) support this: it's "confidently and stably wrong."
Row 3: Zero-shot CoT Instr. + Clarified Q + New Alg. ICLS Metrics: Acc=0% òTCB = 10.95,
Vetr 1.44, Z Zj- 2.09. Interpretation: The standard "Let's think step by step" instruction does not fix the error caused by "7 days". The stability (δTCh = 10.95) is higher than the original baseline (Row 1) but much lower than the "stably wrong" state in Row 2. This suggests the CoT instruction interacts with the misleading clarification and ICLS to produce state that is still incorrect but less internally committed/ssablle than Row 2.
Row 4: Role- Poay-Playyng Instr. + Clarified Q + New Alg. ICLS Metrics: Acc=0% δTCB = 62.14, Van 1.03, 2k 2j- 5.98. Interpretation: Accuracy remains 0. The Role-Playing instruction ("You are a brilliant mathematiciam..." ") leads to an even higher STCB than Row 2. suggesting this type of instruction might encourage the model to commit more strongly a particular reasoning path, even if that path is flawed due to other elements like the "7 days" clarification. Again, very high confidence in the wrong answer.
Row 5: Algebraic Decomposition Instr. + Clarified Q + New Alg. ICLS Metrics: Acc=0% ÔTCB = 10.38, Vat = 1.33, F 2j- 3.62. Interpretation: Similar to Row 3, this more specific algebraic instruction fails to correct the error. The stability is low, suggesting a conflict: the instruction pushes for algebraic rigor, but the "7 days" phrase may prevent coherent algebraic formulation of the (misinterpreted) problem, leading to an unstable incorrect state.
Row 6: Hyper-Specific ICL + Alg. Decomp. Instr. + Clarified Q Metrics: Acc=0% бTCB = 103.87. Ve 1.02, 2 - - 5.55. Interpretation: Even with a perfectly analogous ICL example and an algebraic instruction, the "7 days" clarification persists in causing an error. The êTCB is extremely high. This implies the model rigidly follows the structure of the hyper-specific ICI.. If the "7 days" clarification leads to consistent misapplication of that structure (e.g., always multiplying by at a certain step because it's done in the (misinterpreted) exemplar logic), the result is a very stable, confident, but incorrect prediction.
Row 7: Zero-Shot (No ICLs) Alg. Decomp. Instr. + Clarified Q Metrics: Acc=0% êTCB R 49450, Vatr = 1.00, 2& 2j- = 11.29. Interpretation: This is the most striking re- sult. Removing ICLS (which might have conflicting signals) and relying solely on the strong " "Algebraic Decomposition" instruction in the presence of the "7 days" clarification leads to an astronomically high êTCB and perfect confidence metrics, yet 0% accuracy. The model is utterly convinced by its (flawed) algebraic decomposition of the misinterpreted problem. This is the epitome of a "confidently and extremely stably wrong" state.
Row 8: Formal Language Instr. + Clarified Q + New Alg. ICLS Metrics: Acc=0% ÒTCB = 58.2 388.8817 Vaa 1.04, 2 2j- 5.32. Interpretation: Similar the Role-Playing instruction (Row 4), asking for formal language seems to stabilize the incorrect reasoning path derived from the "7 days" clarification and algebraic ICLs, leading to high ôTCB and confidence in the error.
This detailed analysis demonstrates êTCB's 's power in revealing how different prompt components interact to affect not just accuracy, but the internal stability and commitment of the model to its predictions, whether correct incorrect.
D.3 DIFFERENTIATING ôTCR FROM PERPLEXITY (PPL)
êTCB assesses internal state robustness, quality distinct from PPL's measure of sequence likeli- hood or token-level prediction confidence. Key differentiators include: Robustness of (Potentially Erroneous) Confident Predictions: PPL (or low token probability) flags incorrect predictions (if ground truth is known) but doesn't indicate if the model is stably committed to an error. δTCB quantifies this "stable incorrectness" (e.g., Table 5, Row 7, where êTCB is exceptionally high for an incorrect answer). Sensitivity to Output Embedding Geometry: PPL is solely a function of probabilities and thus blind to the geometric arrangement of output embeddings, which critically impacts ôTCB and the true local stability of the prediction (cf. Section 4.2 and Eq. (10).. Detection of Local Instabilities During Text Generation: Sequence-level PPL. averages token likelihoods,
25

---

Published as a conference paper at ICLR 2026
(a] Semantic Distance vs TCB
(b) Jacobian Norm vs TCB
(c) Taken probability vs TCB
Thery y-3 1209
155
7.0
66
TCB
6.9
5.0
67
13
5.13 6.14 5.15 0.15
0.18 6.5 0.20 6.2) Jacobian Norm Molk
Token Probability (x107)
Figure 5: Relationship between Prefix Semanties, TCB, and Model Internals.
Visual analysis based on semantic distance shows
16 semantically varied prefixes targeting the continuation. (a) êTCB strong positive linear correlation (It" 0.91). Point colors map to semantic distance (see colorbar), and annotations identify prefixes. The empirical (gray dashed line) closely matches the data, aligning well with the thcoretical prediction (black dashed line, Eq. (11)), (b) бTCB versus the Frobenius norm of the final layer exhibits moderate negative correlation. (c) òTcB versus the probability of the first token
Jacobian ([Jw [[)) the continuation shows less distinct relationship for this set of prefixes.
potentially masking transient points of high internal instability that precede errors degener- ate loops. Figure 4 illustrates ôTCB capturing sharp dips in ôTCB (often correlating with spikes in P(2nd best token)). These crucial local dynamics, indicative of the model teetering between alternatives, are often smoothed over by aggregate PPL but are vital for understanding generation failures.
E QUANTIFYING SEMANTIC CONTEXT INFLUENCE.
Effective prompts often rely on precise semantic alignment. We investigated if STCB quantitatively reflects this. Using 16 prefixes with varying semantic distances (cosine distance of embeddings) to target prefix, we measured the θTCB of the first generated token. Figure 5 reveals a strong positive linear correlation (R²² = 0.91) between semantic distance and ôTCB- The empirical fit:
θTCB = 5.012 - dist + 1.249
(11)
closely matches the data and theoretical predictions derived from sensitivity analysis. This shows òTCB provides quantitative measure how semantic (mis)alignment in the prompt impacts the stability of the resulting predictive state. Figure 5 also shows the expected negative correlation between ÔTCB and the Jacobian norm (]lJw F)). This ability to quantify semantic influence further highlights êTCB'S 's utility for fine-grained prompt analysis and optimization.
F STATISTICAL DERIVATION OF ÒTCB APPROXIMATION
In this section, provide detailed derivation of statistical approximation for the Token Con- straint Bound (êTen). This approach models the output weight matrix W as being drawn from a statistical ensemble and aims approximate the TCB based expected values, specifically tar- geting the root mean square (RMS) norm of the Jacobian. The resulting formula connects êTCB model parameters like dimensionality and weight variance, as well higher-order moments of the output probability distribution o, offering potentially greater accuracy than simpler scaling laws, particularly when the output distribution is not diffuse (i.e., when Var is small). This statistical perspective contrasts with the exact, non-statistical expression derived in Section G. which applies deterministically to a specific instance of W and o. The quantity we aim to approximate is:
STCB
Jw F
where the key terms involved are: STCR: The Token Constraint Bound state space Rdd
Section 2.4), representing a distance in the hidden
26

---

Published as a conference paper at ICLR 2026
0: A fixed perturbation threshold, representing the target L2 change in the output probability vector o. This is typically treated as a dimensionless small value (e.g., 0.01). Jw IR Xxd, Jacobian of the softmax output probabilities with respect to the hidden state h. Its Frobenius norm |Jw has units of (probability change)/ (hidden state unit).
|| || The Frohenius
of matrix.
0 (v)) E R.: The vector of softmax output probabilities, Σn 1,
02 0.
S& Σ
i= of: The k-th moment sum of the output probabilities. Note S =.
Ven 1/52: The effective vocabulary size, a measure of distribution flatness (strictly
v(2))
h E Rd. The hidden state vector (embedding dimension d).
W E Rxdd The output weight matrix (embedding matrix).
a²: The assumed variance of the elements of W under the statistical model. This is parameter of the theoretical model. When applied to real model, it might represent an empirical estimate of the variance relevant the specifie forward pass training state.
V: The size of the vocabulary.
As established in (4), the Jacobian is: Here, the matrix M E R vxV is symmetric and depends only on the output probability vector o. It encapsulates how changes in the pre-softmax logits Wh translate into changes in the post- sost-softmx probabilities
Step 1: Recap the Jacobian Matrix Jw.
Step 2: Introduce Statistical Assumptions on W. The core of the statistical approach to model the output weight matrix W not fixed, trained entity, but random matrix drawn from a simple ensemble. We make the following simplifying assumptions about its elements Wfe:
E E JJ] - and E[ = o² (i.i.d.).
(12)
These imply E[W Wm]] = o² 851ōmm- The zero-mean assumption simplifies calculations. The constant variance o² captures a typical scale.
Caveat 1 (Model Simplification): Realistically trained W matrices possess significant structure (e.g., semantic clusters, non-zero mean after layer normalization, varying variances per token, cor- relations between rows w; w,). This i.i.d. model ignores such structure. Specifically, it implies E|WW do assuming orthogonality between rows on average, which might not hold em- pirically. For enhanced accuracy, one might incorporate an empirical covariance term Emp such that WW x do Eemp. though this complicates the derivation.
Note on r²: o² should be interpreted either fixed parameter of the theoretical ensemble, or, when connecting to a real model, as an estimate of the empirical variance of weights relevant the context (e.g., measured during the specific forward pass, possibly after normalization layers).
Step 3: Approximate the Squared Frobenius Norm via Expectation. We want to estimate [Jw 1| = | p. Under the statistical model, [ [ is random variable. We approximate by its expectation Ew [|
Ew J[ 11::1 Ew
WW))
Ew
(T(MW
Ew
(TrM MJ (using (AB) B AT)
Ew
(TrMMWW M))
(since M symmetric)
(13)
Ew
MyWWasWuMii <j/=1k=1
ΣV
(Trace expansion)
(14)
Here introduce the first major approximation:
Approximation (Decorrelation): The matrix elements M,, 8,y0; 0,0, depend on softmax (W)), which itself depends on W. Therefore, M is correlated with W. We make the strong approximation that this correlation can be ignored when computing the expectation involving quadratic terms of W, effectively treating M as constant with respect to the expectation Ew[:]-
Ew[]Jw [1:]] R Tr (MEw WW JM)
(Approximation Applied)
(15)
27

---

Published as a conference paper at ICLR 2026
introduce bias. The correlation is likely non-negligible if dimensions d or are small, or if the distribution o is highly peaked (low Vett, where small changes in W affecting the top logits significantly alter M). The error magnitude requires careful analysis (e.g., via perturbation theory or empirical validation in problematic regimes).
Caveat 2 (Decorrelation Risk):
This approximation E|MAM] R MEJA|M
Now, we use the statistical property from (12). The (i,l)th element of Ew [W]] is:
(Ew[WW Dist = Ew((WW )l] = Ew Ei Wix = Ei= Ew(W,s Wia = E 6,g dz Therefore, Ew [W I Substituting this into (15):
Ewl[Jw R (y)
= do MT-) (Since M = M)
= - (Definition of [M TH(MM²))
(16)
This indicates that the expected squared norm of the Jacobian is approximately proportional to the squared norm of the probabiltyyddeeendent matrix M, scaled by do This connection relies criti- cally on Approximation 1.
Step 4: Use the Exact Squared Frobenius Norm of M. The calculation of [M] Tr(M²) is deterministic once o is known. As derived in Section G, the exact value is:
M = Σ of -111- Σ ((,00))2
(17)
itj
= Σ(o (of - 20 + o + Σ 0,00
(18)
(S2 2Sg + 84) + E Σ of) - E(0))
(19)
= S2 2S + S1 + S2 - S1
(00)
= S2 253 + S2.
(21)
This expression relates the norm of M directly to the second (S2) and third (Sa) moments of the output probability distribution. Note that this step involves no statistical approximation itself.
Step Approximate the Jacobian Norm using RMS. We now introduce the second key approx-
imation:
Approximation 2 (RMS Substitution): We approximate the actual Jacobian norm [[Jwllr ||r for a specific W by its root mean square (RMS) value under the statistical ensemble model:
Jwwle / /EWlll J[]
(22)
This relies the assumption that the random variable |[Jw concentrates around its RMS value (related to concentration of measure phenomena, plausible for large d or V). However, it remains an approximation. By Jensen's 's inequality, v E[ 2 E[X], so the RMS value typically overestimates the expected norm. The magnitude of this overestimation (and the validity of concentration) depends on the variance of [Jwll which might be large if the distribution of norms is heavy-tailed (e.g.. due to specific weight structures or sparse outputs). More rigorous analysis might require bounding or using concentration inequalities (e.g., Hanson-Wright), which is beyond the scope of this derivation.
Substituting the result for the expected squared norm from (16) and the exact expression for |M] from (21) into (22):
Jw || Rs v do |d M v do 2Sg + +
(23)
This equation provides our refined statistical approximation for the Frobenius norm of the Jacobian.
Step 6: Refined Approximation for δTCB. Using the definition êTCB c/l|Jw |F and substitut- ing the RMS approximation for the norm from (23), we obtain the refined statistical approximation for the Token Constraint Bound:
STCB
(24)
/do 2S2 2S + S
28

---

Published as a conference paper at ICLR 2026
This is the main result of this section. It provides estimate for dTcB based the model's 's embed- ding dimension (d), the assumed variance of output weights (o²), and the second (S2) and third (Sa) moments of its current output probability distribution (o).
Step 7: Connection to the Simpler Verr-basee Approximation. A commonly cited simpler ap- proximation for ÒTCB relates it directly to the effective vocabulary size Ven 1/S2, often presented as бTCB 2 <v VeRt/ (do²). We see how this arises from (24) by introducing an additional approxi- mation:
Approximation (Diffuse Distribution): Assume o is sufficiently diffuse, meaning Pmax max, o,11, corresponding large effective vocabulary size, Ven > 1. Under this condition, higher moments S₃ become small relative to lower moments. As argued in Section J ((51)), if o, are roughly uniform over Ven tokens (o, 1/Ven). then S3 S2/Velt and S S2/Vett Thus, for large Ven. - 253 + S 1S2.
This approximation S2 2S3 + SS R S2 might fail the distribution is not sufficiently "uniform- like" even if Vetr is large (e.g.. heavy-tailed distributions where few moderately high probabilities contribute significantly to S3). Subject to this approximation:
S2 - 2S2 + $ 2 S2 (Approximation 3: Diffuse distribution, Voft > 1).
(25)
Substituting into (24):
Volt do
бTCB ~
=
Vdo2S2
do /dot/Volt
This shows the simpler Var based formula is special case relying on both the statistical model and the diffuse distribution assumption.
G EXACT, NON-STATISTICAL EXPRESSION FOR Jwwlle AND ITS RELATION TO WEIGHTED VARIANCE
This section presents the derivation of the exact mathematical expression for the squared Frobenius norm |Jww I|2 for specific, given output weight matrix W E R and hidden state h E Rd (which together determine a specific probability vector o softmax (W))). This derivation is purely algebraic and deterministic; it does not rely on any statistical assumptions about W being drawn from a random ensemble. We establish the correct formula and clarify its relationship to, but distinctness from, the trace of the probability-weightd covariance matrix of the embeddings. Notation Recap:
Step 1: Squared Frobenius Norm as Sum of Squared Row Norms. The squared Frobenius
norm is the sum of the squared Euclidean norms of its rows. Let Jw (i.:) E R xxd row of the Jacobian Jw.
denote the i-th
V J w (i.:)
J w = Σ
(26)
1=1
29

---

Published as a conference paper at ICLR 2026
Step 2: Derive the Expression for Jacobian Row. We need the components of the i-th row,
(i,): [[ww]...... Well Recall Jw = A(o) W. The (i, k)-th element is:
(Jw) Σ Au W
j=1
> (ō1j0) W,0
(Definition of Asy)
Wo of + -
(-0,,)
(Splitting sum: and + i)
= W
Oi ik
+
Σ 371
o, Wik
Oj Wi
(Recombining sum)
1=1
<We
og w)
j=1
oo((w:)) (Hmm) o;(w; pmm)
(27)
Here, Wjx = (w,)s is the k-th component of embedding wy, and (E)- 0g Ws,) is the k-th com- ponent the mean embedding Pw.- Thus, the entire i-th row vector (transposed match w; Puo as column vectors) is:
(w:),)) o, pw).
(28)
This shows that the i-th row of the Jacobian represents the deviation of the i-th embedding from the mean embedding, scaled by the probability oμ-
Step 3: Substitute Row Norm back into Frobenius Norm Definition. Using the row expression (28) in the definition (26):
Jww 11
J Jw (i, (1,
Σ |lo(ww; mmw T
V
Σ
of
M
(Since o is scalar).
(29)
This is the exact, non-statistical expression for the squared Frobenius norm of the Jacobian matrix.
(h) of
(30)
Step 4: Introduce the Trace of the Weighted Covariance Matrix. A related quantity often considered is the trace the probability-weighee covariance matrix of the embedding vectors: Its trace represents the total variance of the embedding vectors, weighted by the probability distribution
30

---

Published as a conference paper at ICLR 2026
Tr|Cov. (w)] Tr Σ ow (w))(w pmw) 17
Σ 0i Tr [wwi - Hww) pmw)
(Linearity of trace)
Σ oz(w0; )cr) (w: pww)
(Using Tr(ab") = bTa)
i=1
0i||
lw;
Me
(31)
This can also expressed using the variance identity:
V Σ
Tr TroCovo (w)] = E. [|
Mw 112
Σ or
of
w
(32)
Step 5: Comparing the Jacobian Norm and the Covariance Trace. squared Jacobian norm (30) with the trace of the covariance matrix (31):
Let compare the exact
Jww 1 Σ
of w.
(33)
V
Tr[Cow (w)) = Σ
oo|l
0:
M
(34)
im1
Step 6: The Exact Expression for Òren. Using the correct formula for the squared Jacobian
norm (30), the exact Token Constraint Bound is:
ÒTCB
(35)
[ww
VI of l|w. P 1
This is the ground-truth value for 8TCR given specific W and h (determining o).
H RELATING THE STATISTICAL APPROXIMATION AND THE EXACT EXPRESSION
This section elucidates the precise relationship between the refined statistical approximation for [JwllF (derived in Section F) and the exact, non-statistical expression for [[wll (derived in Section G). Understanding this connection is crucial for interpreting the validity and limitations of statistical approach and for appreciating why it can serve as useful, interpretable model despite its simplifying assumptions.
Recap: The Two Formulas for I|.-
1. Refined Statistical Approximation Seection F): This approach models W as random matrix. The core result approximates the actual squared norm by its expected value under the statistical model:
Ew Jw | = 2 M do 28, + S3).
(36)
where M = diag(o) oo o² is the assumed variance of W Wjk, d is the embedding dimension, and S. of. The final TCB approximation (24) uses the root mean square (RMS) value derived from this expectation: JJw||
[Ew[]Jwll [
For a specific, given W and o, the squared norm is calculated deterministically using the geometry of the embeddings and the probability distribution:
Exact Expression (Section G)
J w 1 = Tr(Covo (w)) = E. [[|]|]]] E [w]]
1 - pw 112
(37)
31

---

Published as a conference paper at ICLR 2026
where w; is the i-th embedding vector (row of W), Mww is the trace of the probability-weighee covariance matrix of the embeddings. The exact TCB (35) uses the square root of this value.
Eo [w] = , and Tr[Cov, (w)]
The Fundamental Connection:
Expectation Bridge. The theoretical link between these two ex- pressions lies in taking the expectation of the exact squared norm (37) over the statistical ensemble assumed for W. When we apply the statistical assumptions ((12)) and the key decorrelation ap- proximation (Approximation from Section F, see (15)) to the exact formula, we recover the core quantity from the statistical approximation:
Ew
Jw
= Ew (YA(O) A(o)) ~ do
(38)
Exact value from (37)
Statistical result (36)
Let's briefly trace why this works. The calculation performed in Step 3 of Section F (starting from Ew (WwTTTWWW' M))) effectively computes the expectation of the exact squared norm, represented in the trace form old. The crucial step is applying Approximation 1:
Ew (TM(MWWW M)] ~ Tr(MEw [WW MM).
This approximation treats M = diag(o) - oo as fixed when taking the expectation over W. even though itself depends on W via the softmax. Using Ew [WW Ii = Tr(([w,,))))
Σg ElW W Σx dege do dig. we get Ew!WW! yields:
do Substituting this
(y)M do Tr(M²) = [|
This confirms that | M (s, 252 + +3) is indeed the result of calculating the expected value of the exact squared norm under the statistical assumptions and the decorrelation approxima- tion.
Role of Approximations Revisited. The difference between the final бтCв value predicted by the statistical approximation (24) and the exact value (35) for a specific model arises precisely from the approximations inherent in the statistical derivation:
Statistical Ensemble Model for W: The actual trained weight matrix W treated as a typical re- alization from a simple statistical ensemble (i.i.d,, zero-mean, variance o² elements, see (12)). Real trained matrices have complex structure (correlations, non-zero mean, varying variances) not cap- tured by this model. The approximation's accuracy depends on how well the ensemble captures the properties relevant to the norm calculation (specifically, the second moments involved in WW T). Decorrelation (M and W): The calculation in (38) relies treating the Sabbabilitydeppeddee matrix M as approximately uncorrelated with the quadratic terms in W (like WWT) when taking the expectation Ew (Approximation 1, (15)). This approximation ignores the feedback loop where W determines O, which in turn determines M. It may be justified by averaging effects in high dimensions (d, V), but it introduces deviation from the exact expectation.
RMS Approximation ([JJw || x= / /E JE I The final step in the statistical approxima- tion replaces the actual norm | w for the specific W with the ensemblc-averaged RMS value (Approximation (22)). This assumes that the norm of the specific matrix is close the average norm across the ensemble, relying concentration of measure phenomena. While often plausible for high-dimensional random matrices, the actual norm can deviate from the average.
In summary, the refined statistical TCB approximation (24) estimates the TCB by replacing the exact Jacobian norm ||wwlp with / (S2 25g + 53), which represents the approximate RMS norm expected under the simplified statistical model and decorrelation assumption.
I DERIVATION OF THE EXACT JACOBIAN NORM FORMULA
In this appendix, provide detailed step-by-step derivation of the exact mathematical expression for the squared Frobenius norm of the output Jacobian matrix, [ II. This derivation is per- formed for a specific instance of the output weight matrix W E Vxd xd and the hidden state h E Rd which together determine the output probability vector softmax/ Wh). The derivation relies solely on the definitions of the Jacobian and the Frobenius norm, requiring no statistical assumptions about W.
32

---

Published as a conference paper at ICLR 2026
The final result relates J || to weighted of squared distances involving the output bedding vectors. We also clarify its relationship to the trace of the probability weighted covariance matrix of the embeddings, concept central to related analyses but distinct from the Jacobian norm itself.
Step 1: Definition of Squared Frobenius Norm via Rows. The squared Frobenius norm of any matrix is the sum of the squared Euclidean norms (Z2 norms) of its rows. Let Jw (i,:) denote the i-th row vector of the Jacobian matrix Jw.
Jw (1)1 = Σ (1.:)22
(39)
==1
Step Jacobian Definition and its Relation to Weights. Recall the Jacobian matrix given by do Using the chain rule, we can write:
Jw äh
do dz
(diag(o) - 00 J) A(o)W.
W Oz 0h
The element Jww represents The
So,,
Step 3: Explicit Calculation of the i-th Row of the Jacobian. We aim to find the vector (i,
(Jw) (JJ)as )a (8:1........ (Jw)xt )al The component (Jw), is the (i, k))th element of the matrix product Wo
(Jw)ik=
A,
1=1
V Σ - 000;) j=1
(Definition of Aij)
(o, + Σ -o,o;)W,
iti
o. 0: Σ
03 jk
y
oW - 0i
Σ
oj WjiKk os Wi
(Completing the sum over j)
j=1
= o - 0j
0j jk
+ of
j=1
Let's restart the derivation for (Jw) )ak more directly:
(Jw)is Σ
An
Wg
1=1
AnWi
Σ
WijWan
(Separating diagonal term)
j$i
= =0 W + Σ (-o,,o,)W,
(Substituting Ai values)
ifi
WW - o, Wi Σ Woo,WBs 5
jti
= oj
oiWik + Σ W
ttt
Wik 0z
Σ
0j Wjk
(Recombining sum)
(40)
1=1
33

---

Published as a conference paper at ICLR 2026
Now, let's interpret the terms. Wis is the k-th component of the j-th embedding vector wy, Let W0 denote this component.
Σ
0g W
Σ ojWj,
ojwj
(phw)
where (Ww))s is the k-th component of the mean embedding vector pwo- Substituting this back into (40);
(Jw) jk or W o;(Mw)) o.WW.. (frw))) oi(w; MwD)
(41)
This expression gives the k-th component of the i-th row of Jw. Therefore, the i-th row vector itself is:
(i,:) (, mw)
(42)
This shows that each row of the Jacobian is the deviation of the corresponding embedding vector w. from the mean embedding Mws, scaled by the probability oz.
Step 4: Substitute Row Norm back into Frobenius Norm Definition. Now we substitute the expression for the i-th row (42) back into the definition the squared Frobenius norm (39):
J Jw(hl]
Jw (i.:
10, (ww) 1
Σ of [. Mw (Since o, is scalar).
(43)
This equation provides the exact, non non-statistical expression for the squared Frobenius norm of the Jacobian. It is determined by the squared distances between each embedding vector w, and the mean embedding fww, weighted by the square of the corresponding probability of.
JJw (1)
Σ
o pw 12
(44)
Step 5: Relationship to the Trace of the Covariance Matrix. The trace of the probability- weighted covariance matrix of the embedding vectors is closely related but distinct concept. The covariance matrix is defined as: Its trace is:
Tr Tr (w)) = Tr
o oww (w)(w: (ww Mww
0i Tr [[wi (ww) pmw)
(Linearity of trace)
Σ o:(w; mmm) (w:: pw)
(Using Tr(ab") bTa)
Σ
0j pl|lw; Mm
(45)
1=1
As derived previously, this trace also equals the variance identity:
Tr(Covo (w)) = R., [[] II (w)] 112 =
Σ 0z w
12
Σ
(46)
This quantity, Tr[Cov. (w)), represents the total variance of the embedding vectors, weighted by the probabilities o;.
34

---

Published as a conference paper at ICLR 2026
Step 6: Comparing Jacobian Norm and Covariance Trace. Comparing the derived exact
squared Jacobian norm (43) with the trace of the covariance matrix (45):
Jww 1
w; pw
(47)
Tr|Covo (w)] =
Σ oz o.llw; - Mow 11
(48)
These two quantities are clearly different, differing by factor of in the weighting term inside the sum. While both measure aspects of the dispersion of the embedding vectors relative their mean Mw,, they not mathematically identical. The Jacobian norm gives more weight (via of) to the deviation High-probabbiliy embeddings from the mean.
DERIVATION OF
TCB BEHAVIOR
In this appendix, we provide detailed mathematical derivations supporting the regime-dependent behavior of the Token Constraint Bound (δTCB), as discussed in Section 3 and observed empirically Table 1. We analyze the behavior of θTCB </ J/IJw (h)llr in two distinct regimes based on the flatness of the output probability distribution
high flatness (large effective vocabulary size, Vett > 1) and low flatness (highly peaked distribution, Vei & 1).
The key is to understand how the squared Frobenius norm of the Jacobian, |[Jw (h) [ behaves in these limits. We will use different but related expressions for the norm depending on the regime: the statistical approximation for the high- Vest regime and the exact formula for the low-Ven regime. Recap of Key Formulas:
Exact Squared Norm (from Section I, Eq. (43)):
Jww (h)
0 wu M
(49)
where w: is the i-th embedding vector (row of W), o = (o oy) is the probability vector, and pho = E/= 0yWy is the probability-weighted mean embedding.
2. Refined Statistical Approximation (from Section F, Eq. (23)):
[ww(k)l se /doz(S> 2S3 + 52)
(50)
where d is the hidden dimension, o² is the assumed variance of weight elements Wj and S₁ = E, of is the k-th moment sum (Ver =1/S2). This approximation relies on modeling W statistically and approximating the norm by its RMS value.
J.1 Assumptions. In this regime, the probability distribution is spread relatively evenly across many tokens.
REGIME 1: HIGH FLATNESS (LARGE Vott >> 1)
Volf 1/S2 is large.
Consequently, the maximum probability Pmax = max; o; must be small (Pmax11). Roughly,
if probabilities are spread over Var tokens, then o, 1/Vear
We use the refined statistical approximation (50), which implicitly assumes the statistical model for W (i.idd elements, zero mean, variance σ²) is a reasonable proxy for average behavior.
Approximating the Jacobian Norm. We analyze the term |M S2 -2S S2 within the statistical approximation (50). Since o,11 for all i, higher powers of og are much smaller. Let's 's assess the magnitude of the terms relative Sy:
S2 Σ, o² = 1/Vett.
Sg = 2. < (max, 0j) Σ; of Pmax S2. If o; ~ 1/Vott, then S3 ~ Vett x (1/VoH)²
1/V2 = =S2/Vmn
53 (1/Ve)) S2 V.n-
35

---

Published as a conference paper at ICLR 2026
Thus, both S3 and $2 are smaller than S2 by factor of approximately Ven- Since we assume Ven > 1, the terms- -2g and + become negligible compared to S2:
|M 3 S2 28, + S2 S2 (for large VeH).
(51)
This simplification corresponds to Approximation discussed in Section F. Substituting this back into the statistical norm approximation (50):
||w F (S)
(52)
do²
V
Vett
(53)
This expression predicts that the Jacobian norm decreases as Veif increases.
Behavior of бTCB. Using the definition ôTCB e/l]Jw/l F and the approximation (53):
Voft
8TCB
(54)
(do=/Vett
do
Conclusion (High VoH): In the high-flatner regime, δTCn is predicted to be approximately propor- tional to the square root of the effective vocabulary size:
ÔTCB X v Vaft (for Vart >> 1)
This provides clear mathematical basis for the strong positive correlation ra,Var R 0.95 observed empirically (Table 1) in diverse datasets where distributions are often flat.
Negligible Influence of Margin 2 - 2j-. The margin 24 2g- - 2y- between the logits of two specific tokens & (usually the top prediction) and j* (usually the top competitor) directly affects OR and 0j-. In the high-Vott regime, however, both Og and oj- are typically small (e.g., ~ 1/Vet). Changes 2ạ - primarily redistribute small amount of probability mass between these two (and possibly nearby) tokens. The impact the overall sum S2 = Σ of which aggregates contributions from many small probabilities, is minimal. Consequently, changes in 2 2j- have a very weak effect on |Jw || via (53), and therefore also on êTCB. This explains the near-zero correlation =2, observed in diverse datasets (Table )..
J.2 REGIME 2:
LOW FLATNESS (SMALL Votr 2 1)
Assumptions. In this regime, the probability distribution o is sharply peaked on single token. Volf R 1. This occurs when one probability, say og, is close to 1.
Let 0g = 11 c.. where é, fz Oj is a small positive quantity (e,l1).
All other probabilities 0j (j + k) very small, typically 0, O(ex) smaller.
We use the exact expression for the norm (49) as it directly captures the influence of the spe- cific dominant embedding wy and its relation to competitors. Statistical averaging inherent in (50) is less appropriate here.
Approximating the Jacobian Norm. We analyze the exact sum J 11 ='E of First, approximate the mean embedding uw:
w.
M 11.
Σ j=1
0, = 02 +
Σ
o
(55)
= (1 - (1-6,)wr w2 + Σ
(56)
jfk
Wk
w/ + Σ
(57)
jy
wk
wx)
(using 6₂ =
Σ 0,)
(58)
J
This shows am is close to Wg. differing by terms of order Oce,). Now, consider the terms in the sum for [Jw 1::
36

---

Published as a conference paper at ICLR 2026
Term for k: We need ||
w
Mw 122 From (58):
we
Hw
Σ j
0y(wn w)
The squared norm is 10 - - = Σj 0j (w || Since each oj - O(ea) for
j + k. this squared norm is O(e.). The contribution to the total sum is og || Mu 11 R
(1 - e, ((())) ~ O(e)). This term is therefore negligible to the leading order.
Terms for i + k: We need I|w, M 11 Using (58):
W0: Po wt
Og( wik)
JØk
(w:
Σ jtk
Oj 0y(wy - wwk
The squared norm is:
Iw.
1wwl
(w, w6k - Σ
0y(Wg wkk
jph
Expanding the square:
[w: Mw 11 w.
wv wx 10111-(w: wx)
oo,(w, - wk)
+
Σ 53k
ooy wx)
13th
The middle term is (c,)), and the last term is Oc2)).
Thus, leading order:
[w: - Mw l12 = l|w; wy (ca)) (for i + k)
The contribution of the i-th term (i + k) to the total sum J ||? is of o - w 1w1.2 Since
of O(e)), the contribution is: of w. u 12 of (Ilw; mma11 O(ss)) of1lu - w 0c())
Summing the contributions for all i:
Jwll = 12 Mww
+
of 1/w0 M 11
x0(e3)
j
wy +
02 - w (keeping leading order terms, O(e)))
(59)
This approximation reveals that for highly peaked distributions, the squared Jacobian norm is domi- nated by the sum of squared distances between the dominant embedding wy and competitor embed- dings Wj, weighted by the square of the competitors' small probabilities oz.
Connecting to Margin 2 Zy-. The logit margin between the winning token and any other token j is 2 2g- = 2k 2z- The probability o, for # l can be approximated using the softmax definition when zµ is large compared to others:
=
= - -(x-22) 22-
0j EL c + Eift ea
eth e*t (1 Σ Eijt 601
e
This approximation holds because Eiyz =0- = Eie 01/00/0 RS ea/(1 ex) ell Substituting this into the norm approximation (59):
Jww 1
2 2
wwll
2n- [wj wx
(60)
jzk
The specific margin defined as Zz. 2 Zj., where j is the top competitor (highest logit 2j among j # k)), corresponds to the term with the largest e 22,-1 = (smallest 24 - -2--) ) in the sum, which often dominates the sum's value.
37

---

Published as a conference paper at ICLR 2026
Behavior of ÒTCB. As the margin 2 2y- increases, the corresponding 2 2n- for the closest competitors also increases. This leads to an exponential decrease in the terms e 20 in (60). Consequently, |Jwll decreases strongly (exponentially) as = 2j- increases. Since êTCB e/IJwlll ||, an increase in 2 2j- causes decrease in the denominator, leading to increase in ÒTCB. Conclusion (Low Vent): the low-flatness regime, TCB increases rapidly as the logit margin
2k 2j- increases:
ÒTCB
Increases strongly with 2/. 2z* (for Ven Re 1)
j 2(xx=2;) 1|,, wv l
This derivation provides the theoretical underpinning for the strong positive correlation 25, 0.62 observed empirically in the low-Venr data subset (Table )).
Negligible Influence of Residual Vetr Variation. In this regime, Vett is already close Small changes in the probability distribution (e.g., caused by changing 2 2j ) lead to minuscule changes Voft. Specifically, Voi = 1/S, = 1/(o Ejjt 03). As Og 1 and oj = O(s)), we have Vett 1/(1- 2e, + + While changes in 25. Zj- affect e, and thus cause small fluctuations in Ven, these variations are vastly outweighed by the direct exponential impact of 2g 23- on |[ ||? via (60). This explains why the correlation Va, drops nearly zero (+s 0.08) in the low-Vett regime.
38


Reviews:
[{"review_id": "4lB6kiyBeJ", "reviewer": "Reviewer_T26t", "summary": "This work introduces a measure called Token Constraint Bound that measures how much changes in the internal representation an LLM can withstand before the token prediction changes. The goal is to measure this in additional to evaluate metrics to measure robustness citing examples that the models’ output change a lot depending on the prompt formatting, context etc.  This work derives TCB from the Jacobian of the softmax output with respect to the hidden state and show that it is intimately tied to the geometry of the output embedding space. They also give theoretical insights into their metric and provide experiments which corroborate that this metric aligns with effective prompt engineering and in-context examples and identifies brittle predictions missed by perplexity.", "questions": "- How do you expect the findings to change depending on the model size? Moreover, it seems like the values of TCB would depend on the model family and hence, it would be hard to tie a value with stability across various models.\n- The authors mention “The Low-Veﬀ Targeted Dataset (LVD) was created by modifying DPD prompts to generate high-confidence predictions.”. Can the authors please elaborate how these were created? \n- Also, the authors mention ‘We synthetically manipulated W (clustering/dispersing competitor embeddings) while holding h and o (thus local PPL) constant for diverse MMLU prompts.” Can the authors please clarify how they manipulated W?\n- The authors use ~300 prompts for computing their results. Can the authors please comment if these were enough and how much was the variation across runs?\n- How would their method apply to prompts where the answer is not a couple of words?\n- In table 3, the authors measure variance of accuracy. What is the randomness here in these experiments?\n- What temperature is used here for the experiments and how would different sampling techniques affect the results?\n- In table 3, the authors mention that they select prompts based on accuracy and TCB? What is the exact criterion that they use because later, they mention that prompts with very high TCB can be confidently wrong.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- The connection of TCB to the geometry of the output embedding space is quite interesting. \n- Their method can be used to select prompts and in-context examples which have higher accuracy as well as robustness. This approach seems novel to me and would be of interest to the community.", "weaknesses": "- This work considers that the output probabilities should not change by epsilon. I am not sure why we need a fixed epsilon because if the model is already very confident, then this epsilon can be large. Hence, instead of being a constant, this should depend on how confident the probabilities are.\n- Also, it seems like this bound could be very loose in practice.\n- The experiments are only limited to one model Llama 8b.\n- Overall, I feel the experiments section is weak as many details are not clearly presented (see questions below).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bB8nSnA2hk", "reviewer": "Reviewer_iYJA", "summary": "This paper introduces a novel metric called the Token Constraint Bound ($\\delta_{TCB}$) to quantify the local stability and robustness of llms next-token predictions. The authors argue that conventional metrics like perplexity (PPL) fail to capture the true resilience of the model's internal state to minor perturbations. $\\delta_{TCB}$ is defined from a \\textit{geometric perspective} in the output embedding space as the maximum internal state perturbation an LLM can withstand before its dominant prediction significantly changes. Experiments demonstrate that $\\delta_{TCB}$ correlates with effective prompt engineering and uncovers critical prediction instabilities missed by PPL, offering a principled and complementary tool for analyzing and improving LLM prediction reliability.", "questions": "1. $\\delta_{TCB}$ is primarily a local metric focusing on the next token's stability. How do the authors propose to generalize or extend this local measurement to assess the overall stability of an entire sentence, paragraph, or long-form generated result? For multi-step reasoning tasks, how can this local metric accurately reflect global robustness?\n2. $\\delta_{TCB}$ is currently an evaluation metric. Is it possible to integrate $\\delta_{TCB}$ into the model's \\textit{training objective} or fine-tuning process? For instance, could a loss function be designed to explicitly \\textit{maximize $\\delta_{TCB}$} to train a model that is inherently more locally robust?\n3. $\\delta_{TCB}$ is grounded in the geometry of the \\textit{Output Embedding Space}. What is the relationship between $\\delta_{TCB}$ and the stability of other critical internal components, such as \\textit{attention weights} or the outputs of intermediate hidden layers? Can an analysis of $\\delta_{TCB}$ across layer depths help pinpoint the most fragile or robust architectural components within the model?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The core contribution, $\\delta_{TCB}$, is highly original. It shifts away from traditional probability-based evaluation by proposing a geometry-based metric that quantifies stability as a “safety margin,” providing a novel and interpretable theoretical perspective on LLM reliability.\n\n2. The metric is clearly defined and theoretically supported by its intrinsic link to the geometry of the output embedding space. Experimental results effectively validate its use as a complementary tool that can identify prediction vulnerabilities that perplexity fails to capture.\n\n3. The paper clearly articulates the motivation for the metric, highlighting the limitations of PPL in assessing local robustness. The definition of $\\delta_{TCB}$ and its connection to geometric principles are well-explained.\n\n4. $\\delta_{TCB}$ holds significant potential for real-world impact. It can serve as a quantitative measure of \\textit{contextual effectiveness}, guiding the development of more robust prompt engineering techniques, and is a valuable addition to the field of LLM reliability and risk assessment.", "weaknesses": "1. $\\delta_{TCB}$ exclusively measures the stability of the predicted token, without incorporating semantic meaning or factual correctness. This is a critical limitation as the metric cannot distinguish between a \"stably correct\" and a \"stably incorrect\" prediction. Its interpretability must always be tied to traditional accuracy metrics.\n\n2. $\\delta_{TCB}$ is a geometric distance metric, but the model's prediction behavior is governed by the Softmax function. Changes in the Temperature parameter can drastically alter the prediction probability distribution. The authors must clarify whether $\\delta_{TCB}$ is independent of Softmax temperature or detail how temperature variations affect the correlation between the metric value and empirically observed robustness.\n\n3. The exact calculation of $\\delta_{TCB}$ involves finding the maximum perturbation radius by identifying the closest \"competitor\" token in a high-dimensional space. For LLMs with massive vocabularies, the time complexity and computational overhead may be substantial. The authors should discuss the efficiency bottlenecks in practical applications and explore or propose more efficient approximation algorithms to enhance its practical utility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7aLPTOuN8q", "reviewer": "Reviewer_tgxF", "summary": "This paper introduces the Token Constraint Bound, a metric that quantifies the maximum internal state perturbation an LLM can withstand before its primary next-token prediction changes significantly. The authors' experiments demonstrate δTCB’s utility in assessing prompt effectiveness and its ability to uncover critical prediction instabilities missed by perplexity. This evidence positions TCB as a valuable complementary tool for analyzing and potentially enhancing the contextual robustness of Large Language Models.", "questions": "1. **Practicality of TCB for Inference:** The computation of TCB requires a summation over the entire vocabulary *V*. For modern large models, the vocabulary size *V*  typically ranges from 32,000 to over 100,000. This implies that for the generation of *each* token, a complex geometric computation involving the entire vocabulary must be performed. This appears to be computationally prohibitive, potentially rendering TCB infeasible for any practical applications requiring low latency. The manuscript currently lacks a detailed analysis of TCB's computational cost, and I would appreciate it if the authors could provide this information.\n2. The current experiments are based solely on the Llama-3.1-8B model. I would like to understand the extent to which the effectiveness of TCB generalizes to models of different scales and architectures. I would encourage the authors to supplement their findings with experiments on a smaller model (e.g., in the 3B parameter range) or on models with different training provenances, such as Qwen or the Pythia suite, which is designed for such analyses.\n\nTo summarize, my primary concerns lie with **Weakness 1**  and **Question 2**. I would be willing to reconsider my score should the authors provide a satisfactory response to these points.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "This paper defines a novel metric, TCB, to evaluate the connection between a model's hidden state and its output instability. It provides rigorous theoretical proofs and derivations, which are substantiated by thorough experimental validation and data analysis. The establishment of this new metric is highly significant for the interpretability of large models and for prompt engineering. This work contributes to a research area that is both novel and highly promising.", "weaknesses": "1. The entire mathematical derivation of TCB is built upon the assumption of $L_2$ norm perturbations for the hidden state **h** and the Euclidean distance between output embeddings $w_i$. This is a very strong assumption. Semantic similarity in high-dimensional embedding spaces is not always well-captured by Euclidean distance; two semantically related tokens may not be proximal in terms of $L_2$ distance.  It is more likely that meaningful perturbations occur along specific semantic directions. TCB might be completely insensitive to these structured, non-isotropic perturbations. While I understand that the $L_2$ norm is a natural choice, I would like to see a more detailed explanation and analysis justifying this simplifying assumption.\n\n2. The paper compares TCB with perplexity (PPL), but the comparison against simpler, less computationally expensive uncertainty metrics is insufficient. This includes metrics such as the entropy of the output probability distribution and the difference between the top-1 and top-2 probabilities.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7cY2q1toRL", "reviewer": "Reviewer_Y3us", "summary": "The authors propose the Token Constraint Bound ($\\delta_{TCB}$), which is a metric that quantifies the amount the hidden states of an LLM can be perturbed without significant change in the output. They argue that $\\delta_{TCB}$ measures the stability of an LLM for a specific input using the internal geometry of the model. They show through experiments that optimizing for $\\delta_{TCB}$ allows for better prompt engineering and stability.", "questions": "- Do the authors test how large the approximation error in Equation 3? To me, the softmax nonlinearity makes it unclear that this approximation will hold consistently unless $W\\Delta h$ is very small relative to $Wh$\n- Could the authors clarify the context of the experiment in 4.3.1? Without knowing what the \"gsm8k_811\" question is, it is unclear what the the interventions are doing.\n\nMinor typo: spaces missing in lines 47-48", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- The paper is well-motivated. Quantifying the stability of predictions is a useful task for understanding the behavior of LLM predictions.\n- The application to prompt engineering is interesting and has high potential for practical applications.", "weaknesses": "While the motivation and use case of this approach are interesting, how this method compares relative to other approaches, both in formulation and in experimental results, is not clear from the paper. To me, positioning this approach in comparison to other robustness metrics is necessary.\n\n1. There is limited literature review on prior methods for evaluating the robustness of neural networks or LLMs. The authors discuss perplexity and confidence metrics, but there are many approaches related to quantifying robustness that are not discussed in the paper or appendix. Many of these approaches also rely on computing the response to perturbations, so it is not clear to me where the method proposed by the authors is positioned relative to these works. For example, [1] seems like a very related method computed for neural networks. \n2. The experiments provide limited baseline comparisons to existing methods. The only method that the authors compare to for prompt optimization is optimizing for perplexity. Even for this experiment, there is only a gain in worst-case accuracy and standard deviations are not reported. \n3. The experiments are only shown for a single model (Llama-3.1-8B). Additional evaluations for other models would strengthen the experimental results.\n\n[1] Sensitivity and Generalization in Neural Networks: an Empirical Study. Novak et al., 2018.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)