Bounds of Chain-of-Thought Robustness: Reasoning Steps, Embed Norms, and Beyond
Authors: Dingzirui Wang
Xuanliang Zhang
Keyan Xu
Qingfu Zhu
Wanxiang Che
Yang Deng
Venue: ICLR 2026 Poster (2026)
Keywords: Chain-of-Thought; Reasoning Robustness
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Existing research indicates that the output of **Chain-of-Thought (CoT)** is significantly affected by input perturbations.  
Although many methods aim to mitigate such impact by optimizing prompts, a theoretical explanation of how these perturbations influence CoT outputs remains an open area of research.  
This gap limits our in-depth understanding of how input perturbations propagate during the reasoning process and hinders further improvements in prompt optimization methods.  

Therefore, in this paper, we theoretically analyze the effect of input perturbations on the fluctuation of CoT outputs.  
We first derive an upper bound for input perturbations under the condition that the output fluctuation is within an acceptable range, and we prove that:  

- *i)* This upper bound is **positively correlated** with the number of reasoning steps in the CoT;  
- *ii)* Even an infinitely long reasoning process **cannot eliminate** the impact of input perturbations.  

We then apply these conclusions to the **Linear Self-Attention (LSA)** model, which can be viewed as a simplified version of Transformer.  
For the LSA model, we prove that the upper bound for input perturbation is **negatively correlated** with the norms of the input embedding and hidden state vectors.  

To validate this theoretical analysis, we conduct experiments on **three mainstream datasets** and **four mainstream models**.  
The experimental results align with our theoretical analysis, empirically demonstrating the correctness of our findings.

Published as a conference paper at ICLR 2026
BOUNDS OF CHAIN-OF-THOUGHT ROBUSTNESS: REASONING STEPS, EMBED NORMS, AND BEYOND
Dingzirui Wang Xuanliang Zhang! Wanxiang Che if Yang Deng²
Keyan Xu²
Qingfu Zhu'
'Harbin Institute of Technology SSinaapoe Management University {dzrwang, xuan Lnnghang, kyxu, qfzhu, car}0ir.. hit edu. cn ydengesmu. edu.:du.uueeeee...
ABSTRACT
Existing research indicates that the output of Srain-of-Thoughh (CoT) is signifi- cantly affected by input perturbations. Although many methods aim to mitigate such impact by optimizing prompts, a theoretical explanation of how these pertur- bations influence CoT outputs remains open area of research. This gap limits our in-depth understanding of how input perturbations propagate during the rea- soning process and hinders further improvements in prompt optimization methods. Therefore, in this paper, we theoretically analyze the effect of input perturbations on the fluctuation of CoT outputs. We first derive an upper bound for input per- turbations under the condition that the output fluctuation is within an acceptable range, based on which we prove that: (i) This upper bound is positively correlated with the number of reasoning steps in the CoT; (ii) Even an infinitely long rea- soning process cannot eliminate the impact of input perturbations. We then apply these conclusions the Linear as a simplified version of the Transformer. For the LSA model, we prove that the upper bound for input perturbation is negatively correlated with the norms of the input embedding and hidden state vectors. To validate this theoretical anal- ysis. we conduct experiments on three mainstream datasets and four mainstream models. The experimental results align with our theoretical analysis, empirically demonstrating the correctness of our findings.
(LSA) model, which can be viewed
INTRODUCTION
Chain-of-Thooee Thaough (CoT) is effective method that enhances the performance of large language models (LLMs) by prompting the model to generate a step-by-step reasoning process, thereby im- proving the quality of the results (Wei al. 2022). However, numerous studies have indicated that CoT is highly sensitive to input, where subtle input perturbations can lead to significant performance fluctuations (Zhao et al. 2024 Shi al. 2024b). To address this issue, researchers have proposed prompt optimization methods to enhance the reasoning performance of LLMs by refining the input prompt, lowering the effect of the input perturbation (Vatsal & Dubey] 2024, Sahoo et al. 2025). For instance, TextGrad (Yuksekgonul et al.) 2025) optimizes prompts by constructing textual gradients, while OPRO Yang et al. 2024) utilizes the LLM itself to iteratively generate more suitable prompts.
Despite this progress, key gap remains: most studies treat CoT robustness as an empirical phe- nomenon, with little theoretical understanding of why and how perturbations propagate through the reasoning process of LLMs, thereby affecting output fluctuation. Without such analysis, our un- derstanding of CoT robustness remains incomplete, and prompt optimization risks being limited to ad-hoc techniques. This motivates fundamental research question: what governs the CoT robustness of L.MS to input perturbations?
Following the previous work (Huang et al. 2025). consider CoT multistep iterative process, with the output of each step serving as the input for the next. Our theoretical analysis shows that
Work was done during a visit at SMU.
Corresponning Author.

---

Published as a conference paper at ICLR 2026
Table 1: The main findings and corresponding evidence and experiment of this paper.
Finding
Evidence
Experiment
More reasoning steps can reduce the effect of input perturbations.
Theorem
$4.3
The effect of input perturbations cannot be entirely eliminated by continuously increasing the number of CoT reasoning steps.
Equation
4.3
CoT robustness is negatively correlated with the norms of the input embedding and hidden state vectors.
Theorem
under the assumption of Lipschitz continuity QQ et al. 2023 Collins et al. 2025 longer CoT rea- soning indeed reduces the fluctuation of outputs input perturbations, but never fully eliminates them. Even with an infinite number of CoT steps, a non-zero robustness bound remains, suggesting that CoT inherently dampens but cannot completely neutralize perturbations.
To further ground our analysis, we investigate robustness in the Linear Self-Attention (LSA) model (Wang et a 2020a Zhang al., 2024a), which is commonly adopted simplified version of Transformer Vaswani et al.) 2017) for analysis without loss of generality. We prove that CoT ro- bustness highly depends on model-level factors: the sensitivity perturbations correlates negatively with the norm of the input vector and the hidden state vectors. Additionally, we discuss the impact of other factors in LSA on CoT robustness.
Finally, we validate our theory with experiments on four mainstream LLMs (Llama2, Llama3.1 1, Seessek-RI--issilled-Liaee
Qwen3) across three widely used reasoning datasets (MATH, MMLU-Pro, GPQA). The experimental results indicate that the variation in output fluctuation is consistent with the trends of the various factors identified our theoretical analysis, thereby vali- dating our findings. Furthermore, based on the analysis, we propose selecting the prompt by max- imizing the upper bound the input perturbation, which achieves consistent improvements over prior work, aiming to inspire future research in this area.
The main findings of our work are summarized in Table I and our main contributions are as follows:
We provide an upper bound for the output fluctuation with respect input perturbations under the assumption of Lipschitz continuity and prove that even an infinitely long CoT cannot completely counteract the impact of input perturbations.
Taking the LSA model as case study, we demonstrate that robustness to input perturbations is negatively correlated with the norms of the input and hidden state embedding vectors.
Our experiments across multiple mainstream datasets and LLMS validate our theoretical analysis, and improvements based on our analysis also enhance performance compared to existing prompt optimization methods.
ROBUSTNESS OF CHAIN-OF-THOUGHT
In this section, discuss the impact of input perturbations the model output. We begin by providing some fundamental definitions. Then, we derive the upper bound for the output fluctuation given the input perturbation when the model satisfies Lipschitz continuity. Afterward, we determine the upper bound for the input perturbation when the output fluctuation is within an acceptable range. All the proof of this section is shown in Appendix
2.1 PRELIMINARY
Let æ, y Rd denote the embedding vectors of the user query and the corresponding output, where is the dimension of the embedding space. Following previous works (Zhang ct al., [2224a] Von Os- wald et al. 2023). , we directly consider the impact of user query as a whole on CoT robustness. We also prove that the conclusion same with considering multiple tokens in Appendix D.5
embedding vector instead of token embedding for analysis,
Let E &d represent the input perturbation, and ă = x + 8 represent the perturbed input. Following previous work (Huang et al. 2025). model the CoT reasoning process as multistep iterative
2

---

fundamental assumption Published as a conference paper at ICLR 2026
procedure, where the output of each step serves the input for the next step. Let K E N+ be the total number of CoT reasoning steps, and let hy. E R denote the hidden state at step taking I as inpu["] Let f(h, : Rd x IR + I2 represent the mapping function to generate the hidden state corresponding to an arbitrary reasoning model. Thus, we have M f(0, x) and hR. = f(1 D)). We denote the output fluctuation caused by the perturbation 8 on step l as Ek- hg.5 hk.a-
2.2 UPPER BOUND OF OUTPUT FLUCTUATION
We primarily discuss the impact of input perturbations on the model output under the assumption of Lipschitz continuity. Lipschitz continuity imposes constraint the growth rate of the model output, preventing an explosive increase. Considering that the output of current LLMs typically exhibits stable changes, many analytical works adopt this condition
(Qi et al., 2023 Collins et al. 2025). Specifically, for a given bivariate function f(h, x), Lipschitz continuity requires the existence of constants C, y E IK such that:
Il 21) falei)))fhhhh.hh
f(h2, x2) S y //// hell + C||21 x2l1
Considering that the input h at each step is the output of the previous step, by substituting it and expanding recursively, we can derive the following theorem:
Theorem 1. If f is Lipschitz continuous with respect to constants C, E R as defined in Equation ! then for a given input perturbation E Rd, the upper bound of the corresponding output fluctuation EK of the final step K satisfies that:
C (1 (1--()))) 1181
|| S Ar +
1-7
where A max
|eell ||6
can observe that the propagation of the input perturbation be mainly divided into two parts. (i) The part contained in the hidden state vector: since the hidden state vector updated at each step, the coefficient of this part of the perturbation is continuously multiplied by the corresponding Lipschitz constant y; (ii) The part contained in the input vector: since the input vector each step does not change, this part of the perturbation gradually accumulates at each step, = - (1-*).
From Theorem
and thus the corresponding perturbation coefficient is Σ Cyf = (
Besides, considering that when the model is fixed, the corresponding parameters C' and y are also fixed. Therefore, based on Theorem [ the upper bound of the output fluctuation primarily depends on two factors: the number of reasoning steps K and the magnitude of the perturbation [| Based on previous work (Zhou al. 2020: Diehl Martinez et al. 2024), implies that for a well-trained model, the output fluctuation gradually converges to a fixed value rather than diverging infinitely. We also fit the values of y in Appendix [F.l Jusing practical datasets and models. Consequently, as the number of reasoning steps K increases, the corresponding upper bound of output fluctuation decreases, indicating that the increment of CoT steps can mitigate the impact of input perturbations on the model output.
assume that 7 1, which
2.3 UPPER BOUND OF INPUT PERTURBATION
In practical applications, model can tolerate certain degree of output fluctuation while maintain- ing the same final result. For example, in a classification task, as long as the probability of the same option remains the highest before and after the input perturbation, certain level of fluctuation in the output probabilities can not affect the final answer. Therefore, we assume there exists an acceptable boundary R E t such that we consider the output fluctuation be acceptable when the following condition is met:
|||| < R
In practical models, can be viewed as the input embedding vector, and h can he viewed as the enended vector from the last layer of the model.
3

---

Published as a conference paper at ICLR 2026
To ensure that the norm of output fluctuation is less than R, require the expression the right-hand side of the inequality in Theorem [ to be less than R, which yields:
R
|| -
It can be observed that the upper bound of the input perturbation is mainly influenced by R, C, and y. A larger It indicates that greater output fluctuation acceptable, thus leading larger upper bound for the input perturbation. Conversely, larger values of C' and according to Equation 1 sug- gest that the model is less capable of compressing the output fluctuation, implying weaker ability to handle input perturbations, which results in a smaller upper bound for the input perturbation.
Taking 7 < 1 and letting K - oo, we can obtaín that:
R(1 ((1-2)
|
C
Equation F Lindicates that the effect of extending the reasoning process to eliminate input perturbation limited. Even with an infinitely long reasoning process, the input perturbation exceeds a certain threshold, the model cannot eliminate the resulting output fluctuation. For example, if we "perturb" a numerical reasoning problem into a coding problem, the model cannot generate the answer to the original problem, regardless of the reasoning length.
3 CHAIN-OF-THOUGH ROBUSTNESS ON LINEAR SELF-ATTENTION
According to the discussion in 2.3 the upper bound of input perturbation that a model can toler- ate using CoT depends on the properties of the model itself. Therefore, in this section, we discuss the factors that influence the upper bound of input perturbation on the Linear Self-Attention model (LSA) (Wang al., 2020a; Zhang et al., 2024a), which can be viewed the current mainstream T Transformer architecture. All the proofs of this section are shown in Ap- pendix C We also discuss the influence of various non-linear factors in the Transformer on the conclusions of this section in Appendix D.I. and we analyze none-linear attention in Appendix D.3
simplified version of
3.1 DEFINITION OF LINEAR SELF-ATTENTION
We first define LSA following the previous work (Wang et al. 2020a Zhang et al. 2024a). Let WKQ, WPV E Rxb denote the combined query-key and projection-vale value matrices, and let p E R+ be the normalization factor. We denote the parameters as 0 (WKQ KQ W PV p). Let [h, x].
The LSA is then defined as:
fLSA x; = + WV E WKQE
(5)
LSA can be viewed as replacing the non-linear softmax mapping in single-layer Transformer with a linear mapping. Following prior work (Zhang et al., 2024a), we set in this paper.
Based on Equation 5 Zhang et al. 2024a) proves that for well-trained LSA the training data
{(h4, Fis N1) [.). J. its parameters must satisfy:
w KQ = [T>(T-
[-11 0g) 0)
of
W.PV = [T<(T=))2
0axd 0g
0f
(6)
where -= (1+ 1) A Tr(A) E IRdxd and A denote the covariance matrix of the training data. Substituting these optimal parameters into the equation yields:
N
0a
E
E
(Oaxd Oa 0
0
0
fLSA(h, E
E
Considering the gradient explosion without the residual flow, we introduce the residual coefficient 77 E (0, to LSA (Zhang et al. 2019 Bachlechner et al. 2020). The corresponding function is:
T~1
0,8
E
E
(Axxd 0g
0
0
f x; 0.)= = n/ +
(8)
07
1
4

---

Published as a conference paper at ICLR 2026
Next, use Equation 8 the prediction function (L))(h, discuss the effect of input pertur- bations on the LSA output.
3.2 INPUT ROBUSTNESS OF LINEAR SELF-ATTENTION
Based on Equation we provide the upper bounds for the Lipschitz constants in Equation
Lemma 1. If |||| < R2 and || v Rh, let = (T)(T--))-+ - Then we have:
n | R2
S n² R2 1 R
The assumption of R and R₁ Lemma cessively large embedding vectors can lead to unstable inference, the embedding vector norms in mainstream LLMs are typically confined within certain range (Fazlyab al. 2019. Kim et al.) 2021), making this assumption reasonable.
I bounds the norms of æ and h. Considering that ex-
By substitutiig the upper bounds of C and y from Lemma Lemma[iite Linto the Equation Theorem 2. If | < Rg and ||]]| R2 and let
3 we can obtain that:
a = [[>((T-)]
8 == --
8 a R
7 -=V
1/2 RR 222 R2:
With > such that |eal < A||ōl|, the certified tolerable input-perturbation radius of the LSA map for the reasoning step K N+ is:
R1 BB+ (A(1- + B)) 7
|| S
In particular, i < 1, K co, derive that:
| < (1 - 7) R
17 7 3
According to Theorem the impact of input perturbations on model outputs primarily depends on:
R: The range of output perturbation that is acceptable. A larger range indicates a greater toler- ance for perturbations, leading higher upper bound for the input perturbation.
Rg: The tolerable perturbation radius is negatively correlated with Rg, indicating that larger norm of the input lowers the model's robustness to input perturbations. According to the proof, larger R2 leads to larger coefficient of the perturbation in the resulting bound.
RG: The tolerable perturbation radius is negatively correlated with Rí. This suggests that larger norm of the internal state makes the model more susceptible being led astray during the reasoning process, thus weakening its resistance to input perturbations.
P: The covariance matrix of the training data. More inconsistent training data leads to the model being more sensitive inpul perturbations.
qp: A larger residual coefficient indicates that the model retains more information from input, causing the effects of input perturbations to be preserved across layers.
The theoretical results suggest two main robustness levers at inference and training time. At infer- ence, Theorem shows that longer, more structured chains of thought reduce output fluctuations, while smaller norms of input embeddings and encoded representations decrease the effective Lip- schitz constants in the bounds. At training time, Theorem norms and decreasing the training-data covariance (1') (i.c., making the data more consistent), in- creases the certified perturbation radius.
2 indicates that reducing representation
We further discuss the impact vector norms on the CoT robustness in Appendix D.4 Considering that verifying the effects of I and 7 requires modifying the training data and the model architec- ture of LLMs, this work provides only a theoretical analysis of I and r to inspire future work on corresponding empirical studies, while focusing verifying the effects of R, Ra, and Ra.
5

---

Published as a conference paper at ICLR 2026
Table 2: Average exact match (EM) and oulput fluctuation (OF) of different models using various prompts on different datasets. The highest EM and lowest OF under each setting is marked in bold.
MATH
MMLU-Pro
GPQA
Model
EM
OF
EM
OF
EM
OF
1lama2-7b
14.2 5.0
0.475
11.2 + 5.7
0.622
17.5 + 4.7
0.509
Llama3. 1-8b
45.8 7.2
0.366
41.0
10.7
0.350
26.6 36.6+60
5.7
0.467
Llama-RI-88
64.8 3.0
0.158
44.8 8.3
0.292
28.5 2.9
0.371
Qwen3-8b
77.2 1.6
0.097
46.9 5.2
0.162
37.3 + 1.9
0.214
4 EXPERIMENT
4.1 EXPERIMENT SETUP
Dataset Our experiments are conducted on three reasoning datasets: MATH (Hendrycks et al., 202TI. MMLU-Pro (Wang et al. 2024c). and GPQA (Rein et al., (024). Detailed descriptions of these three datasets are provided in Appendix E.1 Considering the high difficulty of these datasets, which require multistep reasoning processes for solutions, we suppose they can effectively reflect the influence of various factors on the model's ability to handle input perturbations. We also adapt experiments on more datasets Appendix F2
Model We conduct experiments on four mainstream LLMs including: Llama2-7b (Touvron et al.) 2023), Llama3.1-88 (Grattafiori et al. 2024). Deepseek- Crepseck-RI--issee Distilled-LLamms.. 1-8b (Llama-RI. 8b) (DeepSeek-Al et al.) (2025) and Qwen3-8b (Yang al., 2025). These models cover range of capabilities, allowing for a comprehensive evaluation of how model type and different factors affect the handling of input perturbations. For Llama2-7b and 1ama3.1-8h. we employ the instruct version. For Qwen3-8b, we utilize its Thinking Mode to fully leverage its performance. We also experiment with the performance under different model scales Appendix F.5
Metric To evaluate both the performance and robustness, we adopt the following two metrics: (i) Exact Match (EM): Whether the predicted answer is the same the correct answer to the question. A higher value for this metric indicates that the model is better at solving the given dataset, reflecting the overall performance in a specific setting. (ii) Output Fluctuation (OF): The normalized entropy (Friedrich) 2021) of the answers generated from different prompts for the question. A higher value for this metric indicates that the output on the given question is less consistent, reflecting the robust- ness of the specific setting. We detail how to calculate OF in Appendix E.4 We also evaluate with the other fluctuation metric Appendix F3
Input Perturbations To fully reflect the robustness to input perturbations, for each model and dataset, we first generate multiple prompts. Then, for each question, we use these prompts to gener- ate multiple answers. We evaluate the performance by analyzing the correctness and consistency of these answers. To ensure the reliability of our results, we collect all prompts during the optimization of three mainstream methods, including TextGrad (Yuksekgonul 2024), and CFPO (Liu et al., (2025). The base prompts used follow (Grattafiori et al. (2024). which shown in Appendix [E.2] The number of prompts used for each dataset and model detailed in Appendix E.3 We also adapt the experiments using the same prompts on different datasets and models in Appendix F.4
al. (022), OPRO (Yang al.)
More experimental setups are detailed in Appendix E.5
4.2 OVERALL EVALUATION
CoT Robustness Scales with Model Capability The average performance and corresponding fluctuations for different prompts across various datasets and models are shown in Table 2 Results show that across all models, as their capabilities increase, not only does the average EM improve, but the corresponding output fluctuation also decreases. Regarding different tasks, multiple-choice sets (MMLU-Pro. GPQA) exhibit larger fluctuation than MATH, where small logit shifts can flip the selected option PPezeskkpour & Hruschka 2024: Wang et al., 2024a). Yet on GPQA, despite lower EM, fluctuation is not excessive, suggesting difficulyy alone does not significantly affect the CoT robustness. Interpreted through our bounds, stronger models typically (i) train on data with higher consistency (better cleaning and synthesis) which increases the upper bound of input perturbations,

---

Published as a conference paper at ICLR 2026
Outpor Fluctation (OF)
0.8
Oulpst Fluchation (OF)
0.6
FM
OF
0.3
Bract Match (EM)
0.8
RM
OF
0.4
Foact March (FM)
0.4
0.2
0.1
CoT Steps
CoT Steps
Figure 2: The change in OF (left Y-axis) and EM (right Y-axis) with the reasoning steps of the generated CoT, averaged across all experimen- tal datasets and models. The X-axis denotes the CoT step and Y-axis denotes the value of each metric. The curves at X and Y axes illustrate the data distribution. The CoT steps are segmented using ROSCOE (Golovneva et al. 2023).
Figure 3: The change in OF (left Y-axis) and EM (right Y-axis) with the reasoning steps on all datasets and models under the reasoning steps from 1 to 16. The X-axis denotes the CoT step and Y-axis denotes each metrie. The curves at X and Y axes illustrate the data dis- tribution. The CoT steps are segmented using ROSCOE (Golovneva et al. 2023).
which governed by the data-consistency constant I' in Theorem and (ii) yield longer, more structured reasoning steps, increasing K in Theorem and thereby tightening the fluctuation bound. Models supporting Long-CoT (Chen et al. 2025) (e.g., Llama-RI, Qwen3) exemplify this effect. Notably, some settings exhibit larger fluctuations in EM despite having smaller OF. This occurs because the average EM differs across settings, where a setting with a high average EM, even a minor output fluctuation result in large absolute EM fluctuation. In contrast, OF directly measures the consistency of the outputs, thus offering a more faithful representation of output robustness.
Greater Input Perturbation Makes Output
Less Robust To observe the effect of input per-
Output Fluctuation (OF)
turbation on the model output, we analyze the change in output fluctuation with respect the input perturbation across all datasets and models.
0.50
For each question, the input perturbation is calcu- lated as the average distance of input embedding vectors from their mean vector. The results are shown in Figure From the figure, we can find
0.25
0.1
0.2 0.3 0.4
Input Perturbation
that: (i) As the input perturbation increases, the output fluctuation also increases (Pearson Corre- lation Coefficient 0.619), which supports the conclusion of Theorem [ (ii) The majority of input perturbations are concentrated the range of less than 0.1, yet the corresponding change in output fluctuation is quite large, which indicates that even slight fluctuations in the input can lead to significant fluctuations in the output, which is consistent with the findings of previous studies (Zhao 2024 Bigelow al., 2024 (ūii) When the input perturbation exceeds 0.2, the output fluctuation becomes relatively robust as the input perturbation increases. This is because the output fluctuation is measured using normalized entropy, whose maximum possible value depends on how many prompts are used to generate answers. This means that even when input changes become larger, the maximum possible fluctuation in the output stays roughly constant.
Figure 1: The output fluctuation across input perturbation on all datasets and models. Each point denotes one question, where X-axis de- notes the input perturbation as the average dis- tance of embedding vectors from their mean vector, and Y-axis denotes OF. The curves at X and Y axes illustrate the data distribution. The Pearson correlation coefficient is 0.619.
4.3 IMPACT OF REASONING STEP LENGTH ON COT ROBUSTNESS
Robustness Positively Correlated with Reasoning Step Length To verify the impact of reasoning steps, we analyze performance a function of CoT steps (steps computed following
ROSCOE (Golovneva et al., 2023)). The experimental results are presented in Figure 2 Figure 2 reveals the following trends: (i) Output fluctuation generally decreases steps increase, matching Theorem T larger tightens the robustness bound. (ii) The model output fluctuation is relatively low for one-step CoT cases because trivially solvable items need little reasoning and are stable even with short chains. (iii) The performance (i.e., EM) does not not necessarily increase with K. More steps often correlate with harder items, accuracy drop K rises.
7

---

Published as a conference paper at ICLR 2026
Outpul Flactuaiin (OF)
Outpot Fluctuation (OF)
0.75
0.50
0.50
0.25
0.25
0.00
0.00
40
70 80
100 110
120
100
150
160
170
180
190
Input Embodding Vector Narm
Hidden State Vector Narm
Figure 4: The change in output fluctuation with the norm of the input embedding vector across all experimental datasets and models. Each point denotes the result of one question, where X-axis denotes the imput vector norm and Y-axis denotes OF of this question. The Pearson corre- lation coefficient is 0.506.
Figure 5: The change in output fluctuation with the norm of the hidden state vector across all experimental datasets and models. Each point denotes the result of one question. where X-axis denotes the hidden state vector norm and Y-axis denotes OF of this question. The Pearson corre- lation coefficient is 0.229.
Infinity Reasoning Steps Cannot Eliminate the Impact of Input Perturbation To further verify the conclusion from Equation thhat even infinitely long reasoning steps cannot completely eliminate the impact of input perturbations, we conduct experiments with an extended number of reasoning steps. We add the instruction "You MUST reason in exactly K steps" to the prompt to guide the model in generating longer reasoning processes, requiring the model to generate outputs for K 1, .,., 16 steps. Considering that the model could not strictly follow the instruction to generate the specified steps, we still use ROSCOE to calculate the actual steps. The results are presented in Figure 3 From the figure, we observe that as the number of reasoning steps increases, the output fluctuation decreases but eventually converges
relatively stable level. This indicates that the role of reasoning steps in eliminating perturbations is limited, thereby empirically validating the conclusion of Equation 4 Since OF begins fluctuate, suppose that current experimental steps have supported our conclusion and do not conduct experiments over 16 steps.
4.4 IMPACT OF EMBEDDING NORMS ON CoT ROBUSTNESS
Larger Input Embedding Norm Makes Output Less Robust To verify the relationship between output fluctuation and the norm of the input embedding vector, we analyze the experimental results across all datasets and models, shown in Figure From the figure, can observe that: (i) As the norm of the input emhedding vector increases, the model output fluctuation shows a general upward trend, which confirms the related conclusions in Theorem 2 (ii) As the input embedding norm grows, output fluctuation saturates, since a normalized entropy capped by the number of prompts, its maximum stays roughly constant even under larger input perturbations. (iii When the norm of the input embedding vector increases from 60 to 70, the output fluctuation exhibits sudden jump, which indicates that a threshold exists for the vector norm that the model can handle stably. Once this threshold is surpassed, most input perturbations exceed the upper bound defined in Theorem 2 causing significant fluctuations the output.
Larger Hidden State Norm Makes Output Less Robust To verify the relationship between the norm of the hidden state vector and the output fluctuation, we analyze the results across all datasets and models. The hidden state vector is extracted from the last layer of the last CoT step. The results are shown in 33 From the figure, we can find that: (i) As the norm of the hidden state increases, the output fluctuation shows general upward trend, which confirms the positive correlation between the two as stated in Theorem 2 (ii) The vector norms for the majority of data points are concentrated on the (140, 150) range, which indicates that a well-trained model tends to encode data into a specific and relatively small norm interval to mitigate the impact of input perturbations. (iii) Overall, the change in output fluctuation with the hidden state norm is not significant. We suppose the reasons for this are that the constant is determined by the upper bound of the hidden state norm rather than its specific value, and that the various normalization structures like LayerNorm (Xiong et al.) 2020 within the Transformer architecture mitigate the output fluctuation to some extent.
4.5 PROMPT OPTIMIZATION WITH HIGHER INPUT ROBUSTNESS
8

---

Published as a conference paper at ICLR 2026
To shed light on future research, discuss how to optimize the performance of prompt optimiza- tion based Theorem 2 Let
Table EM on each model and dataset using different prompt optimization methods. "." de- notes using the single base prompt directly. The best result under each setting is marked in bold.
-'s and F denote the expression on the right-hand side Theorem 2 We hope to select the prompt that makes F' as large as possible, thereby increasing the upper bound of the input perturbation. Let
Model Method
MATH MMLU-Pro
GPQA
Llama3, 1-8b
46.8
45.7
23.7
A = R,,,, we can derive that:
TextGrad
45.2
47.4
27.6
aF
R²
OPRO
44.6
47.1
27.1
2(11 + V + + <
CFPO
47.0
48.1
27.6
0A
Ours
47.2
49.0
32.3
This shows that F is negatively correlated with A, where a larger value of A-1 corresponds to a larger upper bound for the input perturbation, meaning the model can tolerate greater input per-
Qwen3 9-8
77.4
42.3
37.1
TextGrad
77.6
44.9
38,4
OPRO
77.2
45.9
37.4
CFPO
77.0
45.8
38.4
Ours
77.6
49.2
38.4
turbations. Therefore, for each question, we first construct inputs using all obtained prompts and extract the corresponding embedding layer vectors, as well as the vectors from the final layer to serve as hidden state vectors. We then calculate the select the prompt with the highest value of A-1 as the designated prompt for inference. The experimental results are shown in Ta- ble 3 We caleulate only the Exact Match (EM) for each method and not the Output Fluctuation (OF), since each method selects single optimal prompt for each question to perform inference, and consequently yields only one output as the final answer. From the table, we can see that our method brings performance improvements across all settings, which demonstrates the effectiveness of our method. We also discuss the efficiency of our method in Appendix D.2] Since the primary objective of this paper is to analyze the factors affecting input robustness, rather than to optimize prompt op- timization methods, we leave the investigation into how to better effectiveness and efficiency, and more extensive comparison with additional baselines for future work.
norms of both to obtain A. Subsequently, for each question,
5 RELATED WORKS
Robustness of Chain-of-Though Numerous studies show that slight perturbations in the input can lead to drastic changes in the output of CoT (Zhao et al.) [2024] [Shi et al.] (2024b). Therefore, to enhance the performance of CoT, a variety of works are proposed to improve and analyze the CoT ro- bustness. For example, noisy or off-task rationales reliably degrade CoT performance. Contrastive denoising, including CD-CoT and NoRa, mitigates these effects (Zhou et al.] 2024). Break-The- Chain applies semantics- prreetiiin rewrites (narrativization, mild constraint changes, reordering, numeric tweaks) to reveal sensitivity in code generation (Roh et al. 2025). Character-level perturba- tions (R3 ATA) likewise disrupt reasoning (Gan et al. 2024). defensive rationales that resist corruption injection and reduce collapse (Wang et al. (025). Post- hoc Self-Correction Reflection repairs errors under perturbations (Wu et | 2025).
structures
Ceff-Consistency reduces single-path brittleness through voting (Wang al.) 2023b). CoT is sensitive to step order and exemplar relevance (Wang et al. 2023a). Theory indicates that more coherent chains aid error correction but increase vulnerability to noise in intermediate steps (Cui et al. 2024). Evidence also suggests that CoT often functions as constrained imitation rather than genuine reasoning (Shao & Cheng, 2025). Generalization analyses for nonlinear Transformers identify robustness conditions under noise and distribution shift (Li et al. 2024).
Despite these advances. the mechanism by which input perturbations induce output changes remains under-specified. We derive upper bounds that link input perturbations to output fluctuations and analyze the factors that govern CoT robustness, extending prior research.
Prompt Optimization. Prompt optimization methods primarily focus on how to optimize prompts based on the given model and task to enhance the performance. Work on prompt optimization spans RL and gradient-free edit search (Deng et al. 2022 Prasad et al. 2023). influential-toker clustering to shrink the search space (Zhou et al. 2023a). and ensemble style boosting to avoid single-prompt failure (Hou et al. 2023). Refinements include genetic and acttr-critic editing, localized zeroth- order updates, and exemplar-ordene
optimization (Xu et al. 20222] Dong et al.) 20244) Hu et al.] 2024 Wu et al. 2024). APE and OPRO iteratively propose and select improved instructions (Zhou

---

Published as a conference paper at ICLR 2026
al. 2023b Yang al. 2024). ProTeGi and APO implement textual "gradient descent" with beam 2023). TextGrad generalizes to "automatic differentiation via text"
or bandit search (Pryzant et al.
Yuksekgonul al. 2025). Data-driven pipelines such Self-Instrun and Auto-Instruct bootstrap and rank prompt sets (Wang et al. [023c] Zhang et al. 2023). Search strategies include MCTS with reflective error analysis (Wang al.] 2024b). Budgeted best-arm identification supports selection under tight evaluation budgets (Shi et al. (2024)). Preference-based black-box optimization aligns prompts with user goals (Cheng al., 2024). RL improves textual-prompt stability (Kwon al.,
2024). Compiler-style systems such as DSPy learn prompts for multi-stage LM pipelines (Khattab al. 2024). OPRO-like gains may attenuate on smaller open models (Zhang et al. 2024b).
Despite strong empirical progress, the mechanism pathway from input perturbations to output fluc- tuations remains poorly understood. We analyze this affect and its determinants to guide principled designs for the future prompt optimization works.
CONCLUSION
In this paper, we theoretically analyze the influence of various factors the input robustness of COT. We first prove that the impact of input perturbations on the CoT output is negatively correlated with the number of CoT reasoning steps, and that even an infinite number of steps cannot completely eliminate the effects of input perturbations. We then apply these findings to LSA, demonstrating that its input robustness is negatively correlated with the norms of the input embedding and hidden state vectors. To validate these conclusions, we conduct experiments on four mainstream L.LMs and three mainstream datasets. Experimental results reveal that output fluctuations vary with different factors in line with expectations, supporting the validity of our findings. Furthermore, guided by this analysis, we propose to select the prompt by raising the upper bound of input perturbation, which yields consistent performance gains previous works. Moving forward, work opens several promising avenues for advancing robust th-innof-thhougt reasoning. In particular, key next step systematically examine how the parameters I and in Theorem influence input robustness, which could also inform the design of more resilient large reasoning models.
7
REPRODUCIBILITY
We have provided all proofs of this paper in Appendix C The experimental and pre- processed data and code are released in htttps://githbb. com/ zirui HIT/CoT -ST/CoT-Robust
ACKNOWLEDGE
We gratefully acknowledge the support of the National Natural Science Foundation of China (NSFC) via grant 62236004 and 62476073. This research/project was also supported by the Singapore Ministry of Education (MOE) Academic Research Fund (AcRF) Tier 1 grant (Proposal ID: 24- SIS S0--MM- Yang Deng is support by the Lee Kong Chian Fellowship awarded by Singapore Management University.
REFERENCES
Thomas C. Bachlechner, Bodhisattwa Prasad Majumder, Huanru Henry Mao, Cottrell, and Ju- lian McAuley. Rezero is all you need: Fast convergence at large depth. In Conference on Un- certainty Artificial Intelligence, 2020. URL hhttps://aple swwanticscholar..ooo.
DorpusID:2126444
Eric Bigelow, Ekdeep Singh Lubana, Robert Dick, Hidenori Tanaka, and Tomer Ullman. In- context learning dynamics with random binary sequences. In The Twelfth International Confer- ence on Learning Representations, 2024. URL httpss//ppeneeeeww.eee. net. wtw.eee//ooormtoeeooee
62K 62K7mAL.02
Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards reasoning era: survey of long chain-
10

---

Published as a conference paper at ICLR 2026
of-thought for reasoning large language models, 2025. URL 2503. 09567
Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, lana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan Routledge, and William Yang Wang. Finqa: A dataset of numerical reasoning over financial data. Proceedings of EMNLP 2021, 2021.
Jiale Cheng, Xiao Liu, Kehan Zheng. Pei Ke, Hongning Wang, Yuxiao Dong, Jie Tang, and Minlie Huang. Black-box prompt optimization: Aligning large language models without model train- ing. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Long Papers), pp. 3201-3219, Bangkok, Thailand, 2024. Association for Computational Lin- 176.
guistics. doi: 10. 10.118653/1122224..ll...
acl acl-long.1755
Prateek Chhikara. Mind the confidence gap: Overconfidence, calibration, and distractor effects in 22102..028822
large language models, 2025. URL
Liam Collins, Advait Parulekar, Aryan Mokhtari, Sujay Sanghavi, and Sanjay Shakkottai. In-context learning with transformers: softmax attention adapts function lipschitzness. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS '24, Red Hook, NY, USA, 2025. Curran Associates Inc. ISBN 97983313 143855.
Yingqian Cui, Pengfei He, Xianfeng Tang, Qi He, Chen Luo, Jiliang Tang, and Yue Xing- A the- oretical understanding of chain-of-thouu thtthhhhh Coherent reasoning and error-aware demonstration. arXiv preprint 1.Xx::2410.164444 2024. RRLLttts::/axxivvoor////
16540
DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang. Bochao Wu, Bei Feng. Chengda Lu. Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangho Hao, Guanting Chen, Guowei Li, H. Zhang. Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang. Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang. Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. I.. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang. Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei. Tianyu Sun, T. Wang, Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An. Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li. Xiangyue Jin, Xiaojin Shen, Xi- aosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo. Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong. Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepseek-rl: Incentivizing reasoning capability in llms via reinforce- ment learning, 2025. htt //apvivi///aiiviii/ai/iiieaaaiaa org/ abs/ 2501 12948
Mingkai Deng, Jianyu Wang, Cheng-Ping Hsich, Yihan Wang. Han Guo, Tianmin Shu, Meng Song. Eric Xing, and Zhiting Hu. Riprompt: Optimizing discrete text prompts with rein- forcement learning. In Proceedings of the 2022 Conference on Empirical Methods in Natu- ral Language Processing, pp. 3369-3391, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguistics. doi: 1/88633/v12022.mmmllll https://acl antho wonttholog..oro.
URL
http:///aaaaaaat/c/ ogy. ogy.org//.2...... 1m- 2227
11

---

Published as a conference paper at ICLR 2026
Richard Diehl Martinez, Pietro Lesci, and Paula Buttery. Tending towards stability: Conver- gence challenges in small language models. In Yaser Al-Onaizan, Mohit Bansal, and Yun- Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, pp- 3275-3286, Miami, Florida, USA, November 2024, Association for Computational Linguistics. doi: 10. (0053/////222..nndingg.
URL https: //acl antheloog..ogg/3000
Findings ww..p..n......
Yihong Dong, Kangcheng Luo, Xue Jiang, Zhi Jin, and Ge Li. Pace: Improving prompt with actor- critic editing for large language model. Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.). Findings of the Association for Computational Linguistics: ACL 2024, pp. 7304 7323, Bangkok, Thailand, August 2024, Association for Computational Linguistics. doi: 10.18653/ 11/204.findings-ad/ 436/
URL http::ttt/////ttt http:///aallaahhhhaaaa 1owphholoo...r//22244.. fi indings-act
Mahyar Fazlyab, Alexander Robey, Hamed Hassani, Manfred Morari, and George J. Pappas. Ef- ficient and accurate estimation of lipschitz constants for deep neural networks. In Proceedings of the 33rd International Conference Neural Information Processing Systems, Red Hook, NY, USA, 2019. Curran Associates Inc.
Roland Friedrich. Complexity and entropy in legal language. Frontiers in Physics, 9:671882, jun 2021. doi: 00.3389//phy22221.678882
Esther Gan, Yiran Zhao, Liying Cheng, Yancan Mao, Anirudh Goyal, Kenji Kawaguchi, Min- Yen Kan, and Michael Shieh. Reasoning robustness of Ilms to adversarial typographical emrors. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 10449-10455 10459, Miami, Florida, USA, 2024. Association for Computational Lin- guistics. URL https::////........ /acl antho 2024 emn 1ammlppree p-maa----- in. 584/
Olga Golovneva, Moya Peng Chen, Spencer Poff, Martin Corredor, Luke Zettlemoyer, Maryam Zazal-Zarandi, and Asli Celikyilmaz. ROSCOE: A suite of metrics for scoring step-by-step rea- soning. In The Eleventh International Conference on Learning Representations, 2023. URL net wtwparrrwweeeneeefooooooeee---e
Aaron Grattafior,, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshom, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Ko- renev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru. Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song, Danielle Pintz, Danny Livshits, Danny Wyatt, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Francisco Guzmán, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Govind That- tai, Graeme Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Kore- vaar, Hu Xu, Hugo Touvron, lliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Ma- hadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jong- soo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia. Kalyan Vasuden Alwala, Karthik Prasad, Kartikeya Upasani. Kate Plawiak, Ke Li, Kenneth Hcafield. Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Cantala-Yeary Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Ku- mar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoy- chev, Niladri Chatterji, Ning Zhang, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng, Prajjwal Bhargava, Pratik Dubal, Praveen Krishnan,
12

---

Published as a conference paper at ICLR 2026
Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ra- mon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohan Maheswari, Ro- hit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang, Saghar Hosseini, Sahana Chennabasappa, Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang. Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mi- haylov, Tong Xiao, Ujjwal Kam. Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do, Vish Vogeti, Vítor Albiero, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang. Xiaofang Wang, Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang, Yaelle Gold- schlag, Yashesh Gaur, Yasmine Babaci, Yi Wen, Yiwen Song, Yuchen Zhang. Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh, Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajofeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, An- drew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, An- nie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leon- hardi, Bernie Huang. Beth Loyd, Beto De Paola. Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Mon- talvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changhan Wang, Changkyu Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cymthia Gao, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Due Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Eric-Tuan Le. Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smoth- ers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel, Francesco Caggioni, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayanan, Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harri- son Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj, Igor Molybog. Igor Tufanov, Ilias Leontiadis, Irina-Elens Veliche. Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam, Japhet Asher. Jean-Baptiste Gaya, Jeff Marcus, Jeff Tang, Jen- nifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang, Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Jun- jie Wang. Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kiran Jagadeesh, Kun Huang, Kunal Chawla, Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A, Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhaut Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong, Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji. Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj, Qian Liang, Rachad Alao, Rachel Ro- driguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Mahajan, Saurabh Verma, Seiji Yamamoto, Sharadh Ra- maswamy, Shaun Lindsay, Shaun Lindsay, Sheng Feng, Shenghao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar. Shuqiang Zhang, Shuqiang Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satter- field, Sudarshan Govindaprasad, Sumit Gupta, Summer Deng, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo
13

---

Published as a conference paper at ICLR 2026
Koehler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varum Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Ku- mar, Vishal Mangla, Vlad lonescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang. Wes Bouaziz, Will Constable, Xiaocheng Tang, Xiao- jian Wu, Xiaolan Wang, Xilun Wu, Xinbo Gao, Yaniv Kleinman, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang. Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait, Zachary DeVito, Zef Rosnbrick, Zhao- duo Wen. Zhenyu Yang, Zhiwei Zhao, and Zhiyu Ma. The llama 3 herd of models, 2024. URL https://arxils org/ als 2407 21783
Dan Hendryeks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), 2021. URL hwwph:ttss///cctmm /ttp:::://peeeeview net /aaee/foruuee forum BywttBtttmssst
Bairu Hou, Joe O'Connor, Jacob Andreas, Shiyu Chang, and Yang Zhang. Prompthoosting: Black-box text classification with ten forward passes. In Proceedings of the 40th Interna- tional Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 33463-13488 PMLR, 2023. URL htttps://proceeeinggg. wmw.eiiingsm.m.......mmmmm.. press/ 2202/hou220 html
Wenyang Hu, Yao Shu, Zongmin Yu, Zhaoxuan Wu, Xiangqiang Lin, Zhongxiang Dai, See-Kiong Ng. and Bryan Kian Hsiang Low. Localized zeroth-order prompt optimization. In Advances in Neural Information Processing Systems 37 (NeurIPS 2024), 2024. URL https://arxix:
org/ ore/abs/ 2403. 02993
Jianhao Huang, Zixuan Wang, and Jason D. Lee. Transformers learn implement multi-step gra- dient descent with chain of thought. In The Thirteenth International Conference on Learning Representations, 2025. URL https: :/tpps::/opee httpp::/ htt::///ppeeeieeeeee net.eeewweeeeeeee..eeeeeeeeeeeee www.eeeeeorenndrreeeeeeeeeeeeeee
Yue Huang, Jiawen Shi, Yuan Li, Chenrui Fan, Siyuan Wu, Qihui Zhang, Yixin Liu, Pan Zhou, Yao Wan, Neil Zhenqiang Gong, and Lichao Sun. Metatool benchmark for large language models: De- ciding whether use tools and which use. In The Twelfth International Conference on Learning Representations, 2024. URL https: http://ppp.......c////pp/p. net/ Eorum? wtt.uummmio-R02ttaaggag
Omar Khattab, Arna Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vard- hamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. Dspy: Compiling declarative language model calls into state-of- the-art pipelines. In The Twelfth International Conference on Learning Representations, 2024. pttttp::///penneeviee net http:////ppppeeeeeeeeee/////n/nr =o ICLR 2024 (spotlight).
Hyunjik Kim, George Papamakarios, and Andriy Mnih. The lipschitz constant of self-attention, 2021. BRLlhttpss//openeeriie net. www..e//ooruuuoooooaaannrraaaee
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL
ht httt:::///ppneeeiiee net fformmmmmmmmmmm www.mmidddeeebbbyyye 2TBbbyy0BBBFFF
Minchan Kwon, Gaeun Kim, Jongsuk Kim, Haeil Lee, and Jummo Kim. Stableprompt: Automatic prompttuiing using reinforcement learning for large language models. In Proceedings the 2024 Conference on Empirical Methods in Natural Language Processing, pp. 9868-9884, 9884, November 2024. URLpbttps:
aallhoothooooooo logyy . wwtg/////a.. 2224 emn momnlp-maine mai..aii... 551.551..........
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
Hongkang Li, Meng Wang, Songtao Lu, Xiaodong Cui, and Pin- Yu Chen. Training nonlinear trans- formers for ofain-of-thougee inference: A theoretical generalization analysis. arXiv preprint vxXiv22410.02167 2024. URL ID: MTT
02167
OpenReview
14

---

Published as a conference paper at ICLR 2026
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, llya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference Learning Representations, 2024. URL oppnreview.
Yuanye Liu, Jiahang Xu, Li Lyna Zhang, Qi Chen, Xuan Feng, Yang Chen, Zhongxin Guo, Yuqing Yang, and Peng Cheng. Beyond prompt content: Enhancing IIm performance via content-format integrated prompt optimization, 2025. |RL http:: a rxw.. http:/bbs//2222/ wwwp.ii.org/bb///522.... 04295
Jianmo Ni. Jiacheng Li, and Julian McAuley. Justifying recommendations using Eestantly-abbelee reviews and fine-grained aspects. In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing ( (EMNLP-IJCNIPP) pp- 188-197, Hong Kong, China, November 2019. Association for Computational Linguistics. doi: 10.18653/v 1/D URL hattp:://aelaatttooooo.o/////// 10188/
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf. Ed- ward Yang. Zach Vito. Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. Pytorch: an imperative style, high-performance deep learning library. In Proceedings of the 33rd International Conference on Neural Information Processing Systems, Red Hook, NY, USA, 2019, Curran Associates Inc.
Pouya Pezeshkpour and Estevam Hruschka. Large language models sensitivity to the order of op- tions in multiple-choice questions. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Findings of the Association for Computational Linguistics: NAACL 2024, pp. 2006-2017, Mex- ico City, Mexico, June 2024, Association for Computational Linguistics. doi: 10. 1/228533/1/2024. findings-nase naa.c. 130/
URL https: httpss//aelaatbbooo...c../////2. wtwp:///222.....innnnnnn...o
Archiki Prasad, Peter Hase, Xiang Zhou, and Mohit Bansal. Grips: Gradient-free, edit-based instruction search for prompting large language models. In Proceedings of the 17th Confer- ence of the European Chapter of the Association for Computational Linguistics, pp. 3845-3864, Dubrovnik, Croatia, May 2023. Association for Computational Linguistics. doi: 10. 18653/v1/ Bel.https::. a http:///aaaaaathaaoa ogy. org. 2202322233222 11022.eea... eac ma in. 2777
2023.eacl
Reid Pryzant, Dan Iter, Jeny Li, Yin Lee, Chenguang Zhu, and Michael Zeng. Automatic prompt optimization with "gradient descent" and beam search. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 7957-7968, Singapore, December 2023. Association for Computational Linguistics. doi: 10. 200653/v1/202.cmm/// mailll--aii.444,., URL
2023 emnlp ma inn 4
Xianbiao Qi, Jianan Wang, Yihao Chen, Yukai Shi, and Lei Zhang. Lipsformer: schitz continuity to vision transformers, In The Eleventh International Conference on Learning Representations, 2023. URL
Introducing lip-
net www.ee/eooonnndddiieeeeeee..
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google-prood q&a benchmark. In First Conference on Language Modeling, 2024. UtLhttps: /ttp://///////eeiee// http:/oouummmd:::111100
Jaechul Roh, Varun Gandhi, Shivani Anilkumar, and Arin Garg. Break-the-chaid Reasoning failures in Ilms via adversarial prompting in code generation. arXiv preprint 25X,00500660671,, 2025.
URL
abs/ 06971
Pranab Sahoo, Ayush Kumar Singh, Sripamn Saha, Vinija Jain, Samrat Mondal, and Aman Chadha. A systematic survey of prompt engineering in large language models: Techniques and applica- tions, 2025. J /tattp:://axiie org. abs 22022 07927
Jintian Shao and Yiming Cheng. Cot is not true reasoning, it is just tight constraint to imitate: A theory perspective. arXiv preprint arXiv:2506.0 02878, 2025. URL ht pettps://arxia //trx/aaivv//////////////////a// /trpii.i.gg/////o/........ abs/ 2206 02878
15

---

Published as a conference paper at ICLR 2026
Chengshuai Shi, Kun Yang. Zihan Chen, Jundong Li, Jing Yang, and Cong Shen. Ef-
ficient prompt optimization through the lens of best arm identification. vances in Neural Information Processing Systems 37 (NeurIPS 2024), 2024a. URL
In Ad-
httt:s://proceedinge neur http://www.....pppppe.eeeee//epa 2:lc::2:227755888833387445545522
Zhenmei
Wei, Zhuoyan Xu, and Yingyu Liang. Why larger language models do in-context learning differently? In Proceedings of the 41st International Conference on Machine Learning. ICML'24. JMLR.org, 2024b.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie- Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuming Mao, Xavier Martinet. Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nic, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models, 2023. URL hnttps://arkie org org/ abs/ 2207 09288
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, ukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Ad- vances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL file/3f5ee243555 1 Paper Paper.pdl
ccppaper.........epppeppeeee...e files/paper/2017,
Shubham Vatsal and Harsh Dubey. survey of prompt engineering methods in large language models for different nip tasks, 2024. URL/ http://awxww..ogaab//////a/ wttg/bs/////...// 12994
Johannes Von Oswald, Eyvind Niklasson, Ettore Randazzo, João Sacramento, Alexander Mordy- intsev, Andrey Zhmoginov, and Max Vladymyrov. Transformers learn in-context by gradient descent. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org, 2023.
Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, and Huan Sun. Towards understanding chain-of-thought prompting: An empirical study of what matters. In Pro- ceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 2717-2739, Toronto, Canada, 2023a. Association for Computational Lin- guistics. doi: 1/202333//.2...allllllo 153. /ttp//tpppp//aaaaaaaaaaooooooo.p acl-long. 153/
Sinong Wang, Belinda Z. Li, Madian Khabsa, Han Fang, and Hao Ma. Linformer: Self-attention with linear complexity. CoRR, hlss2000.044766 2020a. URL htttp:://axiv..ortt
2006. 04768
Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. Minilm: deep self- attention distillation for task-agnostic compression of pre-trained transformers. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA, 2020b. Curran Associates Inc. ISBN 9781713829546.
Wenxiao Wang, Parsa Hosseini, and Soheil Feizi. Chain-of-defenniv--thouggt: Structured rea- soning elicits robustness in large language models against reference corruption. arXiv preprint arXiv:2504.20769, 2025. URL BtLhhtpss: htt.tpps://axx....ooo//aaaaaaaas
Xinpeng Wang, Chengzhi Hu, Bolei Ma, Paul Rottger, and Barbara Plank. Look at the text: Instruction-tuned language models more robust multiple choice selectors than you think. In First Conference on Language Modeling, 2024a. URL httpp://cpenreviev.
forumm Forrum?11==Hdd585GG2
16

---

Published as a conference paper at ICLR 2026
Xinyuan Wang. Chenxi Li, Zhen Wang, Fan Bai, Haotian Luo, Jiayou Zhang, Nebojsa Jojic, Eric P. Xing, and Zhiting Hu. Promptagent: Strategic planning with language models enables expert- level prompt optimization. In The Twelfth International Conference on Learning Representations, 2024b. htt.hhttts///ooto
net forum? ii i d=22pyNM Toa ICLR 2024.
Xuezhi Wang. Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In International Conference on Learning Representations (ICLR), 2023b. URL https: //openzeview. net/forumiid-ii 1PNNMMMM1
Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self tefe-aeeerteed instructions. In Anna Rogers, Jordan
and Naoaki Okazaki (eds.). Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pupers), pp. 13484- 13508, Toronto, Canada, July 2023c. Association for Computational Linguistics. doi: 10.18653/ htt.tttps:/// a http://alaaaathho ogy. org/ 2023 aa3..cclllong 754/
Yubo Wang, Xueguang Ma. Ge Zhang. Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weim- ing Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, Tianle Li, Max Ku, Kai Wang. Alex Zhuang, Rongqi Fan, Xiang Yue, and Wenhu Chen. MMLU-pro: A more robust and challenging multi- task language understanding benchmark. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024c. URL htttps://openreviieees nnt// id=y 0DM6R2r3
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoe V Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Ad- vances in Neural Information Processing Systems, 2022. URL Srrumiid=_VjoLMeet
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. Transformers: State-of-the-ar natural language processing. In Qun Liu and David Schlangen (eds.), Proceedings of the 2020 Confer- ence on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38- 45, Online, October 2020. Association for Computational Linguistics. doi: 10. 00118865311/22000 6/
emnlp-demoss URL httph: htt://aclaaan http//aaaannooooo..ooooo 2020
Hua Wu, Haotian Hong, Jiayu Mao, Zhexun Yin, Yanxiong Wu, Xiaojing Bai, Li Sun, Mengyang Pu, Juncheng Liu, and Yihuan Li. Forging robust cognition resilience in large language models: The self-correction reflection paradigm against input perturbations. Applied Sciences, 15(9):5041,
2025. URL /tttts://///. //ww:/ mópi www.///006600 3417113//7711111111111
Zhaoxuan Wu, Xiaoqiang Lin, Zhongxiang Dai, Wenyang Hu, Yao Shu, See- Kiong Ng. Patrick Jaillet, and Bryan Kian Hsiang Low. Prompt optimization with ease? efficient ordering-aware automated selection of exemplars. In Ad- URL
vances in Neural Information Processing Systems 37 (NeurIPS 2024), 2024.
pttp:::/ppoocediggg.nuuu wwtppippppppppppppppppppep.ppeee fitpap//eee.eeeeeepeee/////222 dd8e 7
Ruibin Xiong, Yunchang Yang, Di He, Kai Zheng, Shuxin Zheng, Chen Xing, Huishuai Zhang, Yanyan Lan, Liwei Wang, and Tieyan Liu. On layer normalization in the transformer architecture In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 19 of Proceedings of Machine Learning Research, pp. 10534- PMLR, 13-18 Jul 2020. BRLhttps:: hnttps::/ppoceeeinnnn mlr www.sppress/eeeme http:/v111xxxooo wtttess/v199/xxcgg2000 html
Hanwei Xu, Yujun Chen, Yulun Du, Nan Shao, Yanggang Wang. Haiyu Li, and Zhilin Yang. Gps: Genetic prompt search for efficient few-shot learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 8162 8171, December 2022. URL https://aco antho ogy. ogy. www.:0222.mmmmmmmmmmmmmmmm ma n. 559.
17

---

Published as a conference paper at ICLR 2026
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang. Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang. Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin. Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 technical report, 2025. URL ptttttss//aratt. org/ aww/250s.5... 09388
Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoe V Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers. In The Twelfth International Conference on Learning Representations, 2024. URL
net forum 2i
Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Pan Lu, Zhi Huang, Carlos Guestrin, and James Zou. Optimizing generative ai by backpropagating language model feedback. Nature, doi: 015588441586-025-0555555 https://doi.org/ 10. 1038/ 025-0820880000 000-00861-2
639(8055):609-6666 mar 2025. ISSN 1476-4687.
URL
Hongyi Zhang, Yann N. Dauphin, and Tengyu Ma. Residual learning without normalization via better initialization. In International Conference Learning Representations, 2019. URL
https://cppereviier net/
Ruiqi Zhang, Spencer Frei, and Peter L. Bartlett. Trained transformers learn linear models in- context. J. Mach. Learn. Res., 25(1), January 2024a. ISSN 1532-4435.
Tuo Zhang, Jinyue Yuan, and Salman Avestimehr. Revisiting opro: The limitations of small-scale lims as optimizers. In Findings of the Association for Computational Linguistics: ACI. 2024, Bangkok, Thailand, 2024b. Association for Computational Linguistics. doi: 10. (0.18653/v120024 findings-acl.1066 100. URL
org. org/2024 2024. findings acl. ac1.1007
Zhihan Zhang, Shuohang Wang, Wenhao Yu, Yichong Xu, Dan Iter, Qingkai Zeng, Yang Liu, Chenguang Zhu, and Meng Jiang. Auto-instruct: Automatic instruction generation and ranking for black-box language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, Singapore, 2023.
Siyan Zhao, Tung Nguyen, and Aditya Grover. Probing the decision boundaries of in-context learn- ing in large language models. In First Workshop on Long-Context Foundation Models @ ICML 2024, 2024. httttttss///peeeeeeeett net htt.:/formmmmimmmmmmm wdd.:eeeoooommdam-t. 9WUB
Han Zhou, Xingchen Wan, Ivan Vulić, and Anna Korhonen. Survival of the most influential prompts: Efficient black-box prompt search via clustering and pruning. In Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 13064-13077, Singapore, December 2023a. Association for Computational Linguisties. doi: 10. 10111166631/22233 http:///w//oo..nnnnnneemeee findings-emnlp.877. URL htt.://2.....ininneeeeee...
870/
Wangchunshu Zhou, Canwen Xu, Tao Ge, Julian McAuley, Ke Xu, and Furu Wei. Bert loses pa- tience: fast and robust inference with early exit. In Proceedings of the 34th International Confer- ence on Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.
Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. Large language models are human-level prompt engineers. In The Eleventh Interna- tional Conference on Learning Representations, 2023b. htLpstts:: httthttt////eeeeeee http:://///peeeeeeeeee .eet/ ICLR 2023 (poster).
Sorum?id=92gvk8B20
Zhanke Zhou, Rong Tao, Jianing Zhu, Yiwen Luo. Zengmao Wang, and Bo Han. Can language models perform robust reasoning in of prompting with noisy rationales? In Ad- vances in Neural Information Processing Systems (NeurIPS), 2024. URL /tttps://arxiv. brgg/abs/2400 / 23856 arXiv.2410.23856.
18

---

Published as a conference paper at ICLR 2026
Chiwei Zhu, Benfeng Xu, Quan Wang. Yongdong Zhang, and Zhendong Mao. On the calibration of large language models and alignment. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 9778-9795, Sin- gapore, December 2023. Association for Computational Linguistics. doi: 10. 10.18353/1/2023. findnngs-emnl..654. URL ottee:///ecllatbbooooo...ogg///2 f findings 654/]
19

---

Published as a conference paper at ICLR 2026
A ETHICS STATEMENT
All datasets and models used in this paper are publicly available, and our usage follows their licenses and terms.
B LLM USAGE
We have employed the Al tool for coding and writing polishing.
C
PROOFS
Proof of Theorem 1
ER == =fbx)=((b-1(()) ft)x h2(2 + 8)= = 1 + + 0), + 6) - (x))
- -h((t,)
k KENt
=
x 6), +8).
||E/|| = (
x1) - s(h,,)) S <11// - -h1 + C1I=1 - 1
= ||a||| S --||=~111 + C7
K-1
=
||Ex|| s 7 ||ol|| + C1110
Σ 7
K-1
1 k
Σ 5イ 1 7
E (0)
1=
C (1-
|
||
S >* ||=1|| +
7 > ||,1|
|
A = max
|
|| S (Ar* + 1- (1 )イ}
C
71 11811
Proof of Lemma 7
h
- n
= (TE==ETKE KE.
[0 0
[
P P= T=WPVV [T((T
K= = = ( +
0
1
0 0
K. :- = [KK
+ K ) =, Vs(E)= = 2K,E.
vf(E) = 7/ + (L)) (PE)
Bound for C.
of (E) = 1 II, + +l, P. + (PE) (2K.E)
dx
0
K =
+ (K,E), -0 + (2K,E), 0.
0 0
as (L) = 11 + s(E) Ps.
Or
of (B)
< 1 |||||| [b < h || 1.3
EA =
h 0
|F = |k||| S Rm,
|P|| = (T>(T )) ||| = [T>(T- : 1
C S + ||| R
(i.e., C n [T[ - | R
20

---

Published as a conference paper at ICLR 2026
Bound for y
05, ōh [B) = = [T, + EEE Ph. + (PF) (2K, F) Pn. =0 (since PE = [0; [Tr(T )
(2K, - 2 [Ti( --- h.
For any 8 E Rd,
of Oh
[n0]
0 0 + |PE|| 21r (A (m' PF
(E) =
[T)[--))] x| |PET
|| PE|| = [T>(T>>)} | - [T>(T 2) _/4 Re,,
|| - ||ا ||-||| - ||T Rr.
(orthogonal blocks)
(E))
S 11|1= + 2 Rs || Rs. )
2 1100112
7 S
122 + 4 22 - 1| R
(i.e. S v 1/2 [2[TT([-2)] ||- R2
D ADDITIONAL DISCUSSION
D.I INFLUENCE OF NON- FACTORS OF TRANSFORMER
In this section, we discuss the influence of different non-linear factors within the Transformer archi- tecture on the conclusions of Theorem 2 Overall, most non-linear factors contribute to enhancing the model's input robustness. Due the complexity of theoretically proving the effects of these non-linear factors, we only provide an intuitive analysis and leave rigorous mathematical proofs for future work.
Attention Non-linearity (Softmax) The exponential normalization of Softmax produces sharp distributions at low temperatures or with large logit scaling, leading to "winner-takess-ll" switching behavior among highly competitive keys. Intuitively, this amplifies the sensitivity to perturbations in the input and intermediate states, which is equivalent to increasing the effective Lipschitz constant (1) and the input channel coefficient (C). It also causes locally quasi-discrete transitions in attention weights. Therefore, within the framework of Theorem 2 sharper attention typically reduces the tolerable perturbation radius. Conversely, smoother attention (achieved with high temperature or small scaling factors) mitigates this sensitivity, thereby increasing the robustness radius.
Non- Non-linearity of Normalization Layers SLayerNorm/RMSNorree constrains the norm of hidden states through demeaning and scaling by variance. When statistics are stable, this effectively suppresses Rd and weakens the amplification chain across layers, manifesting as smaller effective values for y and C. This aligns with the monotonic relationship described in Theorem 2 where a smaller norm corresponds to stronger robustness. However, it is important to note that when the intra-layer variance becomes abmormally small (close to zero), the scaling fac- tor can locally amplify noise, creating transient high-gain regions and leading edge cases where robustness decreases. Therefore, stable statistics and moderate pre-scaling (such as layer scaling during training) help ensure the positive impact of normalization on the robustness radius.
Normalization explicitly
Non-linearity of Feed-Forward Network Activations (GELL/ReL function determines the gain of the local Jacobian. In saturated regions (such as the left tail of GELU), the local slope approaches zero, which suppresses noise propagation and limits the norm of intermediate representations, thereby increasing the tolerable perturbation radius. In contrast, high-gain regions (resulting from large weights or strong inputs) amplify the norm of intermediate states and the output sensitivity, which translates to larger effective values for y and C. Gated variants (such as SwiGLU/MoE) can also trigger discrete switching of channels or experts near their thresholds, causing the output to undergo abrupt transitions in response to small perturbations. Overall, operating the activations in low-to-medium gain regions and controlling the scale of the weights helps to reduce the effective sensitivity and decrease R which aligns with the monotonic properties described in Theorem 2
The activation
21

---

Published as a conference paper at ICLR 2026
Residual Paths and Layer Scaling (Semantics of n) The residual path directly injects the repre- sentation from the previous layer into the next, scaled by coefficient, which can be viewed as the in Theorem 2 A larger 7 allows more imput and intermediate perturbations to pass through without attenuation and accumulate in deeper layers, leading to an increase in the effective y and a decrease in the tolerable perturbation radius. Conversely, smaller residual coefficient layer scaling tech- niques (such as the ideas behind ReZero/LayerSeee enhance robustness. A trade-off exists, as overly small 7) can limit feature reuse and gradient flow. In practice, moderate but non-zero layer scaling is often adopted to achieve better compromise between expressive power and robustness under the constraints of Theorem 2
can suppress this long-chain amplification and
D.2 EFFICIENCY OF OUR METHOD
In this section, we discuss the computational efficiency of the improved method we propose in $4. Let M be the total number of candidate prompts, and let T(M, denote the time it takes for the model M to run once on the evaluation dataset D. Then, because calculating each candidate requires the hidden state vector for every data point, which necessitates a full inference pass, the total running time is:
O(M - T(M, D))
(10)
Compared to other prompt optimization methods, many approaches also need to run each generated prompt on an evaluation dataset to assess its quality (e.g.. TextGrad, OPRO). Therefore, the effi- ciency of our method considered comparable to that of previous work. Furthermore, since this paper primarily focuses on theoretical analysis rather than methodological improvements, we leave further enhancements to the effectiveness and efficiency as future work.
D.3 INPUT ROBUSTNESS OF NON-LINEAR SELF-ATTENTION
According to the discussion in 2.. the certified upput-perurrattion radius obtainable via CoT de- pends on the model's Lipschitz properties. In this section, we replace the LSA with a non-linear (softmax) attention and derive counterpart of Theorem [
We adopt a standard single-head attention mechanism with a residual flow. Let Wº, W. WV, WPV E Bgxb be the projection matrices, and let >> be the temperature. Denote the parameters by 0 = (WQ, W WV W.P .T) and let E = [h, x]. Define
= E
K= EWW
V=EWV,
A(E) = softmax (QKT).
where softmax is applied row-wise. To mitigate gradient explosion, we introduce residual coeffi- cient 7) E (0, 1) as in the LSA case. The non-linear attention map is
SAua (h, x; 0) 17 E + (PV(AA())V)
(11)
In what follows, we analyze the input robustness of equation I1 under the same Lipschitz framework as in 2..3
We first upper bound the two Lipschitz constants in equation Throughout, (spectral) norm.
||-|| denotes the operator
Lemma 2. Suppose ||~||| S R2 and ||h|| Ra- Let
sQ =|[ SK = I|W sy = [I SPV spv = Il SQK SQ8K-
Let (T) be the (row-wise) Lipschitz constant of the softmax map with temperature T under the chosen norm. Then the constants C and in equation Z admit the bounds
C CS + 8PV sv (T) SQK R2
7 S
n2 4 R3 (srv 8v TTo(T) sQk)² R2.
Plugging Lemma 2 into the general poputpperuurbaiin bound equation 3 we obtain the following certified radius for non- non-linear attention.
Theorem 3 (Certified Penturbertnnation Radius of Softmax Attention). If ||]||| S Rx and ||l||| s Rj, define
8 8PV 8v (T) 8QK R2
Vip 4 R2 (sPv sy To (T) sQk)² R²
22

---

Published as a conference paper at ICLR 2026
With A such that ||oull Σ A|o0|, the certified tolerable input-pertiin pertuuraertrtnoonn radius of the map equation 11 at CoT step E N+
(1-
||6||
(1 + 3) (A(1 (1 + B)) 7k
In particular, if5 1, K o0,
(1-5)
||0]
B
can be seen that Theorem 3 has a similar format to Theorem 2 showing that they have the same conclusion regarding the CoT robustness. The above discussion shows the effectiveness of our conclusion under the non-linear scenario.
BEW Proof. Write fAm(() n + (E) with ((E) W PV (EVV
By composing Lipschitz bounds of the bilinear map E + @W@(EWW) the row-wise softmax (with constant Lo (T)), and the linear maps WV, PV we obtain the stated bounds on C' and y. Substituting them into equation yielss Theorem
D.4 THE IMPACT OF VECTOR NORMS ON THE COT ROBUSTNESS
While very small weight norms can indeed destabilize optimization during training, their analy- sis specifically targets inference-tinge -ieeece-timeeeee robustness to input perturbations, so there is no contradic- tion-practical models must balance norm size for both stable training and robust inference. We further argue that robustness is better captured by absolute perturbations, i.e., the raw change in the output, rather than relative perturbations that normalize by the input magnitude, because the model ultimately makes decisions based on the absolute output vector. For example, in multiple-choice setting, a substantial absolute shift in logits can change the predicted option even if the relative change is small. Therefore, we frame their conclusions in terms of absolute perturbation as more faithful indicator of decision instability.
D.5 ANALYSIS WITH MULTIPLE TOKENS
In our analysis, we treat the entire user query as single embedding vector I E Rd, and the pertur- bation 5 E Id acts on this query-level representation rather than on individual token embeddings.
The bivariate map (h, x) therefore models the interaction between (i) the current hidden state and (ii) fixed embedding of the full input query, not between two literal tokens.
We now show that this is mathematically equivalent starting from the usual multi-token trans- former input.
Corollary 1 (Equivalence of sequence input and bivariate model). Let the model at a given CoT step take as imput
hidden state h Rd,,, summarizing all past reasoning tokens; and
a sequence of question tokens with embeddings (ei,...,eT), each et E RRde.
Assume its next-step hidden state is given by some deterministic map
Faa : TRd x (RR..)T - Rdda.
Then there exist
1. linear isomorphism (Rd.)T Rd (vectorization padding), and
2. a bivariate function f Rd x Rdd Rd
such that the dynamics can be written exactly as
h f(1,,z)), x U(e.....TT).
23

---

Published as a conference paper at ICLR 2026
Moreover, if Fjui is Lipschitz in (h, (et)s), then satisfies a Lipschitz condition of the form (If (222 ?llii h2ll + C||21 -2ll
(12)
for some constants 7, 7,
Proof. Step 1: Reparameterizing the input sequence as a single vector. Fix maximum sequence length max 2 and pad (ei,..., CT) with a distinguished padding embedding so that every input can be regarded as an element of (IRd)) ) This space is linearly isomorphic to IE with d de maax.
Let
U (Rd)) | + R
be any fixed linear bijection (e.g., concatenation followed by a permutation of coordinates). Define eT, Conversely, U-1 recovers the full token-level embedding tuple from z.
T == U(e,, er, er,ped,.... pad..... .pd)) R.
Step 2: Defining the bivariate funccton. Define
x() Frun(h, U-
By construction,
hg+1 Finn( has (el (ei,...,,T)) eT)) f shx, U(l f(h,, x),
so the original multi-token dynamics can be written as a bivariate map in (h,
Step 3: Preservation of Lipschitz continuity. Suppose the model is Lipschitz in (h, (et)t): i.e.
there exist constants 7 2 O, Cook 2 0 such that for all h2, h2 and token sequences (et), (e)),
Fiun (h1, (e+)) (12, (c))) s >l1 h2 |l + Cook (ee) (c()
seq
where || llseq is any norm on (imee)
Using the linear isomorphism U, equip Rdd with the induced norm
| = I (x=)(y)lls
Then for any (h1, (h2, x22) (If(h1, f(h2, x2)1
|| Fru (/1, U 21) (hun U0 U-'z2)))) 'x2)
< <h1 hell Cook (x1) U- (x2) sey
(x1)
> h2ll + Cook || - I2l
Thus f satisfies the Lipschitz condition with constants y and C = Clok-
E ADDITIONAL INFORMATION
E. EXPERIMENTAL DATASET
benchmark for competition-level mathematical reasoning. comprising 12, 500 problems with full step-by-step solutions (7, 500 training and 5, 000 test). It spans diverse subfields (e.g., algebra, geometry, number theory, combinatories, probability, and cal- culus) and is widely used to evaluate and distill chain-of of-thought style reasoning in mathematics. In this paper, we evaluate our conclusions with the subset of MATH, which contains 500 data following Lightman et al. (2024).
MATH (Hendrycks aL. 2021)
MMLU-Pro (Wang et al. 2024c) It a strengthened successor to MMLU that emphasizes higher question quality and robustness. It contains over 12, 000 multiple-choice questions drawn from text- books and exams across 14 academic domains (e.g.. biology, business, chemistry, computer sci- ence, economics, engineering, health. history, law, mathematics, philosophy, physics, psychology, and others). Each item offers 10 options, which reduces guessability and increases discrimination among strong models.
24

---

Published as a conference paper at ICLR 2026
GPQA (Rein et al., 2024) targets graduate-level, "Google-prood" scientific reasoning. The test set includes 448 expert-authored multiple-choic questions in biology. physics, and chemistry, designed such that even with open-web access, non-experts struggle while domain experts achieve only mod- est accuracy. GPQA thus probes high-level knowledge, multistep reasoning, and model reliability under stringent oversight conditions.
E.2 PROMPT
Prompt MATH
Solve the following math problem efficiently and clearly. Regardless the approach, always conclude with: Therefore, the final answer is: Sboxed (boxcd(answers [answer]S. hope it is correct.
Prompt of MMLU-Pro
The following are multiple choice questions (with answers) about domain.
Think step by step and then finish your answer with the answer is (X)where X is the correct letter choice.
Prompt of GPQA
Given the following question and four candidate answers (A, B, C and D), choose the best answer.
For simple problems:
Directly provide the answer with minimal explanation.
For complex problems:
Use this step-by-step format:
## Step 1: [Concise description]
(Brief explanation)
## Step 2: [Concise description] (Brief explanation)
Regardless of the approach, always conclude with: The best answer is [the letter where the athe_answer_lee letter) is one of A, B, C D.
Let's think step step.
Table 4: The prompt used this paper.
In this section, we list the prompt we used in Table
E.3 PROMPT NUMBER OF EACH SETTING
In this section, we present number of prompts used for each dataset and model, as shown in Ta From the table, settings. This because, during prompt optimization, the suitable prompts vary for different models and datasets. To ensure that optimal performance is achieved for each setting, we use different set of prompts for each setting.
observe that the number of prompts is not consistent across different
Dataset
Llama2-7b
Llama3.1-8b
Llama-RI-8h
Qwen3-8b
MATH
14
29
18
13
MMLU-Pro
20
29
20
16
GPQA
11
20
16
12
Table 5: The total number of generated prompts using TextGrad, OPRO, and CFPO under each setting.
25

---

Published as a conference paper at ICLR 2026
E.4 CALCULATION OF OUTPUT FLUCTUATION
Consider a collection of model outputs produced for the same input, represented as multiset of strings of size M. Let p; denote the empirical frequency of the i-th distinct string. The metric computes the Shannon entropy:
H
Σ
pi loga Pi:
(13)
and normalizes by the maximal entropy achievable with M samples, namely log, M. The resulting index:
log2 M E [0.
(14)
is scale-free and directly comparable across different sample sizes. By construction, H = 0 when all outputs are identical (complete consensus, fluctuation) and H = when all M outputs are distinct (maximal dispersion, each outcome occurs once). For empty or singleton sets, the metric defined to be 0, reflecting the absence of observable variability.
Output fluctuation manifests as dispersion in the empirical outcome distribution. Greater variability spreads probability mass more evenly across distinct strings, driving H toward its maximum and increasing f. Greater stability concentrates mass on single outcome, driving H toward 0 and decreasing Å. Normalization by logz M ensures that the same qualitative level of dispersion yields comparable scores even when the number of samples differs, while preserving the desired extremes ("all same" + 0, "all different" 1).
E.5 IMPLEMENTATION DETAILS
The input and hidden state vectors used in our experiments are the encoded vectors from the embed- ding layer and the final layer of the respective LLMs for the corresponding inputs. For each input, set the model to generate single output, with the temperature set to O, top-p to 1.0, and the random seed fixed at 42. Our experiments are run on a single A100-80G GPU, with the average experiment time for each setting being approximately one hour. All our codes are implemented with PyTorch (Paszke et al. 2019), Transformers (Wolf et al., (020), and VLLM (Kwon et al. 2023) using Python3. 10. We detail how plot the analysis figure in Appendix E.6
E.6 PLOT ANALYSIS FIGURE
We visualize conditional distributions with an x-binned box-plot design. The z-range is uniformly partitioned into K 10 equal-width bins; within each bin we compute the first quartile (Q1), median, and third quartile (Qs). Whiskers follow Tukey's 's rule and extend to the most extreme observations within [Q= 1.5 IQR, Qs + 1.5 IQR], where IQR Q3 Q1; bins with very few points are shown by median marker only.
To convey the trend across bins, the binwise medians are connected by a cubic Hermite interpolant (PCHIP). An optional interquartile ribbon is drawn by interpolating Q and Qa with the same scheme. For context, we overlay lightly jittered raw points the background and add marginal density curves along the top (for and the right (for y), estimated via Gaussian KDE with Silverman's 's bandwidth; the right-hand marginal can be computed from an alternative sample when provided. Box widths adapt to local bin spacing prevent overlap in narrow z-ranges, and a unified www-satrattinnnnoo color palette is used for visual consistency.
piecewise
F ADDITIONAL EXPERIMENT
F.1 FITTING 7 OF THEOREM 1
In this section, we verify that 7 < 1 to ensure the reliability of the conclusions derived from Equa- tion 4 Since the right-hand side of the inequality in Theorem i positively correlated with y, we consider the extreme case by replacing the inequality with an equality, which gives:
|| = (Ar K + 11 (1-1--0) xx)) ||
(15)
26

---

Published as a conference paper at ICLR 2026
Model
MATH
MMLU-Pro
GPQA
Llama2-7b
0.662
0.892
0,671
Llama3.1
0.476
0.218
0.014
Llama-RI R1-8-R--bbb -88b
0.879
0.896
0.871
Qwen3-8b
0.754
0.744
0.015
Table 6: The fitted 7 different models and datasets.
Amazon
FinQA
ToolE
Model
EM
OF
EM
OF
EM
OF
Llama2-7b
17.4 11.9
0.711
5.9 3.7
0.479
27.1
17.6
0.383
Llama3. 1-8b
61.1 1 35.6
0.271
37.6 + 6.1
0.377
49.8
1 18.0
0.365
Llama-R R1-8- 1-8b
60.3 + 39.8
0.242
45.7 = 7.4
0.276
51.5
+1.5:00
4.9
0.162
Qwen3-88 8b
61.1 61.1
0.201
54.9 $ 5.9
0.246
56.0 6.6
0.121
Table The performance on Amazon Rview (Amazon), FinQA, and ToolE.
Then, for each question across all datasets, we compute the corresponding ||6]| and || for differ- CoT steps K among all generated answers. We use this data fit the parameter y in Equation [3] using the least squares method. The fitting results are shown in Table From the table, we can ob- serve that the value of y less than in all settings, which validates the reliability of the assumption made in our analysis.
F.2 PERFORMANCE ON MORE DATASETS
To more comprchensively validate the changes output fluctuation across different datasets, we conduct experiments broader range of datasets. We conduct experiment the Amazon Review et al. (019) (sentiment analysis), FinQA (Chen et al.) [221] (financial question answering), and ToolE (Huang al. 2024) (tool use) datasets verify our conclusions in scenarios that more closely resemble real-world applications. The experimental results are presented in Table From the table, observe that as the model performance improves, the output fluctuation shows an overall downward trend. This is consistent with the conclusions we draw in Table 2
F.3 OUTPUT FLUCTUATION WITH OTHER METRIC
To more comprehensively evaluate fluctuations model outputs, this section quantifies semantic variability (SV) across models and datasets. For each question, we first compute embedding vector for every answer using all-Minii.M-L.-vv2 (Wang al., 20206). distance from these vectors their centroid (mean vector) the metric of output variability. As shown in Table g this metric exhibits strong agreement with OF, and the experimental findings are consistent with those in Table 8 thereby corroborating the correctness of our theoretical analysis.
then take average
F.4 PERFORMANCE WITH SAME PROMPTS
To ablate the effect of prompt differences on the evaluation, we conduct experiments using the same prompts across all models and datasets, For all models, we employ the prompts generated with Q-b. The results, as shown in Table 9 indicate that the conclusions drawn from using
MATH
MMLU-Pro
GPQA
Model
SV
OF
SV OF
SV
OF
Ilama2-7b
0.851
0.475
0.706
0.622
0.760
0.509
Ilama3. 1-8b
0.777
0.366
0.631
0.350
0.667
0.467
Ilama- 1-8h
0.669
0.158
0.601
0.292
0.575
0.371
Qwen3-8b
0.653
0.097
0.579
0.162
0.554
0.214
Table 8: The output fluctuation using different metrics.
27

---

Published as a conference paper at ICLR 2026
MATH
MMLU-Pro
GPQA
Model
EM
EM
EM
OF
Llama2-7b
12.2
0.653
13.8
0.578
15.9
0.523
Llama3. 1-8b
48.9
0.375
35.1
0.347
28.1
0.470
Llama-RI Minaa-R1-81 -1-8b
65.4
0.147
40.2
0.303
30.0
0.404
Qwen3-8b
77.2
0.097
46.9
0.162
37.3
0.214
Table 9: The average EM and OF of different models and datasets. prompts of each model are all same with Qwen3-8b.
For the certain dataset, the
MATH
MMLU-Pro
GPQA
Model
Scale
EM
OF
EM
OF
EM
OF
Llama3.1
45.8 7.2
0.366
41.0 + 10.7
0.350
=
0.467
70b
56.0 + 12.8
0.284
63.0 +: 14.4
0.186
42.6 = 9.7
0.232
8b
77.2 1.6
0.097
46.9 5.2
0.162
1.9
0.214
Qwen3
34b
80.8 + 4.0
0.075
67.8 5.1
0.104
43.6 = 3.2
0.177
Table 10: The performance of Llama3. 1 and Qwen3 on each dataset with different model scales.
identical prompts are consistent with those in Table Therefore, in our main experiments, to ensure a consistent methodology, we use each model to generate the prompts for its own inference.
F.5 PERFORMANCE CROSS DIFFERENT MODEL SCALE
To verify how output fluctuations change with input perturbations models of different scales, measure the performance of models of varying scales on each dataset. The experimental results are shown in Table TO From the table, can find that although the EM of larger-scale models could exhibit greater fluctuations, from the perspective of OF, larger-scale models generally demonstrate better input robustness. This because larger-scale models tend to generate a greater number of reasoning steps K (Wei et al., 2022 Kojima et al. (022) and possess stronger ability to widen the confidence gap between correct and incorrect answers, which turn increases the acceptable perturbation threshold R. (Zhu et al. 2023; Chhikara 2025). Consequently. according to Theorem larger-scale models exhibit better robustness.
F.6 PERFORMANCE WITH COT STEPS UNDER EACH SETTING
In this section, we list how performance varies with CoT steps under different models of MATH in Figure
F.7 PERTURBATION WITH EMBEDDING NORM UNDER DIFFRRENT SETTINGS
The variation of output perturbation with respect embedding norm for all models on various datasets is illustrated in Figure to Figure
28

---

Published as a conference paper at ICLR 2026
Oulput Fluctuation (OP)
Output Fluctuation (OF)
0.8
0.01
Exact Match (EM)
0.8
Exact Match (EM)
0.6
OF
0.03
0.6
OF
0.8
EM
EM
0.6
0.4
0.02
0.4
0.4
0.2
0.01
0.2
0.2
0
0
0
2
3
2
4
CoT Steps
CoT Steps
MATH, Llama2-7b.
(b) MATH, Llama3.1-80
Output Fluctuation (OF)
Ontpuu Flictuation (OF)
0.8
0.8
Exact Match (EM)
0.8
Exact Match (EM)
0.6
OF
0.6
0.6
OF
0.8
EM
EM
0.6
0.4
0.4
0.4
0.4
0.2
0.2
0.2
0.2
0
2
CoT Steps
CoT Steps
(c) MATH, Llama-R1 R1-8h.
(d) MATH, Qwen3-8h.
Figure 6: EM and OF on MATH cross CoT steps with different models.
Oogged Blustosons 100
20 160 130 126
106 350
76 30
160 110 120
Ingut www.-ott. Watter Nome
Bnpar Sibwwenng Vocter Norm
Irpat Embwddingg Vester
Figure The change in out- put fluctuation with the norm of the input embedding vec- tor across all experimental mod-
Figure 8: The change out-
Figure 9: The change in out- put fluctuation with the norm of the input embedding vec- tor across all experimental mod- els on GPQA. Each point de-
put fluctuation with the norm of the input embedding vector across all experimental models
els on MATH. Each point de-
on MMLU-Pro. Each point de-
notes the result of question,
notes the result of one question,
notes the result of one question,
where X- X-axis denotes the input
where X-axis denotes the input
where X-axis denotes the input vector norm and Y-axis denotes
vector norm and Y-axis denotes OF of this question. The Pear- son coefficient is 0.415.
vector norm and Y-axis denotes
OF of this question. The Pear-
OF of this question. The Pear-
son coefficient is 0.634.
son coefficient 0.541.
29


Reviews:
[{"review_id": "fQVU95237H", "reviewer": "Reviewer_M3bE", "summary": "This paper studies the robustness of CoT reasoning and theoretically proves that while increasing reasoning steps can partially mitigate the effect of input perturbations, even infinitely long chains cannot completely eliminate them. Further analysis on the simplified Linear Self-Attention model shows that larger input embedding and hidden state norms reduce robustness. Experiments on several mainstream large models and reasoning datasets validate the theory, and a new prompt selection method based on norm minimization is proposed to effectively enhance robustness.", "questions": "see weaknesses", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "This paper provides a formal theoretical derivation of the robustness of CoT reasoning and establishes an upper bound on output fluctuations under input perturbations.", "weaknesses": "1. This paper primarily adopts OF as the robustness metric, but does not account for semantic-level robustness. Could this limitation lead to an underestimation of the model’s actual robustness?\n\n2. This paper does not specify the exact prompt templates used in the experiments. Could the authors clarify the prompt design choices?\n\n3. The analysis suggests that larger input embedding norms reduce robustness. However, given that different models vary in embedding initialization and pretraining corpora, how generalizable is this finding across architectures?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ntN5zQHSDs", "reviewer": "Reviewer_K3vW", "summary": "This paper provides an analysis on the robustness of CoT reasoning w.r.t. input perturbations. The authors derive output upper bounds on how input perturbations propagate through CoT reasoning, claiming that increasing the number of reasoning steps reduces the fluctuation, but it can't be fully eliminated. Besides they show that robustness is negatively correlated with input and hidden state norms.", "questions": "I am not sure about this but the authors considered multiple assumption to justify their modeling. Is there any prove that these assumption doesn't make the results to diverge from real cases?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "Providing a formal mathematical model of CoT perturbation propagation and finding an upper bound on robustness.", "weaknesses": "From what i understand the Lipschitz continuity assumption is an over simplifying assumption", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "I5vekkZhBs", "reviewer": "Reviewer_4uQH", "summary": "The authors propose a theoretical analysis of how input perturbations propagate through CoT reasoning in LLMs. It derives upper bounds on output fluctuations under Lipschitz continuity, showing that: (i) increasing reasoning steps mitigate, but never remove, the impact of input perturbations; and (ii) robustness is negatively correlated with the norms of input embeddings and hidden state vectors. The analysis is instantiated on a Linear Self-Attention model, treated as a simplified Transformer, and validated through experiments on four models and three benchmarks. Empirical results align with theory, confirming that longer reasoning chains dampen but do not eliminate perturbation sensitivity. The authors also propose a simple prompt-pick strategy that maximises input-perturbation tolerance, yielding modest improvements.", "questions": "- Theoretically, why should the negative correlation between robustness and embedding norms hold across non-linear, attention-based architectures?\n\n- Beyond quantifying bounds, how might your framework inform practical interventions to improve CoT robustness during training or inference?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- Theoretical grounding: The paper offers one of the first formal derivations of CoT robustness bounds, establishing links between perturbations, reasoning steps, and embedding norms.\n\n\n- Clear empirical validation: Experiments on diverse datasets and models convincingly support theoretical claims.\n\n- Insightful analysis of LSA: Using Linear Self-Attention provides tractable yet informative insights into Transformer dynamics.", "weaknesses": "- Simplified modelling assumptions: The reliance on LSA as a proxy for real Transformers oversimplifies key non-linear dynamics (softmax attention, layer normalisation).\n\n- Lack of deeper interpretability: While the derivations are sound, the paper does not explore the mechanistic implications for CoT reasoning or noise propagation in practical LLMs.\n\n- Marginal methodological contribution: The proposed prompt optimisation method adds limited value and feels auxiliary to the theoretical core.\n\n- Lack of transparency: The authors could have shared the code in order to improve reproducibility and be transparent", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NQFw5nliwI", "reviewer": "Reviewer_Kkzt", "summary": "This paper provides a theoretical investigation into the robustness of Chain-of-Thought (CoT) against input perturbations, supported by empirical tests on a range of open-source models to validate their arguments. The authors model the CoT process as a bivariate Lipschitz function and leverage classical stability results of Lipschitz functions to derive stability relationships for the Linear Self-Attention (LSA) model. Through theoretical analysis and experimental validation, the authors find that: (1) model performance is positively correlated with robustness; (2) longer reasoning chains reduce the impact of input perturbations, but cannot eliminate it entirely; and (3) the norms of the embedding and hidden state vectors are negatively correlated with model robustness. The paper also proposes a novel prompt optimization method aimed at enhancing model robustness.", "questions": "See weakness.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. This paper rigorously derives bounds on output fluctuation under input perturbation, building from a Lipschitz continuity framework (Eq. 1, Thm. 1) to a specialized analysis of Linear Self-Attention (LSA). This progression yields novel insights, crucially linking perturbation robustness to the norms of input and hidden state embeddings (Thm. 2).\n2. The experimental evaluation is comprehensive, conducted across multiple datasets and state-of-the-art LLMs. The results, notably in Table 1 and Figures 1-3, provide compelling quantitative evidence for the paper's central claims by directly visualizing the relationships between input perturbation, reasoning steps, and output fluctuation.", "weaknesses": "1. The paper models the LLM's reasoning process directly as a bivariate function. However, in reality, the input to an LLM during reasoning clearly consists of more than two tokens. The authors need to more clearly demonstrate why this model holds.\n2. The authors primarily provide a theoretical analysis for the Linear Self-Attention (LSA) model in the main text, while the discussion of other modules is relegated to the appendix and remains rather superficial. A more in-depth discussion on the impact of these other components on the theoretical results is necessary, or alternatively, the authors should justify why the LSA findings are sufficient for analyzing the entire model.\n3. In the experimental section, the authors need to clearly describe their procedure for perturbing the input prompts and explain how they modify the norms of the embedding and hidden state vectors.\n4. In the results shown in Figures 2 and 3, the model performance declines as the length of the Chain-of-Thought increases. This suggests that, under the experimental setup of this paper, extending the number of CoT steps may no longer be beneficial. Consequently, the validity of the conclusions related to Output Fluctuation is called into concern. The authors should identify more suitable tasks where increasing CoT steps demonstrably improves model performance, and then examine the behavior of OF under such conditions.\n5. The results presented in Figures 4 and 5 do not provide compelling support for the authors' claims. Particularly in Figure 4, aside from the notable jump between 60-70, the remaining data points lack a clear monotonic increasing trend.\n6. The paper identifies that larger embedding and hidden state norms reduce CoT robustness, a finding that appears counterintuitive from a practical training perspective. Standard techniques that effectively constrain these norms (e.g., aggressive weight decay, smaller initialization) are well-known to potentially harm training stability. Furthermore, if the absolute magnitude of the perturbation is held constant, a smaller vector norm would intuitively represent a larger *relative* perturbation, which one might expect to be more detrimental to robustness. The authors should provide a more intuitive explanation and discussion to reconcile this theoretical finding with practical intuition. For instance, is the analysis sensitive to the assumption that the perturbation is absolute rather than relative? A discussion on this apparent discrepancy is crucial for the audience to properly interpret and trust the theoretical results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)