Title: Efficient Reasoning Through Suppression of Self-Affirmation Reflections in Large Reasoning Models
Venue: NeurIPS 2025
Accepted: no
Keywords: efficient reasoning

reasoning model
Abstract: While recent advances in large reasoning models have demonstrated remarkable performance, efficient reasoning remains critical due to the rapid growth of output length. Existing optimization approaches highlights a tendency toward "overthinking", yet lack fine-grained analysis. In this work, we focus on Self-Affirmation Reflections: redundant reflective steps that affirm prior content and often occurs after the already correct reasoning steps. Observations of both original and optimized reasoning models reveal pervasive self-affirmation reflections. Notably, these reflections sometimes lead to longer outputs in optimized models than their original counterparts. Through detailed analysis, we uncover an intriguing pattern: compared to other reflections, the leading words (i.e., the first word of sentences) in self-affirmation reflections exhibit a distinct probability bias. Motivated by this insight, we can locate self-affirmation reflections and conduct a train-free experiment demonstrating that suppressing self-affirmation reflections reduces output length without degrading accuracy across multiple models (R1-Distill-Models, QwQ-32B, and Qwen3-32B). Furthermore, we also improve current train-based method by explicitly suppressing such reflections. In our experiments, we achieve length compression of 18.7\% in train-free settings and 50.2\% in train-based settings for R1-Distill-Qwen-1.5B. Moreover, our improvements are simple yet practical and can be directly applied to existing inference frameworks, such as vLLM. We believe that our findings will provide community insights for achieving more precise length compression and step-level efficient reasoning.

Abstract

While recent advances in large reasoning models have demonstrated remarkable performance, efficient reasoning remains critical due to the rapid growth of output length. Existing optimization approaches highlights a tendency toward "overthinking", yet lack fine-grained analysis. In this work, we focus on Self-Affirmation Reflections : redundant reflective steps that affirm prior content and often occurs after the already correct reasoning steps. Observations of both original and optimized reasoning models reveal pervasive self-affirmation reflections. Notably, these reflections sometimes lead to longer outputs in optimized models than their original counterparts. Through detailed analysis, we uncover an intriguing pattern: compared to other reflections, the leading words (i.e., the first word of sentences) in self-affirmation reflections exhibit a distinct probability bias. Motivated by this insight, we can locate self-affirmation reflections and conduct a train-free experiment demonstrating that suppressing self-affirmation reflections reduces output length without degrading accuracy across multiple models (R1-Distill-Models, QwQ-32B, and Qwen3-32B). Furthermore, we also improve current train-based method by explicitly suppressing such reflections. In our experiments, we achieve length compression of 18.7% in train-free settings and 50.2% in train-based settings for R1-Distill-Qwen-1.5B. Moreover, our improvements are simple yet practical and can be directly applied to existing inference frameworks, such as vLLM. We believe that our findings will provide community insights for achieving more precise length compression and step-level efficient reasoning.

1 Introduction

Recent advancements [23, 43, 2, 17] in large language models have delivered remarkable performance across various tasks [10, 4], yet they still struggle with complex reasoning tasks demanding advanced mathematical [15, 22] and formal logical deduction [35] abilities. Works [18, 17, 12, 38, 36] like Deepseek-R1 [12] have driven progress in reasoning models via reinforcement learning. However, current large reasoning models are troubled by high token usage and computational costs due to generating lengthy reasoning chains. Researchers have been delving into different technical strategies to enhance reasoning efficiency, like refining latent representations [14, 33, 7] and combining small models [27, 11]. In order to directly optimize the model itself, several approaches have emerged as promising methods, using supervised fine-tuning (SFT) [41, 13, 25], direct preference optimization (DPO) [32], and reinforcement learning (RL) [3, 24, 3, 1, 44].

Although these methods have yielded impressive results, such as producing more semantically 33

efficient or direct solutions, we observed that redundant reflection persists in the optimized models 34

[3, 24]. Rather than attributing this phenomenon to the more general "overthink" issue (defined in 35

36

37

38

39

40

41

work [34] as the generation of excessively detailed or unnecessary lengthy reasoning steps by large reasoning models during inference, which undermines reasoning efficiency), we conducted a more

detailed examination of these examples (for more details, refer to

Section 3

). As illustrated in Figure

1, we have identified a common phenomenon: affirming prior content typically occurs following correct steps, which we refer to as Self-Affirmation Reflection.

In some cases, this superfluous reflection even leads to longer answers for certain problems in the optimized models.

Since this phenomenon is also widespread in 42 the optimized models [3, 24], such reflection 43 essentially originates from the base models. 44 Therefore, we investigate scenarios where origi45 nal reasoning models frequently execute self46 affirmation reflection. Empirical analysis re47 veals a notable phenomenon: During repeated 48 rollout processes at reflection points, models 49 do not consistently engage in reflection behav50 ior, as illustrated in the yellow box of Figure 51 1. In fact, after generating correct intermediate 52 steps, models often exhibit ambiguity in transi53 tioning between problem-solving steps and re54 flection initiation. Since prior works [31, 26] 55 have established that certain keywords, termed 56 leading words in this paper, significantly influ57 ence a model's decision to trigger the reflection 58 mechanism. To characterize the self-affirmation 59 reflection type, we conduct statistical analysis 60 comparing its leading word distribution to those 61 of other reflection categories. As shown in Sec62 tion 4 , self-affirmation reflection exhibits sta63 tistically significant divergence in leading word 64 distribution patterns. 65

Building on this insight, we identify conditions 66 under which self-affirmation reflections occur 67 during model reasoning and present a straight68 forward yet effective method that explicitly sup69 presses rethinking confidence at these crucial 70 confusion points. Notably, our approach entails 71 no extra cost and is extremely simple to imple72 ment. It can be seamlessly integrated into cur73 rent inference frameworks such as vLLM [19]. 74 Under a train-free experimental setup, while 75 maintaining nearly the same performance level, 76

How many ways can we put 3 math books and 5 English books

ona shelf if all the mathbooks must stay together and all the

English books must also stay together? (The math books are all

different and so are the English books.)

Okay,so I have this problem here:I need tofigure out howmany

waysI canarrange3mathbooks and5Englishbooksonashelf.

The catch is that all the math books have to stay together, and ...

...

Wait, let me double-check my reasoning. So, treating the math

books as oneblock and the Englishbooks as another block,we

have 2blocks.The numberof...

Wait, but in the first approach, the math block can be placed in 6

different starting positions, and the English block is placed in

the...

So, 6 x 3! x 5! = 6 x 6 × 120 = 4320. Yes, that's correct.

[Rollout 1] Wait, let me double-check...

[Rollout 2] Let me double-check...

[Rollout 3] But wait, let me double-check...

[Rollout 4] Just to make sure I didn't make a

D

mistake, let me think through it again...

[Rollout 1] Wait, but in the first approach, the ...

[Rollout 2] So, 6 × 6 × 120 = 4320.

[Rollout 3] Therefore,6 × 3! × 5! = 6 × 6 × 120 =

4320.

[Rollout 4] Thus, 6 × 3! × 5! = 6 × 6 × 120 = 4320.

we have achieved length reductions of 18.7%, 14.3%, 11.1%, 9.1%, and 8.4% for R1-Distill-Qwen77 1.5B/7B/32B, QwQ-32B, and Qwen3-32B respectively, and in some cases, even better performance 78 is obtained. Furthermore, we apply our method to a representative train-based approach [3], which 79 generates responses of varying lengths by model self-sampling. Correct responses are prioritized, 80 with shorter lengths receiving higher rewards. Our method complements this strategy by explicitly 81 suppressing self-affirming reflections within positive samples and achieve significantly shorter outputs 82 with competitive performance. Extensive experiments in Section 5 across different settings validate 83 the feasibility and significance of the findings in this paper. 84

Our work offers three key benefits: (1) We provides the first focused analysis of the Self-Affirmation 85

86

87

88

89

90

91

92

Reflection phenomenon and provides actionable insights for improving reasoning models

:

We present a simple yet practical intervention strategy

. (2)

suppressing crucial leading tokens.

This directly lessens Self-Affirmation Reflection, achieving efficient output compression without modifying the model architecture or training process.

models and datasets in

(3) Extensive experiments across diverse both train-free and train-based settings

demonstrate that mitigating Self-

Affirmation Reflection can effectively reduce output length while preserving or even enhancing performance.

2 Related Work 93

94

95

96

97

98

Recent surveys [29, 39, 34] have summarized progress in efficient reasoning. In this section, we highlight representative methods to provide a concise overview of key developments in this area.

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

89.9%

AllLongerResponses

12%

Longer Responses due to Self-Affirmation Reflection

10%

8%

6%

84.2%

82.2%

4%

83.3%

2%

73.3%

0%

O1-Pruner

ER(α = 0.05)

ER(α = 0.1)

ER(α = 0.2)

ER(α = 0.4)

The first category of methods [13, 42, 20] aim to obtain some information from the input to estimate the cost in advance. Some methods [13, 42, 20] choose to estimate the reasoning cost. For example, TokenBudget [13] instructs models to estimate the token count required to solve a problem upfront, compelling them to generate concise responses within a predefined budget. Other methods [27, 5, 9, 8] estimate the model size. Approaches like RouterLLM [27] and Sot [5] pre-train classifiers to route problems to the most suitable LLM based on task complexity. In contrast, router-free methods such as Self-ReF [9] and Seek-Router [8] predict the necessity of invoking additional LLMs by analyzing internal uncertainty scores. However, how to accurately determine the optimal cost is still a problem.

Recent advances in the second category of methods have focused on refining the output of optimization models. As demonstrated in [28], simply adding intermediate "thinking" tokens, or even nonsensical fillers like "......", can lead to satisfactory performance. Subsequent works, such as those in [14, 33, 7], have explored treating the last layer hidden states of LLMs as "continuous thinking" to replace traditional discrete markers. These approaches optimize latent representations through training. However, such methods may render parts of the thought chain invisible, potentially introducing security risks. In addition to optimizing the output sequence, there are also methods to choose to switch the model during output, such as RSD [21] integrating multiple LLMs to achieve dynamic reasoning. These methods demand precise determination of when to switch models and which model to switch to.

The third category of methods centers on fine-tuning LLM itself to directly address redundancy in the output of the original model. Generally, these methods [24, 3, 3, 1, 44] first obtain responses of varying lengths via model self-sampling. Then, they design a length-based reward conditioned on correctness, with shorter correct answers receiving higher rewards. However, balancing accuracy and output length remains a significant challenge. Although these methods reduce average response length of the entire datasets, they do not guarantee shorter outputs across all instances. Specifically, the figure 2 highlights that at the instance level, some outputs (depicted in blue) remain longer than those from the original model. Our train-based experiments in Section 5.2 is closely related to these methods but differs in focus. Rather than focusing on reward function design, our objective is to directly optimize sampling outcomes, thereby refining the quality of the constructed data pairs and achieving more succinct responses.

3 Observation

Our observation begins with experiments evaluating two representative open-source works [24, 3].

We tested R1-Distill-Qwen-1.5B [12] and QwQ-Preview [37] on the MATH500 dataset [22] alongside their compressed counterparts (EfficientReasoning [3] and O1-Pruner [24]). For each question, three

responses were sampled, with average response length recorded.

As shown in Figure 2, results exhibit a counterintuitive trend: certain ratio of instance-level questions 143 elicit longer responses, yet most of these extended responses remain correct (e.g., 91.6% in QwQ144 Preview settings). This pattern persists across varying compression levels. For instance, the parameter 145

α in EfficientReasoning[3] balances brevity and accuracy. Larger α values shorten outputs at the cost 146 of reduced accuracy. As shown in Figure 2, even at mild compression ( α = 0 . 05 ), 6% of questions 147 generated longer responses. Aggressive compression ( α = 0 . 4 ) still failed to resolve this behavior, 148 though the trade-off led to a significant drop in accuracy. 149

150

151

152

153

154

155

156

To gain deeper insights into the optimized models, we conducted a manual examination and analyzed their qualitative behaviors, as shown in Figure 3. Two key characteristics consistently emerge: (1) the model tends to produce more concise steps during the problem-solving phase. For instance, the brown text in Figure 3 illustrates how the optimized model succinctly summarizes the problem background. (2) certain questions have been solved using more direct approaches. This arises because, during the self-sampling process, the model generates multiple potential problem-solving methods. By favoring shorter solution ideas, the model discards superfluous strategies.

157

158

159

160

161

162

163

164

165

166

However, we also observe that the model frequently engages in prolonged reflections even after executing correct reasoning steps or arriving at the correct answer, resulting in excessively lengthy outputs. Figure 3 illustrates this issue: compared to the baseline R1-Distill-Qwen-1.5B model [12], the optimized EfficientReasoning model ( α = 0 . 05 ) [3] achieves the correct solution more rapidly but produces lengthier responses due to increased self-reflections. These reflections often show a tendency to affirm the previous content. Quantitative results in Figure 2 further confirm that this pattern is prevalent across current efficient reasoning models [3, 24]. Overall, these findings suggest that naively optimizing for solution-level simplicity is not enough to effectively compress the model's outputs. This indicates that existing frameworks may still have opportunities to enhance output simplicity by suppressing step-level self-affirmation reflections.

Notably, while this phenomenon was widely summarized as an overthink problem in previous works 167 [29, 39, 34], its persistence in optimized reasoning models underscores the need for targeted solutions. 168 Therefore, we formally define this behavior as self-affirmation reflection, as introduced in Section 169 1, and focus on it in this paper. To address this issue fundamentally, we analyze how to locate and 170 suppress self-affirmation reflection in the original model, as detailed in the subsequent Section 4. 171

QuestionfromMATH500

Jim and Martha are standing together at the corner of a rectangular field. Jim walks diagonally across the field.Martha gets to the same location by walking along its length and width.The field is 300 feet wide

and 400 feet long. How many feet less than Martha does Jim walk?

Responsefromoriginalreasoningmodel

Responsefromoptimizedreasoningmodel

Okay, so Jim and Martha are standing at the same corner of a rectangular field.he field is 300feet

wide and 400 feet long. Jim is walking diagonally across the field, while Martha is walking along the

rectangular field. Jim walks diagonally across the field, and Martha walks along the length and width.

length and width. I need to find out how many feet less than Martha does Jim walk.

The field is 300 feet wide and 400 feet long.I need to figure out how many feet less than Martha

does Jim walk.

Hmm,let me visualize this.The field isarectangle, so it has two dimensions: length and width.Length

is 400 feet, and width is 300 feet.So,iflimagine this, the field is longerin the length direction.

Hmm,okay,letme visualize this.heres arectangularfield,soit has length and widthJim istaking

Martha is walking along the length and width.Ithink that means she goes along one side and then

the diagonal path,which wouldbethe straight line from one cornerto the opposite corner.Martha is

the adjacent side.So, she would walk thelength of the field plus the width.That makes sense

walking along the length and then the width,whichmeans she's taking the two sides togetto the

because walking along the edges would cover the two sides she's not walking diagonally.Jim, on the

same point. So, the question is asking about the difference in distance between Jim's diagonal path

other hand, is walking diagonally across the field.That means he's taking the straight line from one

subtract them to find out how much less Jim walks. [190 tokens][The description of the problem

theorembecause thediagonalforms aright trianglewiththelengthand width.Alright,so tofindout

background is more concise.]

howmuch less Jim walks compared to Martha, I need to calculate the length of the diagonal (Jim's

path) and the total distance Martha walks (length + width),then subtract the diagonal from the sum

Wait a second, letme double-check that.So,if Jim walks 5oo feet diagonally, and Martha walks700

to find the difference.[294 tokens][Original model use more tokens to describe the problem

feet, then the difference is 700 - 500, which is 200feet.So, Jim walks 200 feet less than Martha.[678

background.]

tokens][The answer was obtained for the first time with shorter tokens.]

Subtracting500from70gives200oyesJimwalks200feetless.[743tokens][riginlmodet

But let me just make sure I didn't make a mistake in my calculations.

the answer for the first time.]

So,yesthat‘scorrecto,thedifferenceis700500,whichis00feet.Therefore,Jmwalks200ft

Hmm,I don't think so.It's straightforward with the Pythagorean theorem and simple addition.

less than Martha. [First Reflection.]

Alternatively,couldhavethoughtabout thedifferenceintheirpathMarthaspathisthesumf

length and width, while Jim's path is the hypotenuse.

Wait, just to add another layer of verification, what if I used the Pythagorean theorem differently?

The difference is the amount by which Martha’s path exceeds Jim's. So, 700 - 500 = 200.

Let me see.

Yes, I think that's solid.I don't see any errors in my reasoning.[First Reflection.]

Yep,that's correct. [Second Reflection.]

*Final Answer**

Jim walks (boxed{200} feet less than Martha. [1011 tokens][The thinking stage is over.]

Therefore, Jim walks 200 feet less than Martha. **Final Answer** Jim walks \boxed{200} feet less

</think>

than Martha.[1110 token][The thinking stage is over.]

Jim and Martha are standing at the same corner of a rectangular field.

</think>

Jim and Martha are standing together at the corner of a rectangular field.

Thus, Jim walks \boxed{200} feet less than Martha. [1241 token]

4 Analysis 172

4.1 Locate Self-Affirmation Reflections 173

To investigate the self-affirmation reflection, we first randomly sampled 500 training instances from 174 the MATH dataset. For each problem, we generated a single solution using R1-Distill-1.5B [12]. The 175 generated reasoning steps were split using the " \ n \ n " delimiter. 176

Then, we utilized the Qwen2.5-72B-Instruct [43] to classify each step as either a reasoning reflection 177 or non-reflective reasoning. To improve the accuracy, we further combine the previous method [6] to 178 assist in judgment. For steps identified as reasoning reflections, we invoked the Qwen2.5-72B-Instruct 179 again to determine whether the reflection affirmed the preceding reasoning, i.e., whether it constituted 180 a self-affirmation reflection. The specific prompts used for classification are detailed in Appendix A.3. 181 To validate this automated classification, we manually annotated self-affirmation reflections in all 182 responses of 20 problems to form a test set. Evaluation revealed the Qwen2.5-72B-Instruct achieved 183 an accuracy of 80.6%. This is primarily because while most self-affirmation reflections contain 184 explicit signal words such as "that's correct" in Figure 3, some self-affirmation reflections merely 185 repeat previous conclusions to indicate affirmation of prior content, making these more challenging 186 to detect. 187

QuestionfromMATH500

Howmayltwordwithtastnnsnantcanntructedfrmthersnd?tethatandarensnatnywordialitjstngliag

words, and letters may be used more than once.)

Responsefromreasoningmodel

The first“wait”:the model is not sure whetherit is necessary to conduct a check.

Okay, so Ihave this problem:Ineed to find how many 4-letter words can be made from the letters A,

[Wait, 0.38]. let me make sure I understand the problem correctly. The letters we can use.

B,C,D,andE.Thecatchis thateachwordmusthave atleastne consonant.Hmm,okay.etme

think about how to approach this.

[Let, 0.21] me break it down. The letters we have are A, B, C, D, E. Out of these, A, E are

[Alright, 0.21] , let's break it down. The letters we have are A, B, C, D, E. Out of these, A, E are

"

[That, 0.04] makes sense because it's easier to calculate the total number of words and then

So,maybeIcan calculate the total number of 4-letterwords possible and then subtract the number

[But, 0.03] wait, let me make sure I understand the problem correctly.The letters given are

of 4-letter words that consist entirely of vowels.

[Given, 0.019] that, let me figure out how many vowels and consonants

So,that's five letters in total.Among these,B,C,and D are consonants,and Aand Eare vowels.So,

Wait,let memake sureIunderstand the problem correctly.Theletters we can use are A,B,C,D,E.

[The, 0.014]letters given are A,B. C. D. E. Let me identify which are vowels

[Right, 0.016] . so let's break it down.The letters we have are A,B, C, D,E

vowels are A and E, and consonants are B, C, D.

The second "wait":the model determines that he must conduct the reflection.

[Wait, 0.83] . hold on. Let me double-check that. Is there another way to.

Therefore,the number of 4-letter words with at least one consonant isthe total number of words

[Hold, 0.05] on, let me double-check that. So, total words: 5^4 = 625. Al

minus the number of all-vowel words. That would be 625 - 16 = 609.

[But, 0.04] wait, let me double-check that. So, 5^4 is 625. 2^4 .

Wait,hold on.Letme double-checkthat.Isthere another waytoapproach this problem?Maybeby

[Hmm, 0.03] , wait a second. Let me double-check that. So, total words-

directly calculating the number of words with at least one consonant? Hmm, that might

[Let, 0.017] me just double-check my reasoning. Total words

[Is, 0.007] that the final answer?Wait, let me double-check my reasoning.

Alternatively,Ican thinkof itasthenumberof wordswithatleastoneconsonantisequalto the total

number of words minus the number of words with no consonants, which is exactly what I did first.So,

[So, 0.002] is 609 the answer? Let me double-check my reasoning.

625 - 16 = 609.

[Just, 0.001] to double-check, let me think if there's another way to.

Wait,anotherthought:sometimeswhenIdo these problemsImightforgetthat letters canbe

The third "wait": the model is not sure whether it is necessary to conduct a check.

repeated,but in thiscase,the problem says lettersmaybe used more than once,sorepetitionis

[Wait,0.20].another thought: sometimes when| do these problems,I might forget that

allowed. So, that's why it's 5^4 and 2^4, which is correct.

[1, 0.13] think that's solid. So. yeah, 609 is the answer.

So, yeah, Ithink 1'm confident that 609 is the correct answer.

[Another, 0.12] way to think about it is using the concept

[Alternatively, 0.08] . if I think in terms of probability, the probability that a word ..

**Final Answer**

os  uss e s ioios s a I[oo ]

The number of 4-letter words with at least one consonant is \boxed{609).

[ls 0.06] there a chance | made a mistake in calculating 5^4? Let me compute 5^4 again:

</think>

5*5=25,25+5=125,125+5=625.Yes,that's correct.

To determine howmany 4-letter..

Thus, the number of 4-letter words with at least one consonant is \\boxed(609}).

[Therefore, 0.059] , 1 think 609 is the correct answer.

[Yeah, 0.03] , I think that's solid. So, I think the answer is 609.

Based on the results of classification, our analysis proceeded from two complementary perspectives to identify patterns in self-affirmation reflection occurrence. First , prior researches [31, 26] has established that language patterns are critical in determining whether models activate specific reflection mechanisms during decision-making. Therefore, we sampled 20 self-affirmation reflections and other reflection types, then performed multiple rollout iterations from their contextual positions. We hope to explore whether models can elicit distinct behavioral responses by producing diverse leading words. As shown in Figure 4, we found that self-affirmation reflections exhibit significantly greater diversity in behavior-guiding continuation patterns compared to other reflection types.

Specifically, during the initial reflection (Self-

Affirmation Reflection), the model attempts to verify its understanding of the problem context. Examining the top 8 predicted word continuations, we observe the model's indecision between further reflection and progressing toward a solution, as evidenced by suggestions such as "break it down". In the second reflection (Other Reflection), the trajectories of the model show a high degree of consistency. Here, the top 8 predicted words uniformly advocate for continued reflection, such as "double-check". The final reflection (Self-Affirmation Reflection) occurs after the model has

Count

Other Reflections

0.85

Self-AffirmationFleflections

4000

Other Reflectiops

0.80

0.75

3000

0.70

Average Probability

0.65

2000

0.60

0.55

1000

0.50

0.45

0

Wait

Alternatively

But

So

First

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

already provided the correct answer. Similar to the first reflection, we observe the model confronting uncertainty, contemplating whether to terminate the task or initiate another round of reflection.

Second , we conducted an additional analysis of the quantity and probability of the leading words across all reflections. It is worth noting that, for simplicity, we only selected the first word as leading words, but a typical leading word is sometimes not necessarily a single word. For example, when sentences start with "But", we found that the next word is "wait" in 38% of self-affirmation reflections. In this case, "But wait" is more suitable to be used as a unique leading word. During statistical processing, we excluded repetitive tail-end reflections from model outputs. Because self-affirmation reflections sometimes exhibited recurring patterns where incremental reinforcement of leading-word probabilities led to closed-loop generation. These loops could skew statistical accuracy. While this phenomenon warrants further study, it falls outside the scope of this work. For an illustrative example of such looping behavior and comprehensive results including looped sentences, refer to Appendix A.4.

The final result is shown in Figure 5. We present the top 5 high-frequency words in self-affirmation reflection. To our surprise, the confidence level of leading words in self-affirmation reflection appears to be lower than that in other Reflections. This finding echoes the results from the Figure 4, indicating that when the model is uncertain about its actions, the output leading words also lack confidence.

4.2 Suppress Self-Affirmation Reflections

Up to now, our analysis highlights a critical insight: leading tokens in self-affirmation reflections exhibit significantly lower generation probabilities compared to other reflection types. Our objective is to suppress self-affirmation reflections in order to achieve shorter outputs while maintaining performance. Consequently, we propose a straightforward intervention: when the model assigns a low probability to generating leading words, we set their probability to zero to suppress self-affirmation reflections. Given the statistical significance of the "Wait" token, we first focus on it in this paper.

"Wait" in Other Reflections (Histogram)

"Wait" in Self-Affirmation Reflections (Histogram)

2.5

"Wait" in Other Reflections (Smoothed Distribution)

"Wait" in Self-Affirmation Reflections (Smoothed Distribution)

2.0

1.5

1.0

0.5

0.0

0.0

0.2

0.4

0.6

0.8

1.0

Probability

To validate our method, we analyze the probability density of the "Wait" token across two reflection types in Figure 6. Though their distributions exhibit partial overlap, they remain distinguishable through thresholding. We acknowledge that suppressing low-probability "Wait" instances may inadvertently filter out other reflections. However, empirical results show that even when such instances are suppressed, other high-probability tokens can still facilitate emergence of necessary reflections via compensatory mechanisms, as exemplified by the second reflection in Figure 4. We also recognize the limitation of exclusively targeting the "Wait" token, which relates to balancing interventions across multiple leading words. In practical results, we found that since "wait" frequently follows "But", suppressing this token also partially mitigates self-affirmation reflections. Consequently, we intervene on both "Wait" and "wait" (collectively referred to as the "wait" tokens hereafter). More discussions of other interfered tokens appear in Appendix A.5.

Our results also demonstrate that suppressing low-probability "wait" tokens maintains performance integrity and produces shorter outputs. In train-free settings (e.g., R1-Distill-Qwen-1.5B in Table 1), output lengths shorten while performance improves when thresholds decrease below 0.5. For training-based scenarios, ablation studies identify optimal performance at a threshold of 0.3. These findings curiously align with the probability distributions in Figure 6, suggesting that thresholds ≤ 0 . 3 effectively suppress self-affirmation reflections while having minimal impact on other reflection types.

5 Experiments

We first assess the impact of removing Self-Affirmation Reflections in Section 5.1. Subsequently, in Section 5.2, we integrate a representative train-based approach to verify whether explicitly suppressing self-affirmation reflections can lead to improved results.

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

5.1 Train-free Experiments

5.1.1 Settings

We hypothesize that removing low-probability self-affirmation reflection will not degrade model performance. To test this hypothesis, we conducted experiments across several prominent reasoning models: R1-Distill-Qwen-1.5B/7B/32B [12], QwQ-32B [37] and Qwen3-32B [38]. We also additionally compared Underthink [40], which intervened with logits within a certain window length. For more details and our discussion on this work, please refer to Appendix A.2. Similar to the previous approaches [6, 3, 24, 32], our experiments are conducted on four datasets: (1) MATH500 [22], comprising 500 problems across seven mathematical domains (algebra, geometry, number theory, etc.) that challenge both humans and LLMs; (2) AIME24, featuring 2024 American Invitational Mathematics Examination problems to test advanced mathematical competition problem-solving skills; (3) AMC23 [15], with problems from the 2023 American Mathematics Competitions covering secondary school mathematics; and (4) GSM8K [10], containing 8.5K linguistically diverse elementary school math problems to evaluate simple arithmetic reasoning. Regarding the results of the out-of-domain dataset, please refer to the Appendix A.6.

For AIME24 and AMC23, we sampled eight responses per question. For MATH500 and GSM8K, we collected one response per problem. The maximum allowed output length was 32k tokens. In order to evaluate Qwen3-32B [38], we utilized the latest vLLM implementation [19] in zero-shot inference settings. We report the average accuracy (Acc) and average token count (LEN) per response across all datasets. All experiments were conducted on NVIDIA L20 GPUs.

5.1.2 Results

As shown in Table 1, we evaluated the impact of different thresholds on performance. Our results indicate that an appropriate threshold can effectively reduce output length without compromising performance, and in some cases, even enhances performance with shorter outputs, which is consistent with our previous analysis in Figure 6. However, as the threshold increases, high-probability "wait" tokens are also filtered out, leading to a noticeable decline in performance due to the removal of some necessary reflections. Despite its simple implementation, our method achieves a significant reduction in output length with almost no performance loss. Specifically, it reduces average length by 18.7%, 14.3%, 11.1%, 9.1% and 8.4% for R1-Distill-Qwen-1.5B (Threshold-0.9), R1-Distill-Qwen-7B (Threshold-0.7), R1-Distill-Qwen-32B (Threshold-0.7), QwQ-32B (Threshold 0.7) and Qwen3-32B (Threshold 0.9), respectively. Moreover, we were surprised to find that the compression ratio adapts dynamically across different datasets. For instance, AIME24 demonstrates a lower compression ratio than MATH500. This aligns with the intuition that AIME24, being more challenging, inherently possesses less compressible structure compared to MATH500.

5.2 Train-based Experiments

5.2.1 Settings

In this section, we incorporate our approach into a representative training-based method. We utilize EfficientReasoning [3], which assigns positive rewards to correct responses and zero rewards to incorrect ones, with longer correct responses receiving smaller rewards. A penalty intensity coefficient α is included to fine-tune the compressive intensity. Owing to resource constraints inherent in training reinforcement learning (RL) models, we restrict our evaluation to the DeepseekR1-Distill-Qwen-1.5B checkpoint [12]. All other experimental configurations are inherited from the EfficientReasoning [3], with rollout budgets specifically set at 10 for AIME24, 3 for MATH500, and 1 for GSM8K. Our intervention is confined to the rollout phase during training. For testing, we rigorously adhere to the original implementation details: environments are constructed using the provided Dockerfile, and inference is conducted with the officially released checkpoints when evaluating EfficientReasoning [3]. 1 While the performance on specific datasets showed minor

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

discrepancies, the average performance across the three datasets aligned closely with the results reported in the EfficientReasoning [3].

During the rollout phase, we filter out all "wait" tokens that have confidence scores below 0.3. Reinforcement learning (RL) requires generating samples of varying lengths to strengthen shorter responses. However, indiscriminate intervention across all samples may compromise the learning of negative samples. Negative samples do not need to be shortened in length by suppressing selfaffirmation reflections. Therefore, our objective is to suppress self-affirmation reflections in positive samples only. Yet, determining in advance which responses are correct and should be intervened is challenging. Therefore, we adopt an approximate method and intervene with a 25% probability each time.

5.2.2 Results

The main results as shown in Table 2. As anticipated, our method achieved substantially shorter average lengths while maintaining comparable performance. It should be noted that text length and accuracy are inherently conflicting objectives. The results from EfficientReasoning [3] demonstrate that the stronger the emphasis on shorter text, the more severe the drop in accuracy. The model pursues shorter positive responses at the same intensity, which is approximately equivalent to using a stronger compression intensity. This explains the slight dip in accuracy. However, as shown in Table 2, the accuracy decrease is minimal and still surpasses the baseline (R1-Distill-Qwen-1.5B). Moreover, at the same text length, our method delivers superior performance.

5.2.3 Ablation Study

In this section, we compared the different thresholds for filtering "wait" tokens and the impact of the probability of intervention responses. All ablation experiments were conducted based on α = 0 . 05 .

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

Impact of different threshold : In this experiment, the probability of intervention responses is fixed on 50%. As shown in Table 3, when the threshold is below 0.9, the model's performance initially increases but then decreases. This indicates that excessive filtering of self-affirmation reflections can significantly degrade performance. However, when the threshold is set to 0.9, performance improves due to increased length. Overall, there is no linear relationship between the threshold and length. We attribute this to the uncontrollable attribute of the RL training process.

Impact of different probability : In this experiment, the thresholds for filtering "wait" tokens fixed on 0.3. As shown in Table 4, as the probability increases, performance exhibits a gradual decline. Overly aggressive intervention in the rollout process may generate shorter negative samples, thereby undermining the model's adversarial learning capabilities.

6 Conclusion

Our paper delves into the distinctive features of self-affirmation reflections in the distribution of leading words and their correlation with confidence. Leveraging these insights, we introduce a strategy to suppress critical leading words. Experimental results demonstrate that this method effectively reduces output length in both training-free and training-based scenarios while sustaining or even elevating model performance. While we acknowledge certain limitations (see Appendix A.1), we aim for this study to offer a fresh viewpoint on achieving more precise length compression at step-level efficient reasoning and to improve the efficiency of large reasoning models.

352

353

354

355

356

357

358

359

360

361

362

363

364

365

366

367

368

369

370

371

372

373

374

375

376

377

378

379

380

381

382

383

384

385

386

387

388

389

390

391

392

393

394

References

Pranjal Aggarwal and Sean Welleck. L1: Controlling how long a reasoning model thinks with reinforcement learning. arXiv preprint arXiv:2503.04697 , 2025.

AI@Meta. Llama 3 model card. 2024.

Daman Arora and Andrea Zanette. Training language models to reason efficiently, 2025.

Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732 , 2021.

Simon A Aytes, Jinheon Baek, and Sung Ju Hwang. Sketch-of-thought: Efficient llm reasoning with adaptive cognitive-inspired sketching. arXiv preprint arXiv:2503.05179 , 2025.

Runjin Chen, Zhenyu Zhang, Junyuan Hong, Souvik Kundu, and Zhangyang Wang. Seal: Steerable reasoning calibration of large language models for free. arXiv preprint arXiv:2504.07986 , 2025.

Jeffrey Cheng and Benjamin Van Durme. Compressed chain of thought: Efficient reasoning through dense representations. arXiv preprint arXiv:2412.13171 , 2024.

Yu-Neng Chuang, Leisheng Yu, Guanchu Wang, Lizhe Zhang, Zirui Liu, Xuanting Cai, Yang Sui, Vladimir Braverman, and Xia Hu. Confident or seek stronger: Exploring uncertainty-based on-device llm routing from benchmarking to generalization. arXiv preprint arXiv:2502.04428 , 2025.

Yu-Neng Chuang, Helen Zhou, Prathusha Sarma, Parikshit Gopalan, John Boccio, Sara Bolouki, and Xia Hu. Learning to route llms with confidence tokens. arXiv preprint arXiv:2410.13284 , 3, 2025.

Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 , 2021.

Jared Fernandez, Clara Na, Vashisth Tiwari, Yonatan Bisk, Sasha Luccioni, and Emma Strubell. Energy considerations of large language model inference and efficiency optimizations. arXiv preprint arXiv:2504.17674 , 2025.

Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948 , 2025.

Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. Token-budget-aware llm reasoning. arXiv preprint arXiv:2412.18547 , 2024.

Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in a continuous latent space. arXiv preprint arXiv:2412.06769 , 2024.

Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874 , 2021.

Andreas Hochlehnert, Hardik Bhatnagar, Vishaal Udandarao, Samuel Albanie, Ameya Prabhu, and Matthias Bethge. A sober look at progress in language model reasoning: Pitfalls and paths to reproducibility. arXiv preprint arXiv:2504.07086 , 2025.

Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, 395 AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv 396 preprint arXiv:2410.21276 , 2024. 397

398

399

400

401

402

403

404

405

406

407

408

409

410

411

412

413

414

415

416

417

418

419

420

421

422

423

424

425

426

427

428

429

430

431

432

433

434

435

436

437

438

439

440

441

442

443

444

445

446

Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai o1 system card. arXiv preprint arXiv:2412.16720 , 2024.

Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Principles , pages 611-626, 2023.

Ayeong Lee, Ethan Che, and Tianyi Peng. How well do llms compress their own chain-ofthought? a token complexity approach. arXiv preprint arXiv:2503.01141 , 2025.

Baohao Liao, Yuhui Xu, Hanze Dong, Junnan Li, Christof Monz, Silvio Savarese, Doyen Sahoo, and Caiming Xiong. Reward-guided speculative decoding for efficient llm reasoning. arXiv preprint arXiv:2501.19324 , 2025.

Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. arXiv preprint arXiv:2305.20050 , 2023.

Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 , 2024.

Haotian Luo, Li Shen, Haiying He, Yibo Wang, Shiwei Liu, Wei Li, Naiqiang Tan, Xiaochun Cao, and Dacheng Tao. O1-pruner: Length-harmonizing fine-tuning for o1-like reasoning pruning. arXiv preprint arXiv:2501.12570 , 2025.

Xinyin Ma, Guangnian Wan, Runpeng Yu, Gongfan Fang, and Xinchao Wang. Cot-valve: Length-compressible chain-of-thought tuning. arXiv preprint arXiv:2502.09601 , 2025.

Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling. arXiv preprint arXiv:2501.19393 , 2025.

Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E. Gonzalez, M Waleed Kadous, and Ion Stoica. Routellm: Learning to route llms with preference data, 2024.

Jacob Pfau, William Merrill, and Samuel R Bowman. Let's think dot by dot: Hidden computation in transformer language models. arXiv preprint arXiv:2404.15758 , 2024.

Xiaoye Qu, Yafu Li, Zhaochen Su, Weigao Sun, Jianhao Yan, Dongrui Liu, Ganqu Cui, Daizong Liu, Shuxian Liang, Junxian He, et al. A survey of efficient reasoning for large reasoning models: Language, multimodality, and beyond. arXiv preprint arXiv:2503.21614 , 2025.

David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling , 2024.

Darsh J Shah, Peter Rushton, Somanshu Singla, Mohit Parmar, Kurt Smith, Yash Vanjani, Ashish Vaswani, Adarsh Chaluvaraju, Andrew Hojel, Andrew Ma, et al. Rethinking reflection in pre-training. arXiv preprint arXiv:2504.04022 , 2025.

Yi Shen, Jian Zhang, Jieyun Huang, Shuming Shi, Wenjing Zhang, Jiangze Yan, Ning Wang, Kai Wang, and Shiguo Lian. Dast: Difficulty-adaptive slow-thinking for large reasoning models. arXiv preprint arXiv:2503.04472 , 2025.

Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan He. Codi: Compressing chain-of-thought into continuous space via self-distillation. arXiv preprint arXiv:2502.21074 , 2025.

Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Hanjie Chen, Xia Hu, et al. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503.16419 , 2025.

447

448

449

450

451

452

453

454

455

456

457

458

459

460

461

462

463

464

465

466

467

468

Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, , and Jason Wei. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261 , 2022.

Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1.5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599 , 2025.

Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025.

Qwen3 Team. Qwen3, 2025.

Rui Wang, Hongru Wang, Boyang Xue, Jianhui Pang, Shudong Liu, Yi Chen, Jiahao Qiu, Derek Fai Wong, Heng Ji, and Kam-Fai Wong. Harnessing the reasoning economy: A survey of efficient reasoning for large language models. arXiv preprint arXiv:2503.24377 , 2025.

Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, et al. Thoughts are all over the place: On the underthinking of o1-like llms. arXiv preprint arXiv:2501.18585 , 2025.

Heming Xia, Yongqi Li, Chak Tou Leong, Wenjie Wang, and Wenjie Li. Tokenskip: Controllable chain-of-thought compression in llms. arXiv preprint arXiv:2502.12067 , 2025.

Silei Xu, Wenhao Xie, Lingxiao Zhao, and Pengcheng He. Chain of draft: Thinking faster by writing less. arXiv preprint arXiv:2502.18600 , 2025.

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115 , 2024.

Edward Yeo, Yuxuan Tong, Morry Niu, Graham Neubig, and Xiang Yue. Demystifying long 469 chain-of-thought reasoning in llms. arXiv preprint arXiv:2502.03373 , 2025. 470

471

472

A Technical Appendices and Supplementary Material

A.1 Limitations

Two core challenges persist. First, methods for systematically identifying candidate tokens for 473 intervention remain underdeveloped. Second, developing principles to balance dependencies among 474 multiple intervened words represents an open research question. Consequently, our intervention 475 framework in this work focuses solely on the most salient specific tokens. Future research will explore 476 strategies to generalize token intervention approaches, addressing both identification scalability and 477 multi-word coordination. 478

We further recognize that the foundational mechanisms governing the model's reasoning process, 479 as well as the emergence of redundant reflections during generation, are not yet fully understood. 480 These knowledge gaps persist within the large reasoning model community. As the field matures, we 481 anticipate opportunities to integrate insights from complementary work and conduct more in-depth 482 analyses of these phenomena in subsequent studies. 483

A.2 Comparison with concurrent works 484

We analyze two concurrent excellent works:

485

486

487

488

489

490

491

SEAL[6] : SEALfocuses on modifying activation values to convert reflective steps (termed "reflection thoughts" and "transition thoughts" in their paper) into executive steps (termed "execution thoughts"). For more details, please refer to the SEAL [6]. In contrast, our work identifies which reflective steps can be preserved and which should be removed. We extend SEAL's t-SNE visualization by highlighting reflective steps with low confidence (see Figure A.2), successfully distinguishing ambiguous reflective steps overlapping with executive steps that typically indicate model confusion.

It is important to note that SEAL only attempts a maximum output length of 10,000 tokens. This 492 limitation arises because SEAL requires intervention on activation values during the "think" phase 493 (between "<think>" and "</think>"), which presents significant implementation challenges, such as 494 integration with vLLM. In contrast, our method achieves a maximum output length of 32,000 tokens, 495 consistent with DeepSeek's testing configuration. This advancement is attributed to our approach's 496 ability to directly integrate with vLLM. 497

498

499

500

501

502

503

504

505

506

507

508

509

510

511

512

513

514

515

516

517

518

519

520

521

522

523

524

Underthink [40] : This paper aims to address the lack of path exploration capability in models by adjusting the logits of certain reflective leading words within a fixed window. This approach encourages models to explore current reasoning paths more thoroughly during the exploration process. Our focus differs. We aim to suppress self-affirmation reflections in the whole output to achieve more concise reasoning. Whether it is during the exploration process (self-affirmation reflection after already correct steps) or after the completed exploration (self-affirmation reflection after outputting correct answers). Notably, while this work provides a preliminary analysis of underthinking, our work offers a more in-depth and specific analysis of self-affirmation reflections, presenting a fresh perspective on efficient reasoning area.

We also compared our method with Underthink [40], and the results are presented in Table 5 and Table 1. To ensure a fair comparison, we only intervened on the same tokens (all "wait" tokens) as described in the main text. In this context, α and β denote the intensity and window length of the intervention, respectively. Underthink [40] recommends parameters of α = 3 and β = 600 , which are also the default parameters in Table 1. As shown in Table 5, the phenomenon identified in this paper also contributes to the improvement of Underthink [40]. For instance, when the intervention window length β in Underthink [40] is set to + ∞ , the performance remains nearly unchanged and sometimes even improves, while the outputs become shorter. However, compared with Underthink [40], we consider that intervening based on probability is more straightforward and better aligned with the analysis in our paper. Therefore, we ultimately chose not to directly intervene in the logits.

A.3 Template

There are the templates we use. During the second judgment phase, the current reflection and the sequence of steps spanning from current reflection step to the subsequent reflection step are processed simultaneously.

The first judgment template

"""

Current step: {str1}

Please help me determine the function of the current step.

Is the current step a reflective behavior? 525

Output the answer directly to <answer></answer>, for example, <answer>Yes</answer> or <an526 swer>No</answer>. 527

528

"""

540

541

542

543

544

545

546

547

548

A.4 Influence of the tail repeated reflections

Figure 8 illustrates a case where the model becomes trapped in a loop. The correct answer is 49, yet the model repeatedly performs self-affirmation reflection and enhances the probability of the leading word. As demonstrated in Figures 9 and 10, the repeated steps filtered to the end of the output significantly impact the statistical results. This presents an intriguing phenomenon worth in-depth investigation. Moreover, the intervention proposed in this paper can partially address this issue. Notably, despite this issue, Self-affirmation reflections still retains significantly confidence bias in leading tokens compared to other reflections. This proves the generalization of our discovery.

Other Reflections

0.85

14000

12000

Self-AffirmationReflections

Other Reflections

Self-AffirmatiorReflections

0.80

12

14

"Wait" in Self-Affirmation Reflections (Smoothed Distribution)

"Wait" in Other Reflections (Histogram)

"Wait" in Self-Affirmation Reflections (Histogram)

"Wait" in Other Reflections (Smoothed Distribution)

0.75

10000

Average Probability

10

0.70

Count

8000

Density

8

0.65

6000

6

4000

0.60

2000

0.55

4

0.50

2

0

Wait

Alternatively

But

So

First

Tokens

0

0.0

0.2

0.4

0.6

0.8

1.0

Probability

A.5 Discussion on interfered tokens

In this section, we evaluate the impact of intervening on different tokens for R1-Distill-Qwen549 1.5B [12]. Results in Table 6 demonstrate consistent patterns across settings. When threshold 550 values are small, all results align with our hypothesis in Section 5.1: removing the Self-Affirmation 551 Reflection does not significantly degrade performance but generates shorter outputs. This behavior is 552 observed in interventions targeting "wait" tokens (Threshold=0.9), "wait" + "alternatively" tokens 553 (Threshold=0.5), and "wait" + "alternatively" + "but" tokens (Threshold=0.5). Increasing threshold 554 values shorten outputs at the cost of reduced performance. 555

For fixed thresholds, output length decreases progressively as the number of intervened tokens 556 increases. However, addressing dependencies between multiple tokens remains challenging due 557 to combinatorial complexity. Notably, "wait" tokens exhibits statistically significant effects in our 558 analysis. Given these factors, we defer exploration of multi-token interactions to future work and 559 focus primarily on "wait" tokens in this paper. 560

561

562

563

564

565

566

567

568

569

570

571

A.6 Results of out-of-domain dataset

We additionally evaluate our model on out-of-domain test data using the GPQA-Diamond benchmark [30]. GPQA-Diamond serves as a rigorous evaluation dataset designed to assess models' capacity for deep reasoning and domain expertise. This dataset represents the highest-quality resource within the GPQA series, comprising 198 graduate-level or competition-level multiple-choice questions. The questions primarily focus on core STEM disciplines including biology, physics, and chemistry, presenting complex problems that require sophisticated reasoning abilities. We sampled four responses for each question and restricted the output length to 32k tokens. We report the average accuracy (Acc) and average token count (LEN) per response. As shown in Table 7, our method generates significantly shorter outputs on out-of-domain dataset while maintaining competitive performance.

A.7 Implementation in vLLM [19]

We provide the simplest implementation method in vLLM [19]. However, for a perfect running speed, 648 we suggest achieving it by directly creating a new CustomSampler class. For example, 649

655

656

657

658

659

660

661

662

663

664

665

666

667

668

669

670

671

672

673

674

675

676

677

678

679

680

681

682

683

684

685

686

687

688

689

690

691

692

693

694

695

696

697

698

699

700

701

702

703

704

705

706

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification: Claims made in the abstract and introduction accurately reflect the paper's contributions and scope.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We have discussed the limitations of our work in Section A.1.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [NA]

Justification: Our paper does not include theoretical results.

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: We fully disclose all the information needed to reproduce the main experimental results of the paper in Section 5.1.1 and Section 5.2.1.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We provide the necessary code.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We describe the implementation in detail to make it reproductiive.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We report other appropriate information about the statistical significance of the experiments.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We provided our employed open-source models and the hardware environments.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We conduct research in the paper conform, in every respect, with the NeurIPS Code of Ethics.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [NA]

Justification: This paper has no social impact of either positive or negative.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [NA]

Justification: The data and models are open-source that have a no risk for misuse.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We are the creators or original owners of assets used in the paper, properly credited, and are the license and terms of use explicitly mentioned and properly respected.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We release no new assets in this paper.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: We didn't employ any human crowd-sourcing projects in this work.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: The contents of our paper have no risk of leaking out or disclosing.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

968

969

970

971

972

973

974

975

976

977

978

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [Yes]

Justification: We describe the usage of LLMs in Section A.3.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Peer reviews:
comments: SUMMARY:
This paper presents a training-free method to shorten CoT through suppressing self-affirmation reflections. The intuition is that self-affirmation reflection is often redundant and does not provide further information. Through analysis, the authors found that the probability of the leading word for those self-affirmation reflections is lower than that of other reflections, thus the paper proposes to zero-out the probability of those leading reflection words then their probability is too low, to suppress self-affirmation behaviors.

STRENGTHS AND WEAKNESSES:
### Strengths

1. Redundant CoT is a critical issue for thinking models and needs more efforts to deal with. It is nice that the proposed approach is training free.  
2. Experiments are conducted across multiple reasoning models.


### Weaknesses

1. Generally I think self-affirmation is useful and should not be suppressed just by heuristics, as it may help future reasoning steps judge which steps are more likely to be correct. I think this is reflected by a mild performance decrease of the proposed approach in Table 1 – actually, my biggest concern is that the proposed approach is not very effective and only reduces the CoT length by around 10%-18% most of the time. This is only a slight reduction of the highly redundant thinking process and this small reduction comes with a performance decrease.  
2. Line 201-204, I don’t think the authors should draw any conclusion just from a qualitative case study.

QUESTIONS:
NA

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 1
ORIGINALITY: 3
DATE: 2025-07-02

comments: SUMMARY:
This paper presents a simple yet effective train-free intervention method that suppresses low-probability leading words (e.g. “wait”) to reduce self-affirmation reflections in reasoning outputs, achieving shorter outputs without degrading performance. Experiments across multiple models and datasets demonstrate consistent improvements.

STRENGTHS AND WEAKNESSES:
Strengths:

1. The paper proposes a simple train-free suppression method that does not require model modifications, providing an interesting and practical extension for efficient reasoning.

2. Extensive experiments on multiple datasets and models show consistent effectiveness, supporting the generality of the approach.

Weaknesses:

1. Table 3 and Table 4 lack clear change indicators.

2. It is unclear whether leading words differ across datasets or LLMs, and whether the confidence threshold separating self-affirmation from other reflections remains consistent in different contexts. This raises questions about the method's generalizability.

QUESTIONS:
1. Since the method involves directly deleting outputs during inference, what is the rate of misclassification that results in removal of necessary reflections? Does this affect the stability and correctness of complex reasoning tasks?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 3
IMPACT: 4
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper addresses a specific cause of over-thinking in LRMs: Self-Affirmation Reflections (SARs), i.e., low-information sentences that simply restate or praise already-correct reasoning steps. By analyzing token-level probabilities, the authors discover that SARs start with a small set of “leading words” (notably wait/Wait) whose logits are systematically low. They exploit this bias with a train-free decoding hack: whenever the next-token probability of a SAR leading word falls below a threshold, its probability is zeroed, preventing the reflection from being generated. A second, train-based variant injects the same rule during RL rollouts. Experiments on four math-reasoning benchmarks and five open models show sizeable length savings (18.7% training-free, 50.2% training-based for R1-Distill-1.5B) with negligible or even positive accuracy change, and the intervention plugs straight into vLLM.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n1. The paper brings attention to SARs, a prevalent and interesting behavior in LRM reasoning.\n\n2. Linking SAR occurrence to low-confidence leading words is original and empirically well supported by Figures 4-6.\n\n3. The decoding-time suppression requires no extra parameters, data, or gradient steps and is drop-in compatible with vLLM and RL training loops. \n\n4. Experiments are extensive.  Results span diverse model sizes and four datasets, plus ablations over thresholds and intervention rates. Length savings up to 18.7 % (training-free) and 50.2 % (training-based) are impressive given minimal accuracy change.\n\n## Weaknesses\n\n1. As SAR is a narrowly targeted phenomenon under overthinking, the proposed fix is conceptually a heuristic token blacklist, and marks limited novelty to the broader literature on efficient LRM reasoning through compressing reasoning chain.\n\n2. In addition to the previous weakness, it seems that Underthink is treated as the main (and only) baseline. However, there exist various approaches for efficient LRM reasoning  through reasoning chain compression, such as Constrained-CoT [1], Chain-of-Draft [2], and LightThinker [3], to name just a few. While not explicitly targeted at reducing SARs, they can implicitly compress those as part of the general reasoning chain. \n\n3. SAR labelling depends on a proprietary 72 B model and is only ~80% accurate (Section 4.1, Line 184), raising doubt about analysis fidelity. \n\n4. Suppressing low-probability wait tokens may also remove beneficial reflections or leave SARs starting with other tokens. The lack of human evaluation leaves answer readability and informativeness unverified.\n\n5. The paper optimises token count but omits runtime, latency, or GPU-cost metrics, which are arguably the true targets of “efficient reasoning”.\n\n\n\n[1] Concise Thoughts: Impact of Output Length on LLM Reasoning and Cost. arXiv:2407.19825\n\n[2] Chain of draft: Thinking faster by writing less. arXiv:2502.18600\n\n[3] LightThinker: Thinking Step-by-Step Compression. arXiv:2502.15589\n\nQUESTIONS:\nPlease refer to the weaknesses section and address all issues raised.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

comments: SUMMARY:
The paper addresses the challenges faced by large language models in complex reasoning tasks, particularly focusing on the issue of redundant reflection, termed Self-Affirmation Reflection. Despite advancements in reasoning models through methods like reinforcement learning and supervised fine-tuning, these models often generate unnecessarily lengthy reasoning steps, leading to inefficiencies.

STRENGTHS AND WEAKNESSES:
Strengths:
1. The paper provides the first in-depth analysis of the Self-Affirmation Reflection phenomenon, offering valuable insights for improving reasoning models.

Weaknesses:
1. The proposed method relies heavily on the identification and suppression of leading words, which may not be universally applicable across all models or tasks;
2. The simplicity of the intervention strategy, while advantageous, might overlook more complex underlying factors contributing to reflection behaviors in reasoning models.

QUESTIONS:
none

LIMITATIONS:
See above
TITLE: Official Review
CLARITY: 3
IMPACT: 2
ORIGINALITY: 3
DATE: 2025-07-12