Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoning
Authors: Chen Qian
Dongrui Liu
Haochen Wen
Zhen Bai
Yong Liu
Jing Shao
Venue: NeurIPS 2025 poster ()
Keywords: Large Language Model
Reasoning
Information Theory
Primary area: deep_learning

Abstract: Large reasoning models (LRMs) have demonstrated impressive capabilities in complex problem-solving, yet their internal reasoning mechanisms remain poorly understood.
In this paper, we investigate the reasoning trajectories of LRMs from an information-theoretic perspective. 
By tracking how mutual information (MI) between intermediate representations and the correct answer evolves during LRM reasoning, we observe an interesting MI peaks phenomenon: the MI at specific generative steps exhibits a sudden and significant increase during LRM's reasoning process. 
We theoretically analyze such phenomenon and show that as MI increases, the probability of model's prediction error decreases.
Furthermore, these MI peaks often correspond to tokens expressing reflection or transition, such as "Hmm", "Wait" and "Therefore," which we term as the thinking tokens.
We then demonstrate that these thinking tokens are crucial for LRM's reasoning performance, while other tokens has minimal impacts.
Building on these analyses, we propose two simple yet effective methods to improve LRM's reasoning performance, by delicately leveraging these thinking tokens.
Overall, our work provides novel insights into the reasoning mechanisms of LRMs and offers practical ways to improve their reasoning capabilities.
The code is available at \url{https://github.com/ChnQ/MI-Peaks}.

Demystifying Reasoning Dynamies with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoning
Chen Qian Dongrui Liu Haochen Wen', Zhen Bai Yong Liu', Jing Shao Gaoling School of Artificial Intelligence, Renmin University of China
Shanghai Artificial Intelligence Laboratory
3 University College London, University of London 4 Dalian University of Technology
{qianchen2022, htw.::gaiiiiuoooogaaaaaaaaaaaa enu.cn (liudongrui, shao @pjab, org.
Abstract
Large reasoning models (LRMs) have demonstrated impressive capabilities in complex problem-solving, yet their internal reasoning mechanisms remain poorly understood. In this paper, investigate the reasoning trajectories of LRMs from an informatioon-hhoortiie perspective. By tracking how mutual information (MI) between intermediate representations and the correct answer evolves during LRM reasoning, we observe an interesting MI peaks phenomenon: the MI at specific generative steps exhibits sudden and significant increase during LRM's reasoning process. We theoretically analyze such phenomenon and show that as MI increases, the probability of model's prediction error decreases. Furthermore, these MI peaks often correspond to tokens expressing reflection or transi- tion, such "Hmm", "Wait" and "Therefore," which term as the thinking tokens. We then demonstrate that these thinking tokens are crucial for LRM's reasoning performance, while other tokens has minimal impacts. Building on these analyses, we propose two simple yet effective methods to improve LRM's s reasoning performance, by delicately leveraging these thinking tokens. Overall, our work provides novel insights into the reasoning mechanisms of LRMs and offers practical ways to improve their reasoning capabilities. The code available https: /gi thub. com www.eoomm//aeeeeeee CChnQ/MI-Peaks
Introduction
The reasoning ability of large language models (LLMs) has emerged as one of their most powerful and crucial capabilities [53] 21 (22. By explicitly thinking through a question before providing an answer and breaking down complex problems into multiple steps, LLMs have made impressive progress in complex reasoning tasks, such as mathematics, programming, and logical inference 28
45 7
Understanding and improving LLMs' reasoning ability represents crucial pathway toward achieving
Artificial General Intelligence (AGI) [6 52
By undergoing reasoning-intensive training on foundational LLMs, recent large reasoning models (LRMs) such as OpenAl's (22] DeepSeek's R1 [19], and QwQ 46 have demonstrated exceptional reasoning capabilities, significantly pushing the boundaries of complex problem--olvingg However, despite recent advances, the mechanisms underlying these capabilities remain largely under-explored. The internal dynamics of the reasoning process. well as the influence of each intermediate step on the final answer, are still largely "black box." While some research in the field of trustworthy AI
Equal contribution. This work was done during internship at Shanghai Artificial Intelligence Laboratory. supervised by Dongrui Liu. Corresponding author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

a be pos live real number such that all the roats of x ax" 1 - O are real, Find the smallest possible value
High
Therefore
The
Since
MI Penka
But
Mutual Informmtion
siruugt
ifa
Okay, so I have this problem here: Lii be positive.... ... First, lot me write
That
Hmm
down the equation again make sure
hhvve right:... Wait, ls that right?
Regular Ml
Then,
First
Himm, maybe shoud double-
NC
Now, combine all these termsr... So
okayan Jon
which wauld
mean the cubic has a multiple root...
Reasoning Steps
To NOW Decousee
Wait t Maybe
(a)
(b)
Figure I: Illustration and analysis of the MI peaks phenomenon in LRM reasoning. (a) The left side
shows an example of LRM performing a multi-step reasoning task. To investigate the underlying reasoning mechanism, we compute the MI between the model's '' representation at each step and the golden answer. Interestingly, as shown on the right side, certain steps exhibit sudden and significant increases in MI, which we refer to the MI peaks phenomenon. (b) Token distribution at MI peaks. We further find that the tokens generated at these high-MI steps are often reflective or transitional expressions such as "So," "Hmm," and "Wait."
suggests the existence of "critical tokens" that directly impact the safety of the LLM's answers [55 29 36), a natural question arises: are there critical reasoning steps or intermediate states that significantly affect the final results in the reasoning process of LRMs?
In this paper, explore this question from an informatiio-thhorrete given a question, we dynamically calculate the mutual information (MI) between the LRM's repre- sentation at each step of reasoning process and the golden answer (i.e., the ground-truth response), observing how the MI evolves. Interestingly, we find that certain steps' representations exhibit a sudden and significant increase in MI with the golden answer. As shown in Figure [a), these representations with MI peaks are sparse and occur non-uniformly throughout the reasoning process. This suggests that certain crucial reasoning steps, LRMs' representation becomes highly informa- tive about the correct answer. Naturally, this raises a question: are these MI peaks potentially related to model's 's reasoning performance? Theoretically, we provide preliminary insights into the MI peaks phenomenon, demonstrating that as the cumulative MI between the representations and the golden answer increases, the probability of LRM's wrong prediction lowers. Furthermore, our experiments show that the base models corresponding to these LRMs (e.g.. LLaMA-3.1 1.BBBB (17), does not exhibit this MI Peaks phenomenon as clearly. These analyses suggest that the distinet MI peaks observed during LRM reasoning are potentially stemming from the reasoning- inttensve training, and may hold a potential relationship with LRM's advanced reasoning abilities.
4 perspective. Specifically,
This naturally leads to the question: what semantic roles do the representations al MI peaks play during reasoning? Intriguingly, we find that these representations with MI peaks predominantly correspond to tokens such as "Wait," "Hmm," " "Therefore," "So," which typically express reflectiveness, self-correcting, or transitions, as shown in Figure [b). Here, we refer these tokens with MI peaks as "thinking tokens". Since these thinking tokens explicitly prompt the model to reflect and reason, and their representations carry enriched information with the golden answer, we hypothesize that these thinking tokens may play critical role in the model's 's reasoning ability. To validate this hypothesis, we suppress the generation of these thinking tokens and observe how the model's s reasoning performance changes. As shown in Figure 5] fully suppressing the generation of these thinking tokens significantly harms the model's s reasoning performance, while randomly suppressing the same number of tokens has little impact. This indicates that these thinking tokens are indeed crucial to LRM's S reasoning ability.
Finally, drawing insights from the above analyses, we propose to improve the reasoning performance of LRMs in two training-free free ways. 1) By allowing the representations at MI Peaks undergo multiple iterations within the model, we propose a method called Representation Recycling (RR). RR encourages the model to better exploit these informative representations. Experiments show that RR consistently improves the LRMs' reasoning performance across several benchmarks. For instance, improves the accuracy of DoapSSeek-R--iisill--LLaa.a Motivated by our analysis of thinking tokens, we propose Thinking Token based Test-time Scaling (TTTS). That is, when additional token budget remains, we force the model to continue reasoning by begin with the thinking tokens. Experiments show that TTTS leads to stcady performance
by 20% relatively on AIME24. 2)
2

---

DeepSeak- R1 Diaal Qwen 78
DecpSock 81 Distll Liama 8B
DecxSook 81 Diall Qwen 14B
0.2
200
400
300
522
200
500
Proasoniing Step
Reaponing
Rauserning Step
DaspSeok-RR- wwt..eRRR..Dai-eennn-n332133
QwC-338
Doep
2.3
2.00
200
500
400
2.0
200
320
1.22
220
400
500
Phomcrtingg Step
Reasoning Shep
Deunsriing Step
Figure 2: The evolution trajectories of MI between each step's representations and the golden answer during the reasoning process in LRMs.
improvements the token budget increases compared to the original LRMs. These applications further demonstrate that our observations can offer new insights into enhancing the reasoning abilities of LRMs.
2 Emergence of MI Peaks in LRMs' Reasoning Trajectories
Despite the impressive reasoning capabilities demonstrated by recent LRMss such as DeepSeek's RI series models [19] and Qwen's QwQ 466, the underlying mechanisms driving these capabilities remain poorly understood. In this section, we investigate the reasoning trajectories of LRMs from an informition-theerrtiee perspective. We begin by introducing the notations and preliminaries (Section 2. In Section 2.2 we demonstrate the MI peaks phenomenon. We then provide theoretical insights into this phenomenon Section 22.3 Finally, we examine whether similar patterns emerge in the corresponding non-reasoning LLMs of LRMs in Section 2.4
2.1 Preliminaries
Extracting representations in LRM generation process. Given data sample = (xx, yy. where x is the input query and denotes the corresponding golden answer, which consists of both the intermediate chain-of-thought reasoning steps and the final solution. For a LLM M, when prompted with x, it auto-regressively generates ş = (5r, U2...., ber, where 7 is the total number of tokens and Úr denotes the token produced at step t. To analyze the dynamic generation process, we collect the hidden representation corresponding to each generated token. Let Af(-) denote the representation extraction function that extracts the representation of the i-th token at layer I of LLM when given an input. For simplicity, we omit the superscripts and subscripts on A. In this way, the representation corresponding to the 1-th generated token is denoted by h, AMM(,, D<()), where jee denotes the subsequence of ò before the i-th token. Similarly, we also extract the representation of the gold answer by feeding y into the LLM, e.g., h, = A(M()))
Estimating MI between each generated token and golden answer. After extracting the represen- tation, we then measure the MI between each generated token's representation h, and the golden answer's 's representation hy obtaining a MI sequence: I[h1; hy [[h;; hyl.... .... I[hr; hyy In this way, we observe how MI evolves, thus analyze the reasoning dynamics during LLM's generation
process. Specifically, we follow [32 38
13 to use the Hilbert-Sehhe Schmidt Independence Criterion
(HSIC) [8] to estimate MI 27 35 The formal definition of HSIC is stated Definition 4 and we
provide more implementation details in Appendix B
Definition I (Hilhert-Schhiid Independence Criterion (HSIC) (8). HSIC is the Hilbert-Schmin norm of the cross-covarinnce operator between the distributions in Reproducing Kernel Hilbert Space
(RKHS). Formally:
H)IC(X,Y) =ExVX'Y' [Ax (X, X') x''Kk ky Y.Y"]]= + Exx [kx (X, (X.X')] Ery [ky (Y,Y")]
-2 Ex kx (X, X')] By [ky (Y,Y"]],
(1)
where X', Y' are independent copies of X, Y. respectively, and kx ky are kernel functions.
3

---

Table 1: Statistical properties of MI peaks across different LRMs. Here, #MI Peaks and #All Steps refer the number of MI peaks and the total number reasoning steps, respectively. Interval of Peaks denotes the number of steps between two consecutive MI peaks.
Ratio of Max Interval
Min Interval of MI Peaks
Avg Interval of MI Peaks
Model
#MI Peaks #All Steps
MI Peaks
of MI Peaks
DeepSeek- RI D--ssttill Sael---eennen
2.57
507.97
0,0051
152.67
52.74
87.38
DeepSeek-R1- L--iastill----aaa-B
24.54
511.03
0.0480
69.37
6.65
27.84
DeepSeck-R11DD Seaeilll--w-----e 14B
18.30
510.09
0.0359
85.50
5.33
31.09
DaabscckRR---isil----eeeee QwQ-32B
10.82
511.22
0.02112
138.07
19.35
59.30
5.41
489.80
0.0110
167.85
19.35
66.53
DeepSeek
16.60
512.00
0.0324
93.03
6.77
34.71
2.2 Investigating LRM's Reasoning Trajectories with MI
In this subsection, we track how the MI between each step's representation and the gold answer evolves, following the procedure in Section 2.1 Specifically, we conduct experiments several pop- ular LRMs of varying scales, including the CrepSeck-RI-Dssilll series [19] and QwQ-32B [46]. We
use the training split the MATH dataset (20), which comprises 12k competition- level mathematics problems, each accompanied by detailed step-by-step solution.
Certain steps exhibit sudden and significantly increases in MI during the reasoning process of LRMs. Figure shows the MI evolution trajectories for one data sample during LRMs generation" Surprisingly, across all tested LRMs,
observe consistent pattern: while most steps exhibit relatively low and stable MI values as reasoning proceeds, certain steps' MI suddenly and significantly increases. We refer these steps with abrupt increase in MI the MI peaks. Formally, define MI peaks as follows:
Definition (MI Peak). Given a MI sequence {m-)]= let Q1 Qa denote the 25-th percentile (first quartile), and the 75-th percentile (third quartile) of the sequence, respectively, We then define IQR(m) = Q3 - Q1 as the inter-quartile range. In this way, we identify the set of MI peaks as
o = {t: me > Q3 + IIQ(mm)
where T scale factor. Empirically, we ser T 1.5 (48).
MI peaks are sparse and distribute non-uniformly throughout the total reasoning process. As shown in Table MI peaks occur quite sparsely in the reasoning processes of LRMs, accounting for no more than 5% of all reasoning steps. Notably, for DeepSeek-R--Disilll- 1-Distill-Qwen-775 the MI peak ratio only 0.51 Despite this sparsity, these MI peaks are scattered across the entire reasoning trajectory, as illustrated in Figure 2 Moreover, the interval statistics reported in Table [ indicaae that MI peaks not occur at uniform intervals. Such a sparse and non-uniform distribution pattern suggests that MI peaks may emerge opportunistically at key moments during reasoning.
2.3 Theoretical Insights: Higher Leads to Tighter Bounds on Prediction Error
our empirical exploration reveals the emergence MI peaks in LRMs' reasoning trajectories, indicates that certain representations encode substantially rich information about the gold answer. This raises natural question: would such pattern be potentially related to the LRM's reasoning performance? In this subsection, we provide theoretical insights into this question, showing that higher MI between representations and the gold answer yields tighter lower and upper bounds on the model's prediction error.
In Section 2.2
Theorem 1. Consider a sequence of representations h1 h2,,.., hy during an LLM's reasoning process, where T denotes the number of total reasoning steps. Let y. ÿ denote the golden answer and the LLM's 's prediction answer, respectively. Define Pe = Pr(ū # y) as the LLM's prediction error probability. Then the following inequality holds:
T Σ
H(v)
T(y: h2 hej) Ho(pa)
Pe
log (D>) -
Results for more examples and more LRMs reported in Appendix D

---

[8] Deepseok-FR Baa.kkR----Diie I--iitil--lama---B- and Liama-3.
24
141 Veluee
0.4
5.1
9.1
Recponing S0op
Recoontnng Stop
Raazonin Stop
DeepSeekk-R Distill-Qwwen- 14B and Q2wen2.5-144 5-14B
Voluee
11
3.50
220
250
502
390
220
400
502
Roasonigg
Roasering Stop
Rossoning Sicp
Figure 3: Comparison of MI trajectories between LRMs and their corresponding non-reasoning LLMs.
where [3] is the size of the support of and Hy(pe) denote the binary entropy of Pe. that defined by Holpe) -P log (1 Pe) log(1 -pp).
Remark 1. Theorem Tsstablishes lower bound on the LLM's prediction error Pe- Intuitively, suggests that for an LLM to achieve low enror rate, its sequence of internal representations during generation should capture more information about the golden answer. In other words, higher MI throughout the generation trajectory may help lower model's minimal achievable error. Note that, this result be viewed as modified application of Fano's 's inequality by decomposing the mutual information along the trajectory.
adapted to step-wise reasoning
Theorem 2. Following the notations in Theorem 7 the following inequality holds:
1 H(y)
Σ I(y: h2 hes)]
Pe & 2
Remark 2. Theorem provides upper bound on the prediction error Pe, which complements the lower bound in Theorem It demonstrates that a higher cumulative MI between the sequence of representations and the golden answer leads to a tighter upper bound on LLM's error probability.
between representations during reasoning and the golden answer leads tighter upper and lower bounds on the model's eиror probubility. In other words, the model is more likely to arrive at the correct answer. Notably, the presence of MI peaks can effectively increase this cumulative MI, thereby potentially helping LLMs to perform more accurate reasoning.
Remark 3. summary, Theorems D and 2 jointly suggest that, higher cumulative
2.4 Will Non-rcasoning L.MSs also Exhibit the MI Peaks Phenomenon?
Since the MI Peaks phenomenon is commonly observed in LRMs, would non-reasoning LLMs (i.e., foundation LLMs not specifically strengthened for complex reasoning, such as Llama-3. 8B [7)) also exhibit similar behavior? To explore this question, we select the corresponding non-reasoning counterparts the DeepSeek-Rl-Distilll series models and follow the workflow described in Section 2.1 to conduct experiments.
Metrics. To facilitate quantitative comparison between LRMs and their corresponding base models in terms of the properties of MI sequence {m during reasoning, we adopt the following
metrics: (1) Mean:
E.. mi; (2) Standard deviation (Std): om V V
m)² (3)
AOM: AOM goj Σieo
Im meddaani(mm] where is the set of MI peaks defined in Definition 2
(QR(m)
median(m) is the median of the sequence (m.)._1- Specifically, Mean reflects the overall MI
magnitude, while the Std and AOM capture the degree of MI fluctuation.
Non-reasoning LLMs exhibit weaker and less pronounced MI peaks compared to LRMs. As shown Figure while certain steps non-reasoning I.LMs' reasoning process do exhibit increased MI relative the average, the increase is generally mild and lacks the sharp spikes observed in
5

---

Table 2: Statistical comparison of MI sequences between LRMs and their corresponding non-
reasoning LLMs.
Llama-3.1-8B
Qwen2.5-Math-77
Qwen2.5-14B
Qwen2.5-32B
Llama-3,3-70B-Ill
Metric
Origin Reasoning
Origin
Reasoning
Origin Reasoning
Origin Reasoning
Origin
Reasoning
Mean
[0.0863
0.1279
|2.1971
3.3016
1.3128
3.3508
1.7669
4.0352
|0.0400
0.0599
Std
0.0512
0.0707
[0.6699
0.8936
0.4326
0.6703
0.5113
0.6036
0.02277
0.0484
AOM
3.3573
4.5176
2.6330
2.7541
2.6541
3.0820
2.5466
2.5998
2.4326
3.2866
DeepSeek-R1- Di--iii-------sii----a-------
DeepSeek-R1 Barpseee--R-Disilll--e R1-Distill-Qwen-114
QwQ-328
300
300
50
Frequency
200
40
200
30
100
100
20
10
Theat
"Thertton Ineretooe Itaylo
Beet
Maybe
Tokens at Peaks
Takens at Ml Peaks
Tokens at Peaks
Figure 4: Frequency distribution of tokens at MI peaks.
their LRM counterparts. Quantitatively, this observation is further supported by the Std and AOM metrics reported in Table which consistently indicate lower MI fluctuation and peak intensity in non-reasoning LLMs. These findings suggest that the MI peak pattern may emerges from complex reasoning enhanced training.
The overall MI in non-reasoning LLMs during the reasoning process is lower than their corresponding LRMs. Figure 3 and the Mean metric in Table intuitilll and quantitatively validate this observation, respectively. This indicates that after reasoning- intensive training. LRMs seems to fundamentally encode more information relevant to correct reasoning within their representations at each generation step. Furthermore, the presence of MI peaks in LRMs could contribute to raising the overall MI throughout the reasoning trajectory. These observations provide partial empirical support for the theoretical insights presented in Section 2.3 which indicate that higher MI between representations and the golden answer correlates with a greater likelihood of generating correct response.
3 Thinking Tokens are Information Peaks in LLM Reasoning
In Section we identify a distinctive phenomenon in LRMs' reasoning trajectories: the emergence of MI peaks. Then a natural follow-up question is: what semantic information is encoded in the representations at these MI peaks? In this section, we investigate this question from a token-level perspective. Specifically, in Section 3.I we project the representations at MI peaks into the token space and analyze the characteristics of the corresponding tokens. Then Section 3.2 we design experiments to assess the functional role of these tokens, demonstrating that they are crucial for LRM's reasoning performance, while other tokens have minimal impact.
3.1 Exploring MI Peak Representations in Token Space
Projecting representations to token space. To interpret the semantics of representations at MI peaks, we decode these specific representations into the token space using I.LM's output head [50] 59 [5]. Specifically, for representation h;, first compute the corresponding token probability distribution, and then employ a greedy decoding strategy to extract the token with the highest probability:
Pt Softmax MallmmxxWW.lllle Wo + b), â, = arg max ie SE(I....,V)) V) [pe]e
(5)
where W E R Vxd is the output projection matrix, b E RRV is the bias vector, and V is the vocabulary size. We apply the above decoding procedure to all representations at MI peaks across the evaluation datasct. In this way, we analyze the empirical distribution over these decoding tokens, uncovering

---

GSMBK
MATH500
AIME24
85
80
Accuracy
80
70
25
75
60
70
50
20
0
5 10
15
0
5
10
15
0
5
10
15
Number of Suppressed Tokens
Number of Suppressed Tokens
Number of Suppressed Tokens
Token Type
Thinking Tokens
Other Tokens
Figure 5: Impact of suppressing the generation of thinking tokens versus other tokens on LRMs' reasoning performance.
patterns about what types of semantic tokens tend to correspond to these high-MI representations. Specifically, we use the same models and dataset as described in Section 22 conduct experiments. For each model, we aggregate all decoded tokens at MI peaks across the dataset, and then compute their frequency distribution for further analysis.
The tokens that appear at MI peaks are mostly connective words that express self-reflection or transitions in LRM's reasoning process. In Figure we illustrate the top-30 tokens decoded at MI peaks DeepSeek-RR BiteeekkRR--istillee D---istill-LLLMMA
DeepSeek-R1-Disse --Distill--wen---ee and Qw Interestingly, we observe that the MI peak tokens in LRMs are predominantly logical markers and reflective expressions such as "So", "Hmm" ", and "Wair", which are commonly associated with pause, thinking, or internal deliberation. Intuitively, tokens like "IImm" " and "Wait" often prompt the model to self- reflect, consider alternative reasoning paths, etc. For example, we randomly extract responses from LRMs where these tokens appear and observe the follow-up statements: "Wait, let me think differently. Let's denote...." "Hmm, so must have made a mistake somewhere. Let me double-check my calculations. First, ... This behavior aligns with prior work suggesting that such tokens can motivate to perform multi-step reasoning and improve answer accuracy 19 We provide more discussions in Appendix C
3.2 Tokens at MI Peaks are Critical to LRM's Reasoning Performance
Here, we refer those decoded high-MI tokens in Section 3.1 as thinking tokens. These thinking tokens appear to play a dual role: (i) linguistically, they serve as discourse cues that encourage the model think or reflect; and (ii) in hidden space, their corresponding representations contain high MI with the golden answer. Thus, we hypothesize that these thinking tokens may be critical to model's final reasoning results. In this subsection, we conduct experiments to validate this hypothesis.
Suppressing the generation of thinking tokens significantly impairs the reasoning performance of LRMs, while suppressing other tokens has minimal effect. To investigate the role of thinking tokens identified at MI peaks, we conduct controlled intervention experiment. Specifically, during inference with LRMs, we suppress the generation of certain number of thinking tokens by setting their generation probabilities to zero. As comparison, we randomly suppress the same number of non non-thinking token. In this way, we evaluate the model's performance on several math reasoning benchmarks under different numbers of suppression tokens. As shown in Figure [5] suppressing thinking tokens leads to significant degradation in the model's reasoning performance, while suppressing non-thinking tokens has little to no effect (more discussions are provided in Appendix C This indicates that the thinking tokens indeed play a critical role in LRMs' reasoning capabilities, providing empirical support for our previous hypothesis.
4 Applications: Leveraging MI Peaks to Improve LRM Reasoning
Drawing insights from our previous analyses, we propose two simple yet effective techniques to improve LRMs' reasoning performance. In Section 4.]] introduce a method that reuses internal representations at MI peaks to allow the model to further exploit the information in latent space. In
Resulls for the other models are provided in Appendix D
7

---

Section 42
we incorporate the thinking tokens into test-time scaling scenario to improve model's
reasoning accuracy.
4.1 Recycling High-MI Representations During Inference
The MI Peaks phenomenon analyzed Section 2.2 suggests that some representations in LRMs' reasoning process may encode particularly useful semantic information for reasoning. Motivated by this, we propose a simple technique named Representation Recycling (RR). Intuitively, RR feeds the representations at MI peaks back into the model, thereby allowing the model to process and exploit these representations more thoroughly.
Method. Recall that each layer in an LLM typically consists of Transformer block 49 Given an input, the forward computation flow through the layers of an L.L follows:
h TFe(he-1), L 1= L,
where h2 is the output representation of the 1-th transformer block TF:()), and L is the total number of layers. To encourage deeper processing of a potentially important representation he modify the forward computation by feeding it back into the same layer once more: h/4 = TF y (hr*), ), instead of directly passing it to the next layer. Then, for layers & > 8*, we continue the forward pass as usual: h (h' In this way, the above "recycling" operation allows the model to reprocess the high-MI representations to further extract critical reasoning features.
at layer t*,
GSMBK
40
MATH500
AIME44
Experimental setup. To evaluate RR's effective- ness, we conduct experiments on three mathemat- ical reasoning benchmarks using DeepSeek-RI - Distill-Llama--8 and DeepSeek-RI--istill--nee
Accursiy
7B. Since ground-truth answers unavailable during inference, we first record the thinking to- kens using the training set of MATH dataset (as introduced in Section 3.I). and then trigger RR whenever the model generates one of these think- ing tokens. We empirically set 8* to middle or high layers of the LLMs, since previous studies suggest that these layers tend to encode more se- mantically rich content 6
40
82
82
Origin
Figure 6: Reasoning performance of the original LRMs and our RR method across multiple math benchmarks.
Results. As shown in Figure 6 RR consistently improves LRMs' reasoning performance across all benchmarks. In particular, RR yields notable performance improvement the AIME24 dataset, problems. This suggests that recycling the MI-peak
which consists of challenging competition- representations could help LRMs further unlock and leverage their inherent reasoning potential, leading to better reasoning performance.
4.2 Test-Time Scaling with Thinking Tokens
With the diminishing returns of scaling laws in LLMs' training stage, test-time scaling is becoming an increasingly important paradigm for improving the reasoning performance of LRMs [ 42
Prior studies have shown that LLMs' reasoning performance can continue to improve as more compute allocated inference time (22]. Inspired by prior work [33], we propose a simple yet effective strategy called Thinking Token based Test-time Scaling (TTTS).
Method. Given the set of thinking tokens identified in Section 3.1
filter tokens with little semantic content (e.g., punctuations and single characters, see Appendix B for more details) and retain tokens like "So," "Hmm," which often indicate reflection, transition, or further thinking. Then during inference, we append one of these thinking tokens to the end of the model's initial output and allow it to continue generating additional reasoning steps.
Experimental setup. We evaluate TTTS using LLaMA-88 on GSM8K, MATH500, and AIME24. Specifically, consider controlled test-time scaling setting: given LRM with initial token budget, we gradually increase the token generation budget and compare the model's reasoning performance with and without TTTS.
Results. As shown Figure 7 under the same token budget, TTTS consistently outperforms the original LRM on both GSM8K and MATH500. Notably, GSM8K, the original LRM's

---

GSMBK
MATH500
86-
75
(96)
84-
82-
70-
30.0
80
27.5
78
60
25.0
250512
1024
1536
2040
2560
3072
512
2048
3072
4096
5120
1024
4096
6144
5192
12288
Token Budget
Token Budget
Token Budget
Method
-
Origin
TTTS
Figure 7: Reasoning performance of TTTS and the original LRMs across multiple math benchmarks under varying token budgets.
performance plateaus once the token budget exceeds 1024, whereas TTTS continues to yield performance improvements as the token budget increases. On the harder AIME24 benchmark, observe that the original model's performance saturates once the token budget reaches around 3000. contrast, although TTTS underperforms slightly at some intermediate token budgets, its performance continues to improve steadily and eventually surpasses the original model once the budget exceeds 6144 tokens. These results suggest that as more inference-time resources become available, TTTS could effectively prompt LRMs further think, and stably improve the model's reasoning performance.
5 Related work
Reasoning in LLMs. LLMs have achieved significant advancements in understanding, particularly for complex reasoning tasks [53] 28
45 61] The development of multi-step reasoning frameworks began with the chain-of- offthought (CoT) paradigm [53), which introduces structured prompting to formalize explicit intermediate reasoning steps. Surprisingly. this principle is further simplified by (26). where the authors demonstrate that minimalist prompts (e.g.. "Let think step by step") could achieve comparable performance. Authors [62] systematize problem decomposition via cast-to-most prompting hierarchies. This trajectory culminated in [60) formalizing reasoning as tree-structured search processes, enabling backtracking and strategic exploration through explicit state-space modeling. Refinement Strategies also address practical limitations. Wang et al. [51]
introduced self-consistency voting to mitigate output instability.
provides valuable theoretical basis for analyzing the behavior of language models [23 34). with applications spanning numerous fields: reasoning process diagnostics through quantification of unsupervised information gain [47], model optimization via information bottleneck distillation [8] systematic behavior analysis capturing de-
Information Theory in LLMs. Information theory
[2]
Recent extensions formalize synthetic demonstrating information theory's 's ver- satility in bridging theoretical insights with engineering practices. Ren and Liu [40] show that Transformers exhibit an inductive bias toward lower-entropy representations when approximating target distributions.
pendency laws [9] and error propagation dynamics 13
data generation through reverse-bottleneck metrics 14
Critical Tokens in LLMs. Prior work has shown that a small of "critical tokens" can dispro- portionately affect an LLM's behavior, prompting methods to identify them (31). quantify their
influence [16 and mitigate their impact via selective training or pruning (30 44). Recent advances in LLM safety alignment have increasingly focused on the pivotal role of potential critical tokens.
Zou al. (65] propose method craft universal adversarial suffixes that induce aligned L.MSs to generate inappropriate content. Lin et al. (29] find that after alignment, tokens like "sorry," "however," and "apolog" are learned by the model to prevent generating harmful outputs. Qi al. (36) show that simply forcing an unaligned LLM to begin its responses with certain safe tokens can significantly improve the model's safety.
6 Conclusion
In this work, we systematically investigate the reasoning mechanisms of LRMs through an information-theorede perspective. By tracking the MI evolution between intermediate represen-

---

tations and the golden answer, we unveil an interesting MI peaks phenomenon. Further, we find that these MI peaks predominantly correspond to thinking tokens (e.g. "Hmm," "Wait," "There- fore") that express self-reflection, logical transitions, self-correction. Theoretically, we show that higher cumulative MI correlates with tighter bounds on model error, offering insights to the MI peaks phenomenon. Building on these analyzes, we introduce Iwo simple, training-free meth- ods Risseepprsenttttoon Recycling (RR) and Thinking Token based Test-time Scaling (TTTS) that effectively improve LRMs' reasoning performance. We hope analyze could shed new light on the internal structure of LRM reasoning and open up new directions for inference-tines reasoning enhancement.
Acknowledgments
This research was supported by Shanghai Artificial Intelligence Laboratory. National Key Research and Development Program of China (NO. 2024YFE0203200), National Natural Science Founda- tion of China No.6277,,777), CCF-ALIMAMA TECH Kangaroo Retrieval. We also acknowl- 2024008), and Mluawei-Renmin University joint program on Information edge the support provided by the fund for building worldelass universities (disciplines) of Renmin University of China and by the funds from Beijing Key Laboratory of Big Data Management and Analysis Methods, Gaoling School of Artificial Intelligence, Renmin University of China, from Engineering Research Center of Next-Generation Intelligent Search and Recommendation, Ministry of Education, from Intelligent Social Governance Interdisciplinary Platform, Major Innovation & Planning Interdisciplinary Platform for the "DoubleFirst Class" Initiative, Renmin University of China, from Public Policy and Decision-makin Research Lab Renmin University of China, and from Public Computing Cloud, Renmin University of China.
OF
References
[1] AIME Problems and Solutions. httee:///rrooooooommmmmo///////a php/ AIME Problems and Solutions
[2] Sina Abbasi, Mohammad Reza Modarres, and Mohammad Taher Pilchvar. Normxlogit: The head-on-top never lies. arXiv preprint arXiv:2411.16252, 2024.
[3] Mikhail I. Arbuzov, Alexey A Shvets, and Sisong Beir. Beyond exponential decay: Rethinking error accumulation in large language models. arXiv preprint arXiv:2505.24187, 2025.
[4] Robert B Ash. Information theory. Courier Corporation, 2012.
[5] James O Berger. Statistical decision theory and Bayesian analysis. Springer Science Business Media, 2013.
[6] Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering latent knowledge in language models without supervision. arXiv preprint 2rXiv:2212.0388777 2022.
[7] Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards reasoning era: A survey of long cfain-of-thought for reasoning large language models. arXiv preprint arXiv:2503.09567 2025.
[8] Xin Chen, Hanxian Huang, Yanjun Gao, Yi Wang, Jishen Zhao, and Ke Ding. Learning to maximize mutual information for chain-of-thought distillation. arXiv preprint arXiv:2403.03344 2024.
[9] Zhuo Chen, Zhuotao Jin, Di Luo, Marin Soljačié, al. I. m: Mutual information scaling law for long-context language modeling. arXiv preprint arXiv:2503. 04725, 2025.
[10] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reichhro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.1411 2021.
[11] Thomas M Cover. Elements of information theory. John Wiley & Sons, 1999.
10

---

[12] Yunkai Dang, Kaichen Huang, Jiahao Huo, Yibo Yan, Sirui Huang, Dongrui Liu, Mengxi Gao, Jie Zhang, Chen Qian, Kun Wang, et al. Explainable and interpretable multimodal large language models: A comprehensive survey. arXiv preprint arXiv:2412.021004, 2024.
[13] Zeyu Gan, Yun Liao, and Yong Liu. Rethinking external slow-thinking: From snowball errors to probability of correct reasoning. arXiv preprint arXiv:2501. 15602, 2025.
[14] Zeyu Gan and Yong Liu. Towards a theoretical understanding of synthetic data in Il post- training: A reverse-bottleneck perspective. arXiv preprint arXiv:2410.01729, 2024.
[15] Mor Geva, Avi Caciularu, Kevin Wang, and Yoav Goldberg. Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 30-45, 2022.
[16] Roni Goldshmidt and Miriam Horovicz. Tokenshap: Interpreting large language models with monte carlo shapley value estimation. arXiv preprint arXiv:240.100114 2024.
[17] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama
3 herd of models. arXiv preprint arXiv:2407.21783, 2024.
[18] Arthur Gretton, Olivier Bousquet, Alex Smola, and Bernhard Schölkopf. Mcasuring statistical dependence with hilbert-schmidt norms. In International conference on algorithmic learning theory, pages 63-77, 2005.
[19] Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501. 220i::2200..2948,.. 2025.
[20] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Truck (Round 2), 2021.
[21] Jie Huang and Kevin Chen-Chuan Chang. Towards reasoning in large language models: A survey. In Findings of the Association for Computational Linguistics: ACL 2023, pages 1049-1065. Association for Computational Linguistics, 2023.
[22] Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed Bl-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai ol system card, arXiv preprint arXiv:2412.16720, 2024.
[23] Hong Jun Jeon and Benjamin Van Roy. Information-theoretii foundations for machine learning. arXiv preprint 1xX1:22407.12288, 2024.
[24] Di Jin, Eileen Pan, Nassim Oufattole, Wei-Hung Weng, Hanyi Fang, and Peter Szolovits. What disease does this patient have? large-scale open domain question answering dataset from medical exams. Applied Sciences, 11(14):6421, 2021.
[25] Haibo Jin, Peiyan Zhang, Man Luo, and Haohan Wang. Reasoning can hurt the inductive abilities of large language models. arXiv preprint arXiv:2505.24220 2025.
[26] Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 15:22199-22213, 2022.
[27] Alexander Kraskov, Harald Stögbauer, and Peter Grassberger. Estimating mutual information. Physical review F,, 0666138 2004.
[28] Hunter Lightman. Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, llya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference Learning Representations, 2024.
11

---

[29] Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chandra Bhagavatula, and Yejin Choi. The unlocking spell on base LLMs: Rethinking alignment via in-context learning. In The Twelfth International Conference on Learning Representations, 2024.
[30] Zhenghao Lin, Zhibin Gou, Yeyun Gong, Xiao Liu, Yelong Shen, Ruochen Xu, Chen Lin, Yujiu Yang, Jian Jiao, Nan Duan, et al. Rho-1: Not all tokens are what you need. arXiv preprint WrXiv2244.007666 2024.
[31] Zicheng Lin, Tian Liang, Jiahao Xu, Xing Wang. Ruilin Luo, Chufan Shi, Siheng Li, Yujiu Yang, and Zhaopeng Tu. Critical tokens matter: Token-level contrastive estimation enhence Ilm's 's reasoning capability. arXiv preprint arXiv:2411. 19943, 2024.
[32] Wan-Duo Kurt Ma, JP Lewis, and W Bastiaan Kleijn. The hsic bottleneck: Deep learning without back propagation. In Proceedings of the AAAI conference on artificial intelligence, pages 5085-5092, 2020.
[33] Niklas Muennighoff, Zitong Yang. Weijia Shi, Xiang Lisa Li, Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling. arXiv preprint arXiv:2501.19395 2025.
[34] Zhixuan Pan, Shaowen Wang, and Jian Li. Understanding Ilm behaviors via compression: Data generation, knowledge acquisition and scaling laws. arXiv preprint (50.:::2200009597,, 2025.
[35] Ben Poole, Sherjil Ozair, Aaron Van Den Oord, Alex Alemi, and George Tucker. On variational bounds of mutual information. In International Conference on Machine Learning, pages 5171-5180, 2019.
[36] Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, and Peter Henderson. Safety alignment should be made more than just few tokens deep. In The Thirteenth International Conference on Learning Representations 2025.
[37] Chen Qian, Dongrui Liu, Jie Zhang, Yong Liu, and Jing Shao. Dean: Deactivating the coupled neurons to mitigate fairness-privacy conflicts large language models. arXiv preprint 1rX1:22410.16672, 2024.
[38] Chen Qian, Jie Zhang, Wei Yao, Dongrui Liu, Zhenfei Yin, Yu Qiao, Yong Liu, and Jing Shao. Towards tracing trustworthiness dynamics: Revisiting pre-training period of large language models. arXiv preprini arXiv:2402. 19465, 2024.
[39] David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel Bowman. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference on Language Modeling, 2024.
[40] Ruifeng Ren and Yong Liu. Revisiting transformers through the lens of low entropy and dynamic sparsity. arXiv preprini ar0iv:2504. 250.:22904.8999, 2025.
[41] Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Turner. Steering Ilama 2 via contrastive activation addition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15504-15522. Association for Computational Linguistics, 2024.
[42] Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling I1m test-time compute opti- mally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.033314 2024.
[43] Jiankai Sun, Chuanyang Zheng, Enze Xie, Zhengying Liu, Ruihang Chu, Jianing Qiu, Jiaqi Xu, Mingyu Ding, Hongyang Li, Mengzhe Geng, et al. A survey of reasoning with foundation models. arXiv preprint arXiv:2312.11560 2023.
[44] Yao Tao, Yehui Tang, Yun Wang, Mingjian Zhu, Hailin Hu, and Yunhe Wang. Saliency-driven dynamic token pruning for large language models. arXiv preprint arX1v:2504.04514, 2025.
12

---

[45] Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changju Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi kl. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501. 1w..22551..2999 /2599, 2025.
[46] Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025.
[47]
Fran-Francois Ton, Muhammad Faaiz Taufiq, and Yang Liu. Understanding chain-of of of-thhuuht in Ilms through information theory. arXiv preprint arXiv:2411.11988 2024.
[48] John Wilder Tukey et al. Exploratory data analysis, volume 2. Springer, 1977.
[49] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 30, 2017.
[50] Boshi Wang, Xiang Yue, Yu Su, and Huan Sun. Grokking of implicit reasoning in transformers:
A mechanistic journey the edge of generalization. Advances in Neural Information Processing 2024.
Systems,
[51] Xuezhi Wang. Jason Wei, Dale Schuurmans, Quoe Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint 1tt.ii:22333117771 2022.
[52] Yaoting Wang, Shengqiong Wu, Yuecheng Zhang, Shuicheng Yan, Ziwei Liu, Jiebo Luo, and Hao Fei. Multimodal chain-of-thought reasoning: A comprehensive survey. arXiv preprint arXiv:2503. 26005, 2025.
[53] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824-24837, 2022.
[54] Sean Welleck, Amanda Bertsch, Matthew Finlayson, Hailey Schoelkopf, Alex Xie, Graham Neubig, Ili Kulikov, and Zaid Harchaoui. From decoding meta-generation: Inference-time algorithms for large language models. arXiv preprini arXiv:2406. 16838, 2024.
[55] Yuyang Wu, Yifei Wang, Ziyu Ye, Tianqi Du, Stefanie Jegelka, and Yisen Wang. When more is less: Understanding chain -f-thuughtt length 1lms. arXiv preprint arXiv:2502.07266, 2025.
[56] Fengli Xu, Qianyue Hao, Zefang Zong, Jingwei Wang, Yunke Zhang, Jingyi Wang, Xiaochong Lan, Jiahui Gong, Tianjian Ouyang, Fanjin Meng, al. Towards large reasoning models: survey of reinforced reasoning with large language models. arXiv preprint arXiv:2501.0990 00666,, 2025.
[57] An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang. Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men. Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. arXiv preprint arXiv:2412. 15/15, 2024.
[58] An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, et Qwen2. 5-math technical report: Toward mathematical expert model via seff-improvement, arXiv preprint arXiv:2409. /2/22, 2024.
[59] Sohee Yang, Elena Gribovskaya, Nora Kassner, Mor Geva, and Sebastian Riedel. Do large language models latently perform multi-hop reasoning? In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 10210-10229, 2024.
[60] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Ad- vances in Neural Information Processing Systems, 36: 1 1809-11822, 2023.
13

---

[61] Xinhao Yao, Ruifeng Ren, Yun Liao, and Yong Liu. Unveiling the mechanisms of ex- plicit cot training: How chain-of-thouse enhances reasoning generalization. arXiv preprint arXiv:2502.n 04667,, 2025.
[62] Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoe Le, et al. Least-to-moss most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.106230 2022.
[63] Zhi-Hua Zhou. Machine learning. Springer nature, 2021.
[64] Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engineering: A top-down approach
transparency. arXiv preprint arXiv:2310.01405 2023.
[65] Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307. 15043, 2023.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims in the abstract and introduction align rigorously with the paper's contributions and scope. The abstract explicitly highlights three key findings:
(a) Discovery of MI peaks: This is thoroughly explored in Sections 2.2-2.4, where the phenomenon is quantified, theoretically grounded, and contrasted with non-reasoning base models.
(b) Critical role of thinking tokens: Section 3 validates this via token suppression experiments and semantic analysis of MI-aligned tokens.
(c) Practical methods (Representation Recyeling and RTTS): These are empirically tested in Section 4, with results on MATH and AIME24 benchmarks directly supporting the claims.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We write it in the Appendix.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they seale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
15

---

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [Yes]
Justification:
The paper provides complete assumptions and proofs for its theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proof's provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Answer: [Yes]
Justification:
The paper provides sufficient details to reproduce its main experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
16

---

(c) If the contribution is a new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification:
We will provide in the technical appendices and supplementary materials.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification:
The paper provides critical test details necessary for full reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
17

---

Justification:
We considered the errors present in the experimental data and multiple experi- ments presented in the paper.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report I-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative eтror rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question:
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Answer: [Yes]
Justification: We write it in Appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http:///eereeecccpppaaa//cccccca
Answer: [Yes]
Justification:
We read and confirm the NeurIPS Code of Ethic carefully.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
18

---

Answer: [Yes]
Justification:
We write it in Appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deploymentt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning conrectly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our models and datasets are publicly available and widely used the commu- nity, with no indication of sensitive or harmful content.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification:
We write it in Appendix.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
19

---

The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g.. CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g.. website). the copyright and terms of service of that souree should be provided.
assets are released, the license, copyright information, and terms use in package should be provided. For popular datasets, paperssi thoode com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors encouraged to reach out to the asset's ' creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We will provide in the technical appendices and supplementary materials.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the datasct/cod//model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing research with human subjects.
Guidelines:
20

---

The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to LLM for what should or should not be described.
21

---

Contents
Introduction
2
Emergence of MI Peaks in LRMs' Reasoning Trajectories
2.1
Preliminaries
2.2
Investigating LRM's Reasoning Trajectories with MI
2.3
Theoretical Insights: Higher MI Leads to Tighter Bounds on Prediction Error
2.4
Will Non-reasoning LLMs also Exhibit the MI Peaks Phenomenon?
3
Thinking Tokens are Information Peaks in LLM Reasoning
3.1
Exploring MI Peak Representations in Token Space
3.2
Tokens at Ml Peaks Critical to LRM's Reasoning Performance
Applications: Leveraging MI Peaks to Improve LRM Reasoning
4.1
Recycling High-MI Representations During Inference
8
4.2
Test-Time Scaling with Thinking Tokens
5
Related work
Conclusion
Proofs and Definitions
23
A. Proof of Theorem
23
Proof of Theorem
24
3 Definitions
25
Experimental Implementation Details
25
Discussions
26
Additional Experimental Results
27
D.I Additional Experiments on Other Reasoning Domains
27
D.2 MI Peaks in LRMs
28
22

---

A Proofs and Definitions
A.1 Proof of Theorem
Theorem 1. Consider a sequence of representations h1, hu...... hor during an LLM's reasoning process, where T denotes the number of total reasoning steps. Let y, D denote the golden answer and the LLM's prediction answer, respectively. Define P.. = Pr(ū # y) the LLM's s prediction enoor probability. Then the following inequality holds:
T Σ
log (1)1 H(y)
T(: h2 he2)) Ho(pe)
Pe
where [3] is the size of the support of y, and Holpe) denote the binary entropy of Pe that defined by Holpe) = P log Pe (1 - Pe) log(1 -pe).
(2)
Proof. We first define an indicator random variable E = 1(0 * y), where E = 1 if ú # and E 0 otherwise.
By the chain rule of entropy, we have:
Hv 1) = =H 5) + Hyy Us E)
HHE
Hy Ú, 0) Pr(E 0) H(y ú, E. Pr(E 1).
(3)
Since F 0 indicates i
y. we have H(y 0, = 0) = 0. And for H(F) ū). we have:
H(E 5) & H(E) Holve).
(4)
Thus, can derive:
Hly 9) & Holp.) + H.y ú, E = 1).
(5)
Since E 1 indicates ú + y. the random variable y can take at most [>] - values given ÿ as condition. Hence, we have 13
Hly Ú) & Ho(pe) + Pe log((>) 1).
(6)
Based on the definition mutual information, we have:
[l::0 0) = H(y) Hvv
(7)
Combining Eq. (6) and Eq. 7 derives:
Pe > log (1>) -) H()) I(y: 0) Ho(p.)
(8)
Consider an LLM's reasoning process, given the intermediate representations h (hi,h2,.... he...., hr), the output ý is computed as a function of these representations ý [(hrr).. Thus, based on the Data Processing Inequality (DPI), we have:
I(:: 9) < [(y I..T).
(9)
Combining Eq. 8 and Eq. (9) and applying the chain rule of mutual information, we have:
T
Pe log (12) - 1)
(1()) Σ (y: h2 he2)) Holp.)
(10)
which completes the proof.
23

---

A.2 Proof of Theorem 2
Theorem 2. Following the notations in Theorem
the following inequality holds:
Pe
2 H())
Tyr hj hes)
(11)
Proof. The output of a reasoning model can be formulated as multi-class classification task with predicted probabilities p; Pr(ğ = i hir).. According to Bayesian decision theory 63 the conditional error probability is given by:
- (ar( - h1:)))
(12)
For binary classification (D' 2), we have:
min p} $ / 1-pplooge --lloop-pp--- (1-))-l1pp)og
1
log - -p)]-
(13)
Then take an expectation over p:
Pe Eplmin(p, p}l & - p (1 p) log (1 p)]-
(14)
So we derive:
Pe < B h.T)) = H(y h1T).
(15)
This extends multiclass problems through recursive application (see Eq. (6)).
We prove the following inequality by mathematical induction that for any m-class discrete probability distribution (PI....,Pml) pm):
Pe = 1= max(p) < H .m.)
(16)
Base case (m 2y: Direct verification using binary entropy function Eq. 13)
Inductive step: Assume validity for m classes. For m + classes, assume without loss of generality Pm+1 max;(p<) Consider the merged distribution .P....,,m--1y Pm + pm+13 and apply:
The induction hypothesis:
1 { H(I..... . 1.Pm + Pm+1).
1 1 -pmm + Pmn+1)
(17)
2
2. The grouping axiom
Pm
Pm+1
H(p 1m)+1) Hp Pm + Pm 1) (pm Pm+1)
(18)
Pm + Pm+t1 Pm + Pm+1
Binary entropy bound for the final term:
Pm+1
Pm
Pm+1
(19)
Pm
Pm+1
Pm + Pm+1 Pm + pm+1
Combining Eq.
17
Eq.
18 and Eq. 19 completes the induction:
H (pt...... Pm+1) = H(p Pm + Pm++) + (pm + pm+1)
Pm
Pm++1
2
Pm pm+ti Pm + Pm++1
Pm+1
> 1 (pm + pm+1) + (pm + Pm+1)(1-
Dm + Pm++
Pm+1
max(pi)
Thus, have proved the Eq. (16).
24

---

Taking expectation over h1r in Eq. 12 and applying the Eq. (6). we have
Pe E |1 max = [l
2 Bayr [H(y)rr)
H (yllur))
2
T
H(y) Σ
T(y hy hees)
2
==1
which completes the proof.
A.3 Definitions
Definition 3 (Mutual Information 4 27)). Given two continuous random variables and Y, the
mutual information is defined as:
J p(z,z) log log
p(r,y) p(z)p(y)
I(XY))
drdy,
(20)
where p(x,)) 3) denotes the joint probability density function of X and Y; p(x), ply) denotes the marginal probability density functions of X and Y, respectively.
Definition 4 (Hilbert-Schmiid Independence Criterion (HSIC) (8). HSIC is the Hilbert-Schmee norm of the cross-covariance operator between the distributions in Reproducing Kernel Hilbert Space
(RKHS). Formally:
HSIC(X, =ExYX'Y [ix (x, x') by (Y,Y)] + Exx [ex (x, (X,X)]]B Byy fax (Y,Y))
_2 [x [kx X') By [ky (Y,Y
(21)
where X', Y are independent copies of X, Y, respectively, and kx ky are kernel functions.
B
Experimental Implementation Details
Practical implementation of HSIC. Due to the difficulty of accurately computing MI in high- dimensional spaces [27 35 13
we employ the HSIC to estimate MI. Following 32 38 empirical HSIC from Definition is computed
13
the
1 (n 1)2
HSIC(X,Y)
tr Kx H Ky H),
(22)
where Kx and Ky are kernel matrices with entries
Kx,, = (x(,,, KY, = ky((,,
and T- IIT is the centering matrix. Consistent with kernel to implement the kernel:
we adopt the Gaussian
x y1
k(x,)) exp
(23)
202
where the bandwidth a is selected by grid search over the range [50, 400J.
Datasets. 1) Evaluation of LRMs' reasoning performance. We select three widely-used math reasoning benchmarks to evaluate the reasoning capabilities of LRMs, ordering from easy to hard: GSM8K [10], MATH500 [28), and AIME24 M We adopt the evaluation framework provided by Qweh2.5-Math (58). To ensure the reproducibility of our results, we fix the temperature to in all experiments. 2) Observing the MI trajectories during LRMs' reasoning process. We use the training set of the MATH dataset (20. Specifically, we randomly sample 100 instances to compute MI along the reasoning trajectories.
25

---

DeepSeek-R1 Mwit----Diii--eeeeeee Owen-78
DeepSeek-R1 -isstiil---Q-n-32 Owen-328
DeepSeek-A -i-sstil---lamm-70
200
300
80
Frequency
150
60
200
100
100
20
merretor
4
"hey'
Theyt
Waylee
Horm
Ther
Tokens at MI Peaks
Takens at MI Peaks
Tokens at MI Peaks
Figure 8: Frequency distribution of tokens MI peaks for ----Seee-R---isill--O-- RI-Distil--Qwen-32B and DeepSeek-FR -areekk-----iiiii -Dsitiill-Llama--00
DeepSeek-
Models. We conduct experiments DeepSeek's ' RI series models [19] and QwQ-32B [46]. For DeepSeck's RI series models, we pair each LRM with its corresponding non-reasoning LLM counterpart follows: DeepSeek-R1-Disill--Nne 1-Dissilll-Qwen-T1 and Qwen2.5-Math-778 (58) DeepSeek-RI Distill-LJama-8h and Llama-3.1-88 [17). -ee--eeeee-----iiii---N---- 14 and Qwen2.5-14B (57), - -lama-33-770B-Insttt [7]. As observed, all LRMs in the RI series are trained from founda- tion LLMs, except for Dippseek-R--Dstil---eeeeeeee LLM. As for QwQ-32B, existing public report [4] has not disclosed which specific LLM it was trained from. All experiments are conducted on four NVIDIA A100 GPUs.
and Qwen2.5-32B 57 DeepSeek-R1-DD -isbssl--R--iissll--aaaaaaa
and
which is trained from math-specialized
More implementation details. For all experiments involving MI computation, we extract the rep- resentation from the last layer of the model. We concentrate the last layer since higher layers have been shown to encode more semantic content 664 41 and the last layer directly influence the model's 's output text [37. For TTTS in Section 4.2, ensure that the model begins continuation with semantically meaningful tokens, we filter out tokens with little semantic information, such as punctuation, single characters, etc. In this way, the resulting token list is: [So, Let, Hmm, I, Okay, First, Wait, But, Now, Then, Since, Therefore, If, Maybe, To]. All exper- iments are conducted on four NVIDIA A100 GPUs.
C Discussions
Limitations. This work has several limitations. First, we analyze the MI dynamics of LRMs at the token level. Alternative granularities such as dividing reasoning steps by semantic units or logical steps may reveal additional insights. Second, while we observe the interesting MI peaks phenomenon and provide insights into the reasoning mechanisms of LRMs, the underlying mechanisms that give rise to these peaks remain underexplored. We leave deeper analysis of their origin to future work. We hope that our work will inspire further research along these directions and contribute to a deeper understanding of the reasoning process in LRMs.
Broader impacts. This work contributes in LRMs. We first observe the MI peaks phenomenon during LRMs' reasoning process, and then propose two simple training- free methods to enhance LRMs' reasoning performance based on the findings. These analyzes may have positive impacts by making Al systems more transparent and effective. However, there are also potential risks. If used carelessly, the same methods could be applied to manipulate outputs or reinforce biased thinking patterns. It is important to consider these concerns when applying our techniques and to encourage responsible use through further study and monitoring.
deeper understanding of the reasoning mechanisms
Discussion on Tokens at MI Peaks. As shown Figure 4 in the main text and Figure g in the appendix, different LRMs exhibit slightly different token frequency patterns at MI peaks. For models trained from foundation LLMs, i.e., SeepSeek-R--Dsstil--e ---ieei---------------------
DeepSeek-RI-Dissee the frequently
Qwen- 14B, occurring tokens include So, Let, Hmm, The, and Okay. And for DeepSeee-RI-Distil---eee which is trained from math-specialized LLM, tokens such as So, The, Let, To, and, and Since are more prominent. For QwQ-32B, tokens like To, the, we, and Let appear more frequently. Semantically, these tokens commonly express reasoning- functions such initiating thinking
and DeepSeek-R11 --si---lil-----LMMMA
26

---

Table 3: The AOM metric of MI sequences between LRMs and their corresponding non-reasoning LLMs on GPQA and MedQA.
Model
Type
GPQA
MedQA
Llama-3. 1--8B
Origin
3.5193
3.6337
Reasoning
4.0809
3.8436
Qwen2.5- 14BB
Origin
2.7671
2,4355
Reasoning
2.9444
2.6281
(So, Hmm), logical transition (Since, Therefore), discourse structuring (Let, Then, To), which likely help facilitate the model's continued reasoning. We hypothesize that the distribution of tokens MI peaks may be influenced by factors such as the nature of the foundation LLM,, the reasoning- intensive training paradigm, etc. We leave a deeper investigation of the relationship among MI-peak token distributions, foundation LLM characteristics, reasoning-intensinn training paradigms, and model reasoning performance to future work.
Further discussion on thinking token suppression (Section 3.2, Figure 5). As shown in Figure 3.2 while the overall trend indicates that LRMs' reasoning performance degrades as more thinking tokens are suppressed, the decline is not strictly monotonic. In some cases, performance improves temporarily. We conduct an empirical analysis to better understand this phenomenon. Specifically, observe that when certain tokens suppressed, the model tends to adopt alternative expressions to convey similar meanings. For instance, when the generation of the token "Wait" suppressed, the model may instead produce phrases like "But wait", which could lead to slight improvements in performance. The observed performance fluctuations across different numbers of suppression tokens further support that these thinking tokens play critical role LRMs' reasoning capabilities.
Discussion on Theorem 1 and Token Length. In main body, Theorems[aadd are intended to provide theoretical insights into the MI peaks phenomenon. Specifically, they suggest that when the token length T is fixed, the presence of MI peaks may lead to a higher cumulative MI, and thus potentially lower error probability. This implication is also partially supported by the experimental results in Figure 3. Therefore, Theorems [and] heepp establish thcoretical connection between the MI peaks phenomenon and reasoning performance. However, the current formulation of Theorem T may be misinterpreted if extended to arbitrarily large T, as it does not explicitly account for the natural limitations of reasoning in practice. First, there exists upper bound the total amount of information that can be extracted from the input:
Σ hy hee) = Ily; h1r)) & min (v), H & &C,
where C is constant. This bound reflects that increasing the number of reasoning steps cannot provide unlimited additional information. Second, reasoning performance in practice does not improve monotonically with longer sequences. As 7 increases, noise may accumulate, potentially leading to performance degradation due to distraction or loss of coherence. To capture this effect, Eq. (1) can be extended with an additional term + (I',, r). where denotes the noise introduced during step-wise reasoning. The function f can flexibly model different forms of error accumulation. including super-linear [3]. approximately linear (55), and more complex behaviors 25 Together, these considerations provide more faithful description of the relationship between cumulative MI, reasoning length, and prediction error in practice.
D Additional Experimental Results
D.1 Additional Experiments on Other Reasoning Domains
To further examine the generality of the MI peaks phenomenon beyond mathematical reasoning, we ad- ditionally conduct experiments on two reasoning-intensive benchmarks: GPQA (39 and MedQA [24]. -isiesi-t---isst--lllllll
We evaluate both along with their corresponding base models. The results in Table 3 show that the MI peaks phe- nomenon consistently persists across these domains.
and
27

---

Moreover, we examine the tokens corresponding to MI peaks. For GPQA, the set of tokens at MI peaks largely overlaps with that obtained from the MATH dataset in the main text. For MedQA. while common tokens such as "Let", "So", and "Buf" remain prominent, we also observe additional tokens such as "Admin", "Perform", "She", and "He", which may reflect the specific characteristics of the medical domain.
These additional experiments further support the generality of the MI peaks phenomenon, indicating that it is not confined to mathematical problem-solving but also emerges in other domains that require complex reasoning.
D.2
MI Peaks in LRMs
Figures
9
20
illustrate the MI trajectories of various LRMs across more data samples.
28

---

Sampl1
2ample
Semple
Sample
100
400
300
100
303
203
200
300
400
506
100
230
283
400
500
0.5
L00
2:03
203
200
9.53
522
104
20.0
402
540
110
2.0
203
0.6
100
330
703
400
500
330
200
300
400
583
300
206
330
483
500
100
330
303
400
500
Sarrgle
Samplee
Sande
Sample
100
2.30
203
300
230
200
25.
583
200
200
230
540
155
2.00
203
300
Sumphe
Sumple
Sande
Sempie
5.25
000
3.35
Seatoring Stp
Reddoning SER
Rasstning Step
Reato1ing Stop
Sarrpln 21
Somple
Sande
Sample 21
6.0
8.0
6.8
0.0
Persiciing 5tp
Peeserting Stce
Rooson Stop
Teasering Stop
Sumphh 25
Sumple
Sandde
Sumpn 25
Reatcrkin Stop
Reason ng Step
ReatoTing Stop
Sample
Sande
Sample
8.5
300
400
582
206
330
540
100
220
203
500
Scmple
Sumrle
Semone
100
2-31
103
Vmn
193
203
$10
103
104
200
401
100
2.0
193
420
Stop
Sample
0.2
100
220
203
406
530
230
203
305
430
580
203
206
330
480
500
100
230
380
400
500
Sanile
1000
2.33
100
3233
20.5
104
250
481
520
100
2.01
103
300
Sample
Sample
Sande-47
Semple
0.4
8.0
500
300
500
230
500
Nopp
Figure 9: MI trajectories of --SSeee-RR--iissill--aaaaaaa
29

---

DeepS Lisel
Sampla
24
182
203
423
332
J00
300
400
SCB
103
403
330
230
28
420
Saopess
Sueqeo 14
100
203
110
423
131
524
100
330
383
2-0
Pessering
Sinpa53
04
Scrgle 28
SWYTa 56
Sencie 62
224
202
453
383
100
153
230
230
28
Stnpe
Noass Nea
Sompp
Socgle
Sampha E2
Maa
04
06
203
420
123
120
109
399
Swrch X
Sampla
233
18
423
532
2006
100
100
330
230
2.0
323
420
Syrib ST
120
231
11
423
200
103
393
530
2.0
Sripass
Sanpla 20
Sarch 91
Sarcia B2
150
202
2.
453
383
200
530
220
2.0
Stapest
94
Bocorring
Sunglo
Sanola
Sarplo 106
202
feascrring
2110
T10L111772 Clep
Figure 10: (Continued) MI trajectories of RI-Distill---aame
30

---

DespSewk-RL Qweel---
Sample
Sample
Sample
Phasuriigg Step
Rasssi mg Step
Reaicring Sup
Sumple
Sample
Sample?
Samooe
Step
Rousssn mg Step
Feasuningg
Sumbte
Sammbu
Surpph
Suempl
240 Siep
2000 mg Step
2000
3000
542
Sumpre
Sammah
Sueppl
3C0
100
300
5:00
200
542
Sampre
Sumppe
3C0
200
300
5.00
200
543
100
2013
120
200
303
2010
320
210
100
1010
2-30
5050
102
210
550
20.3
2010
250
5-30
2.03
00.0
200
500
2010
20.6
203
402
5:00
200
200
5:30
220
2000
2:00
302
300
200
200
500
100
200
200
500
500
200
200
400
5:00
200
2000
330
400
502
300
200
300
400
200
100
200
200
430
500
500
240
300
403
500
200
2.00
300
400
500
300
220
300
400
200
200
230
200
400
500
500
240
300
400
500
230
2000
330
400
500
Figure 11: MI trajectories of
31

---

DeeoSeek OISUI Qweon
Sampla
309
599
100
200
330
533
100
300
200
522
120
Siop
Paascning Sepp
Slup
Passoring
Teasccing Seap
PRaICI mg 204p
Stup
Sumppe
Samote
Haaiccing Xiop
Sumpu
Sumple 73
Cample
Sumpe
Sumpla
Sire
Sarppio
52
Semtoring Step
Teesceng Siep
Tmasion Sep
Emaasring Step
Sumpesa M4
100
Tusscrin Shp
FM0ACOT 200
Sample
Sarpete
Scrviee 50
Samete 52
593
500
Sample 20
Sumpg 54
Sundie 25
Semoic 35
353
2.50
400
Sample
Sumpas 56
Sundi 53
Sampl 100
252
306
203
320
253
350
420
530
120
200
300
400
530
200
300
400
500
Figure 12: (Continued) MI trajectories of Ri-ieee-----iittll--eeeee
32

---

DuepSeek-RLL 1:
Sample
Sample
Step
Rauson Fi Step
Reaicriing Sup
Rnauining ning Siep
Sumple
Sample
Sample?
Samoie
302
Step
Roisenng Step
Teaicringg Supp
Teasuningg Ti
Sumple
Sumpph
Sumppe
340
200
200
202
Step
Step
Sampre
Sumphi
Sumple
340
200
300
300 Sumple
500
200
542
Sampre
Sumple
300
200
500
300
500
200
320
543
Sampre
Sumple
Scmppe
120
100
201
120
200
320
210
100
200
100
200
5-30
102
210
550
2000
250
302
210
500
203
200
200
200
266
5:00
2000
2:00
302
302
200
200
500
100
200
200
430
500
200
200
400
530
230
2000
300
500
502
300
200
306
400
208
100
200
200
400
500
100
240
300
400
530
200
2000
330
400
500
330
220
300
400
208
200
220
200
400
500
500
200
300
403
500
230
2:00
330
400
500
Figure 13: MI trajectories of DeepSeek-R1--istill-ooee
33

---

DerpSeck RJ Owen
Sampla
383
599
100
290
300
500
Sample
So
Sample
319
533
120
300
500
203
592
Teeseenn Seep
Teasciin Stpp
Emaustig Stop
Jassoring Kupp
Teascing Exp
PRascIin 20p
Sample
Semote
Asstocing Knpp
Masicingg Kiep
Sample:7
Sumppe
Sumple 75
Sample
Sampe
Sumpla B2
Perserin Siee
Teaseenn Ste
Sampasi
Sandda
Sampla
Sentoring Srop
Teaiennn Sap
Emassiig Step
Samplook
Suepess
330
530
500
Aassoring 2010
Tusicrin
Feaichin 240
Sample
Sampeto
Sordie
Sarcic 52
253
523
500
Ausoring
Seep
Sample
54
55
Samdie 35
352
25.3
223
2.00
Passoring
Sampleon OT
50
Sunds 53
Samal 100
252
304
203
513
320
203
330
530
252
200
230
432
100
200
300
400
500
Figure 14: (Continued) MI trajectories of -l-pSeee-RI-Distllll -welll--een-1443 14B.
34

---

Sample
Thasuriing Step
Rausoiin mg Step
Reaicring Sup
Bnavining Siop
Sumple
Sample
Sample
Samohe
302
Step
mg Step
Teaicringg Sup
Feasuningg To
Sumpph
Sueppl
210 Step
200
200
502
Sumpre
Sumphi
3C0
200
300 Sumpph
200
502
Sumpre
Sumple
3C0
200
300
500
200
503
302
110
100
201
120
200
5/30
2000
320
210
100
120
200
203
2110
20.0
1020
210
550
200
2000
250
202
2.00
200
500
203
200
200
200
206
5:30
220
2000
2:00
302
200
500
100
200
200
430
500
100
240
200
403
5:00
230
2000
330
400
502
200
300
400
200
100
230
200
430
500
500
240
300
463
530
200
2000
300
400
500
330
220
300
400
200
100
230
200
430
500
200
240
300
403
500
230
2.00
330
400
500
Figure 15: MI trajectories of
35

---

DeesSeck owen
Sandda
433
189
504
300
2C4
380
53
200
300
2C4
Feanning Dxp
Reatoring Sizz
Seaceirg S1oz
Feasiring Sapp
Eamooring
Reaicing Sieo
Sample
Sumpo
Sample
Skp
Stzp
Roacers Step
Samplh
Corpp
Temprting 2kp
Faleoring Stez
Rawcers Stzz
5C4
Saaeriig
Ramcerg S102
Sample
Sample 53
Simpe 31
201
200 STRO
Sampl::3
Sample 04
Sumpe 25
Serdice 26
223
423
Eameriing
400
120
27
03
Simpo 30
Sample 103
250
232
3-
452
330
180
230
300
430
500
320
200
200
400
545
300
254
200
455
550
Figure 16: (Continued) MI trajectories of
36

---

QwQQ 328
Samele
Sumpl
Sample
Reaiceing Step
Roxsentng Sicp
Teascenrg Step
Rouventng Soop
Sampe
Sampatt
Sampun
Samphe
382
Rearoning 2013
ResIceing STRO
Aabldniigg Stop
SamEE
Sumppe 30
Semrec 33
Sarmaho
0.02 0:00 0.2
1000
2:11
1:00
2000
100
Rasssnning Supp
Sample
2012 320
6:30
102
100
2:30
450
+02
200
50.
5:30
Sumpote
Surrefc
Rasiceing S182
Ressoring Soos
STRD
Astaston Stop
Sample
Sumple 26
Sumeet 2r
Samsako 20
Reaiceing 21e2
Reasoning Soes
Taaasninng Sto2
Reasinningg Srip
Sarrple
Cample 20
Sumpli 33
Samsle 32
2012 320
102
200
Sumple
Sumple
Surpoee 28
Sarrch
100
230
300
3.00
233
320
400
530
100
200
350
420
533
350
230
300
100
330
100
2010
4.00
1:00
200
100
100
2000
350
2.33
120
Reaiceing S1o2
Reasening Sies
Taassening 5183
Reusening Soop
Eurrlle
Surrple
Sernila
Rasccelig STR0
Assterting Ship
Figure 17: MI trajectories of QwQ-32B.
37

---

ONO 328
Samola
20 Fearening
300
300
303
300
200
305
530
100
30%
312
400
200
Sample Va
Savpa-
1700
300
Scmple
Samdle 50
Sample 53
Sampte 50
Feeveniing Stpp
Ratssoiing ong Dtep
Deasortigg Stop
Sumple
Curroa
Sumple
Semple
Sumppi
Sarrae
Samila 26
-0.07
Surple
Savpao
2305 170
120
500
Sumplo
Sampts an
-130 3.M
Raasoring Copp
Teasoring 283
Savedo
Teaseding Ste
Sarrie
Sample
Fasuciing
Raassting
Teessriing
400 325
Figure 18: (Continued) MI trajectories of QwQ-32B
38

---

Deepdieekc RL Distil--lama-700
1ample
Sample 2
Sande
Sappl
24
Reasering Sup
Reaisein SA1
Seasortig Shop
Teaicning Sap
Sample
Sample
Sande
Sample
0.00
Reasertng Sup
Teaiceing Stpp
Samoh
Sumple 20
Sumsh
22
0.0
0.00
Repsentng Srp
Reasceingg
Step
Sumshe 33
Sumpoe
62
3.00
3.00
200
230
300
403
500
430
500
Surrote
Sumsoe
Sempe 20
633
200
430
383
330
330
403
500
500
Sarrete
130
0.00
105
203
915
350
200
123
253
201
105
110
200
103
253
500
Sumpll
152
312
310
250
203
023
102
233 230
5400
200
440
253
303
350
200
203
430
352
233
230
903
500
203
202
500
32
203
230
302
530
200
200
300
300
233
320
403
500
332
203
202
430
500
33
200
330
303
400
530
200
200
320
400
500
350
230
320
400
300
236
200
312
430
500
280
330
303
400
530
200
203
303
400
500
250
230
300
400
300
236
243
332
400
500
Figure 19; MI trajectories of
39

---

Deep 0.50
Senvik
Saneda
100
243
100
500
L0C
400
Sanoa
8:0
109
10
200
L0C
35
400
113
502
30
0.00
30
300
100
Aacsoring SHip
Ramsring
Rassariig S1eo
Semole
Sample
Sendds
Sample
Sanok
Sample
Sandls
Sample
Samplh
Samola
Sample
Temorting 282
Ressoriig S1ea
Sanedo
Sample
10
243
336
500
Rassaring
Rassoring 2142
Sensie 20
Sample
Serdie 31
Sample 32
153
253
23
150
253
402
302
304 Rassonin
Semse 30
Sample
Serdie 25
Samplc 25
100
153
J00 Sample
252
Sunde 97
Sendle 50
Sample 100
210
153
2EB
202
12
120
220
123
433
100
W2
410
Figure 20: (Continued) MI trajectories of DeepSeek-R11DD 70B.
40


Reviews:
[{"review_id": "Vn4XJuIed5", "reviewer": "Reviewer_xCDc", "summary": "This paper investigates the reasoning dynamics of large reasoning models from an information-theoretic perspective, focusing on how mutual information (MI) between intermediate representations and the correct answer evolves during the reasoning process. The authors observe a phenomenon they refer to as \"MI peaks,\" where certain tokens exhibit sharp increases in MI during reasoning. These tokens often correspond to reflective or transitional expressions like \"Hmm,\" \"Wait,\" and \"Therefore,\" which the authors term \"thinking tokens.\" The paper further demonstrates that suppressing these thinking tokens significantly harms reasoning performance, while other tokens have minimal impact. Based on these insights, two methods, Representation Recycling (RR) and Thinking Token-based Test-time Scaling (TTTS), are proposed to improve reasoning performance. Experimental results show consistent improvements across multiple benchmarks.", "questions": "Please refer to the weakness", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "### Strength\n1. The paper presents a well-structured and methodical approach to analyzing reasoning dynamics in LRMs. It starts with identifying MI peaks, then explores their semantic meaning, validates their functional importance, and finally proposes practical applications. The flow is intuitive and easy to follow, making the findings accessible even to readers not deeply familiar with information theory.\n\n2. The experimental results consistently support the theoretical claims throughout the paper. Both proposed methods (RR and TTTS) demonstrate stable improvements in reasoning performance across different datasets (e.g., AIME24, MATH500, GSM8K). These results suggest that the insights gained from the MI peak analysis are not only theoretically sound but also practically useful for enhancing LRM performance.\n\n### Weakness\n1. While the paper identifies MI peaks as important phenomena, it does not address why MI values across different models (even of the same family) are not directly comparable. Additionally, although model sizes vary (e.g., 7B, 8B, 14B, etc.), no clear relationship between model size and MI behavior is established. Given that larger models generally exhibit better reasoning capabilities, the lack of correlation between MI and model scale raises questions about the universality of the MI peak phenomenon.\n\n2. The experiment comparing the suppression of thinking tokens versus randomly selected non-thinking tokens may not be entirely fair. Since non-thinking tokens include many function words or punctuation marks that naturally carry less semantic content, randomly suppressing them might disproportionately affect meaningless tokens. A more rigorous and informative evaluation necessitates the careful selection of token pools that are both semantically meaningful and sufficiently challenging for comparative analysis.\n\n3. The RR method, introduced in Lines 240–248, lacks sufficient detail and hard to understand. Specifically, it is unclear whether this mechanism is equivalent to prompting the model to “think again” using the same input—a strategy akin to asking the model to generate multiple responses and choosing the best one. To validate RR’s novelty and effectiveness, the authors should compare it against a baseline where the model generates multiple responses without recycling and evaluate whether the improvement stems from the recycling mechanism itself or simply from ensemble-like effects.\n\n4. In evaluating TTTS, the paper assumes that appending thinking tokens improves reasoning by encouraging deeper reflection. However, the improvement could also stem from merely extending the output length. A more rigorous evaluation would involve testing whether appending other non-thinking but semantically neutral continuation tokens (e.g., “continue,” “more”) yields similar benefits. Without such a control, it is hard to determine whether the improvement is specifically due to the semantic role of thinking tokens or just the increased token budget.", "was_revised": true, "final_justification": "I thank the authors for their thorough rebuttal and the additional experimental results. They have convincingly addressed most of my initial concerns, reinforcing my confidence in the paper's merits. Consequently, I will maintain my positive score and recommend the paper for acceptance.", "rebuttal": "Thank you for your great efforts in reviewing this paper. We will try our best to answer all your questions. **Please let us know if you still have further concerns, or if you are not satisfied with the current responses, so that we can further update the response ASAP.**\n\n---\n\n**Q1**: \"While the paper identifies MI peaks as important phenomena, it does not address why MI values across different models (even of the same family) are not directly comparable. Additionally, although model sizes vary (e.g., 7B, 8B, 14B, etc.), no clear relationship between model size and MI behavior is established. Given that larger models generally exhibit better reasoning capabilities, the lack of correlation between MI and model scale raises questions about the universality of the MI peak phenomenon.\"\n\n**A1**: This is a good question. \n\n- **There is no clear relationship between model size and MI behaviors according to experimental results in Tables 1 and 2 of the main text**. Specifically, quantitative metrics including #MI Peaks, #All Steps, Ratio of MI Peaks, and Max Interval do not show obvious patterns as the model size scales. Different families and sizes of LLMs have different architectures and dimensions of latent representations, and are trained with different strategies and data. These factors are all closely related to the reasoning capabilities, which need to be further investigated in the community [c1-4]. \n- Nevertheless, **MI peak is a universal phenomenon across LLMs' families (Llama and Qwen) and sizes (7B，8B，14B，32B, and 70B)**, as shown in Figures 2-3 in the main text, Section 2, and Figures 2-13 in Appendix.\n\n[c1] Demystifying Long Chain-of-Thought Reasoning in LLMs\n\n[c2] SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild\n\n[c3] LIMO: Less is More for Reasoning\n\n[c4] Dr.GRPO Understanding R1-Zero-Like Training: A Critical Perspective\n\n---\n\n**Q2**: \"The experiment comparing the suppression of thinking tokens versus randomly selected non-thinking tokens may not be entirely fair. Since non-thinking tokens include many function words or punctuation marks that naturally carry less semantic content, randomly suppressing them might disproportionately affect meaningless tokens. A more rigorous and informative evaluation necessitates the careful selection of token pools that are both semantically meaningful and sufficiently challenging for comparative analysis.\"\n\n**A2**: Thank you for your comment.\n\n- We would like to first clarify that in the original experiments, when selecting random tokens, we filtered out punctuation and other meaningless symbols, keeping only English words.\n\n- In addition, we have **followed your instructions to select more meaningful tokens for comparison**. Specifically, using the training set of the MATH dataset, we extract tokens from model answers by (i) removing the thinking tokens, (ii) requiring English tokens with length > 2, and (iii) selecting the top-frequency tokens. These tokens include *answer, final, can, which, from, have, etc*. Based on this, we follow the experimental setting of Figure 5 and set the number of suppression tokens to 15.\n\n  - | **Dataset**              | **GSM8K** | **MATH500** | **AIME24** |\n    | ------------------------ | --------- | ----------- | ---------- |\n    | **Random**               | 82.9      | 79.2        | 33.3       |\n    | **Thinking tokens**      | 69.0      | 43.8        | 20.0       |\n    | **Top-frequency tokens** | 81.3      | 69.8        | 26.7       |\n\n  -  The results show that suppressing **top-frequency tokens** hurts model performance more than suppressing random tokens. However, suppressing **thinking tokens** still **leads to the most significant performance drop**.\n\n---\n\n**Q3:** \"The RR method, introduced in Lines 240–248, lacks sufficient detail and hard to understand. Specifically, it is unclear whether this mechanism is equivalent to prompting the model to “think again” using the same input—a strategy akin to asking the model to generate multiple responses and choosing the best one. To validate RR’s novelty and effectiveness, the authors should compare it against a baseline where the model generates multiple responses without recycling and evaluate whether the improvement stems from the recycling mechanism itself or simply from ensemble-like effects.\"\n\n**A3**: Thank you for your valuable suggestions.\n\n- **Intuitively, the core idea of RR design is that when the LLM aims to generate \"thinking tokens\", we do not let LLM directly generate subsequent tokens. Instead, we feed the representation of the current token back into the same layer once more**, as introduced in Lines 244-245, hoping that the LLM can better utilize the information in the representation of the \"thinking token\". For example, suppose the sequence currently generated by the LLM is \"... Step 4 is finished. \" At this point, we detect that the next token going to be generated is \"So\" (one of \"thinking tokens\" with high MI). Instead of letting the LLM generate \"So\" directly, we send the corresponding representation of this token back to the current layer once again. Then, the LLM may realize that it needs to check the previous results, thus generating \"Wait\". The subsequent generation process then returns to normal, unless a \"thinking token\" is encountered again, in which case the above process is repeated. **Therefore, RR is not equivalent to prompting the LLM to \"think again\" using the same input**.\n\n- Prompting the LLM to \"think again\" means asking the LLM to re-answer the question, while RR only reuses the representations of some key steps to \"temporarily\" intervene in its thinking process. In this way, the former requires far more computing resources than RR, so a direct comparison is unfair. In fact, prompting the LLMs to \"think again\" and RR are two **orthogonal methods that can be used in combination**. To verify this, **we conducted new experiments** using DeepSeek-R1-Distill-Llama-8B on AIME24.\n\n  - | Method             | Result |\n    | ------------------ | ------ |\n    | Original           | 33.0   |\n    | \"think again\"      | 50.0    |\n    | RR                 | 40.0    |\n    | \"think again\" + RR | 56.66 |\n\n- The experimental results show that both \"think again\" and RR can improve the performance, but their working principles are different (\"think again\" requires significantly more computing resources). Moreover, **\"think again\" can be used in combination with RR to further improve the model's performance**.\n\n---\n\n**Q4**: \"In evaluating TTTS, the paper assumes that appending thinking tokens improves reasoning by encouraging deeper reflection. However, the improvement could also stem from merely extending the output length. A more rigorous evaluation would involve testing whether appending other non-thinking but semantically neutral continuation tokens (e.g., “continue,” “more”) yields similar benefits. Without such a control, it is hard to determine whether the improvement is specifically due to the semantic role of thinking tokens or just the increased token budget.\"\n\n**A4**: Thanks for your comments.\n\n- We agree with your point that \"merely extending the output length can improve model performance.\" We already mentioned this in Lines 267–271, and the performance of the \"Origin\" line in Figure 7 (where only the token budget is increased) also confirms this observation. \n\n- In addition, we **have followed your suggestion to conduct new** **continuous generation experiments** using neutral tokens such as \"continue\" and \"more.\"\n\n  - | **Budget** | **Origin** | **TTTS** | **Continue&More** |\n    | ---------- | ---------- | -------- | ----------------- |\n    | 512        | 57.2       | 59.83    | 59.9              |\n    | 1024       | 63.2       | 65.65    | 65.2              |\n    | 2048       | 70.4       | 72.02    | 70.5              |\n    | 3072       | 73.2       | 75.35    | 72.9              |\n    | 4096       | 75         | 77.27    | 75.4              |\n\n  -  The results show that **continuation with neutral tokens like \"continue\" and \"more\" is less effective than continuation with thinking tokens**."}, {"review_id": "StinSgWQjE", "reviewer": "Reviewer_DDE5", "summary": "The paper presents an information-theoretic investigation into the internal reasoning dynamics of Large Reasoning Models (LRMs). The authors take inspiration from information theory and focus on tracking the mutual information (MI) between intermediate hidden representations and the correct answer as the model generates step-by-step reasoning outputs. They claim to identify a distinctive phenomenon in LRMs, i.e., sharp MI peaks that emerge during certain reasoning steps, which are largely absent in non-reasoning LLMs. These peaks often correspond to the generation of specific tokens like \"Hmm\", \"Wait\", and \"So\", referred to as thinking tokens, which appear to reflect moments of self-reflection, correction, or logical transition. To leverage this observation, the authors propose two training-free inference-time techniques: Representation Recycling (RR), which reprocesses the high-MI representations within the model to enrich reasoning, and Thinking Token Test-Time Scaling (TTTS), which appends thinking tokens to prompt further reasoning and improve output quality.\n\nThe authors present empirical results showing that both RR and TTTS enhance reasoning performance across multiple math benchmarks, including GSM8K, MATH500, and AIME24. Quantitatively, they show that LRMs consistently exhibit higher MI means, greater variance, and more prominent MI peaks compared to their non-reasoning counterparts. Suppressing the generation of thinking tokens during inference leads to substantial performance degradation, while suppressing other tokens does not, confirming their functional importance. Furthermore, TTTS is shown to improve model performance even under tight token budgets, while RR offers consistent gains by reprocessing latent reasoning signals.", "questions": "**Questions**\n\n* The paper defines mutual information between intermediate representations and the golden answer using ​($\\hat{y}$) [specifically in Theorem 1], but it’s not clearly explained whether this includes just the answer or also the input question during computation. Since the MI is highly sensitive to what is treated as “target,” it would be good to clarify 1) Was the pre-question context included in the target representation for MI computation? 2) If not, how was temporal alignment handled between question and answer during MI estimation across timesteps? From the lines of application, this distinction is crucial for accurately interpreting the theoretical results and the empirical MI peaks as reported in the paper. \n\n* Another question is regarding the thinking tokens; it seems many of the identified thinking tokens (e.g., “So,” “Therefore,” “Wait”) are commonly used to begin new sentences (i.e., the sentence always begins with these words/tokens). Did the authors analyze whether these tokens are simply serving as sentence-onset (SOS) markers, rather than uniquely contributing to reasoning via high-MI representations? An ablation comparing the MI of all sentence-starting tokens vs. only those labeled as thinking tokens would clarify whether their impact is truly semantic/functional or a structural artifact of generation that the Language model has learned (this is also related to the previous question, if the gold answer/target is only passed without the pre-text question, then the MI will be high due to this property of start of sentence token). A more causally grounded and controlled experiment would be required to make the proposed finding/claim more reliable and concrete. \n\n**Minor Comments**\n* The paper appears to use or refer to the \"logit lens\" technique, but the citation provided is incorrect or missing. Given that this is a foundational method in interpreting hidden representations, it's important to correctly acknowledge prior work (https://www.lesswrong.com/posts/AcKRB8wDpdaN6v6ru/interpreting-gpt-the-logit-lens). It is recommended to verify and correct the reference accordingly. \n\n* Regarding the presentation quality, Figure 2 aims to show the MI evolution trajectories across different steps of the reasoning process. However, it is visually challenging to extract meaningful patterns without any prior context. The figure should better highlight the trend emphasized in Figure 1, for e.g., via vertical markers at peak positions or in-figure annotation. Additionally, the caption could explicitly state and help the reader by summarizing the key observed trends and their implications, making the figure more self-contained and easy to follow.", "limitations": "Yes, the authors have acknowledged several important limitations in their analysis, particularly the granularity of reasoning steps and the lack of a mechanistic explanation for the emergence of MI peaks. However, as per the NeurIPS guidelines, it is recommended to keep them in the main paper; in the current version, they are present in the appendix.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n* The authors present an empirical analysis showing that MI between intermediate token representations and the final answer exhibits distinct peaks at certain “thinking tokens.” This finding (if weaknesses/questions are addressed properly) is a new information-theoretic perspective on the inner workings of language reasoning models (LRMs), showing interpretable patterns aligned with existing findings regarding procedural knowledge and the importance of reasoning steps in detail.\n\n* The identification of tokens with high MI that correspond to linguistic markers of reasoning (e.g., “So”, “Wait”) provides an important interpretability finding that directly supports the hypothesis that LRMs engage in discrete reasoning steps (via reflection), also detectable through MI computations.\n\n*  The paper proposes extended methods like Representation Recycling (RR) and Test-Time Thinking Token Scaling (TTTS), both training-free interventions inspired by the MI peak observations. These methods demonstrate consistent performance gains on challenging math reasoning benchmarks, showing practical value. (However, a contrast with the existing TTTS approach could be made for better highlighting the contribution, [https://arxiv.org/pdf/2501.19393v2])\n\n\n**Weaknesses**\n\n* One of the major weaknesses of this work is regarding the core theorems presented in this work. The core theorems presented (Theorems 1 and 2) essentially restate or slightly extend well-known results related to Fano’s inequality in mutual information theory. However, the paper neither cites this foundational work nor sufficiently elaborates on how its results substantively advance understanding of MI in the context of LRMs. Furthermore, Theorem 2’s bound scales with the number of tokens T, becoming tighter as T grows, a dependence that appears counterintuitive and is not adequately justified, raising concerns about the practical significance and interpretation of the theoretical analysis. A more detailed justification for increasing the context length and a suitable relation to Fano’s inequality would be good to add in the updated version of the paper. \n\n* The Test-Time Thinking Token Scaling approach strongly resembles the “Simple Test-Time Scaling” technique from existing literature (s1: Simple test-time scaling). The paper could contrasts with this prior work (as mentioned in one of the inspirations [line 270]), highlighting the originality and potential impact of the TTTS method. [https://arxiv.org/pdf/2501.19393v2]\n\n* There are some ambiguities in the MI calculation setup, particularly it is not clearly specified whether the “golden answer” representation used for computing MI includes the pre-question context or only the answer itself. This lack of clarity affects reproducibility and the interpretation of the MI peak dynamics, making it hard to understand in one go and keeping things ambiguous in general. \n\n* Regarding the analysis of “thinking tokens”, the paper does not provide a clear, explicit identification method of the tokens termed as “thinking tokens.” (seems more like a heuristic). Moreover, many of these tokens seem like the sentence-initial markers, which could confound the interpretation that MI peaks correspond to reasoning steps rather than simply sentence boundaries. A systematic analysis of the overlap between sentence-start tokens and thinking tokens is missing, making it more difficult to come to a proper conclusion/finding about the thinking tokens.", "was_revised": true, "final_justification": "After carefully considering the rebuttal and discussion, I would maintain my overall rating. While the paper presents an interesting information-theoretic way of analyzing LLM reasoning and introduces empirical observations, several core issues remain insufficiently addressed. \n\n* The paper introduces Theorems 1 and 2 to ground the observed MI peaks in an information-theoretic framework. However, even after the rebuttal, the theoretical framing remains oversimplified and potentially misleading, especially regarding the dependence on token length 𝑇 (the bound becoming tighter with an increase in the number of tokens seems to be counterintuitive, and may be an oversimplification). Also, as stated in the original review, the theorems restate variants of existing results (Fano-inequality bounds), without proper citations or technical advancement. This limits their originality and utility. Overall, the current theoretical component does not robustly support the primary claims and may mislead readers into overinterpreting the impact of MI accumulation or token length on LLM reasoning performance.\n\n* The authors confirm that MI is computed using representations from an isolated answer-only pass (including reasoning steps, excluding the question). This still raises two major concerns: 1) Feeding only the answer (no question) into the model for the target representation decouples it from the context in which reasoning occurs. Even if CoT steps are included, the model isn't processing them as a response to a specific question. 2) The paper does not adequately justify why the answer is isolated in this way for MI computation. This is somewhat problematic given that reasoning is inherently context-dependent.\n\nThough the idea of using information theory for interpretability is unique, the paper still has some open ends that were not cleared in the rebuttal phase.", "rebuttal": "Thank you for your great efforts in reviewing this paper. We will try our best to answer all your questions. **Please let us know if you still have further concerns, or if you are not satisfied with the current responses, so that we can further update the response ASAP.**\n\n---\n\n**Q1:** \"One of the major weaknesses of this work is regarding the core theorems presented in this work. The core theorems presented (Theorems 1 and 2) essentially restate or slightly extend well-known results related to Fano’s inequality in mutual information theory. However, the paper neither cites this foundational work nor sufficiently elaborates on how its results substantively advance understanding of MI in the context of LRMs. Furthermore, Theorem 2’s bound scales with the number of tokens T, becoming tighter as T grows, a dependence that appears counterintuitive and is not adequately justified, raising concerns about the practical significance and interpretation of the theoretical analysis. A more detailed justification for increasing the context length and a suitable relation to Fano’s inequality would be good to add in the updated version of the paper.\"\n\n**A1**: Thank you for your valuable comments. We will follow your suggestions to add more connections with Fano's inequality and cite relevant works, so as to make the positioning of the theoretical part clearer.\n\n- **In Lines 138-141 and 144-151, we discuss how the theoretical part helps us understand the MI peaks phenomenon.** Specifically, the higher the accumulated MI during the reasoning process, the lower the probability of the LLMs making mistakes. The emergence of MI peaks will potentially increase the final accumulated MI. Experimental results in Figure 3 and Table 2 verify that LRMs have emerged with the phenomenon of MI peaks, and improve the overall MI during the reasoning process (Lines 171-179) compared with their base models. At the same time, the reasoning performance of LRMs is also significantly better than that of base models.\n- **Explaining the number of tokens T in Theorem 2**. Intuitively, as T increases, allowing more MI to be accumulated enables the LLM to explore more during the reasoning stage to approach the correct answer, thus reducing the lower bound of the model's error probability. In addition, the experimental results in Figure 7 in the main text verify the theoretical results: as the number of tokens T increases, the model's performance improves (i.e., the lower and upper bounds of errors in Theorems 2 & 3 are getting tighter, respectively).\n\n---\n\n**Q2**: \"The Test-Time Thinking Token Scaling approach strongly resembles the “Simple Test-Time Scaling” technique from existing literature (s1: Simple test-time scaling). The paper could contrasts with this prior work (as mentioned in one of the inspirations [line 270]), highlighting the originality and potential impact of the TTTS method.\"\n\n**A2**: Thank you. The prior work \"s1: Simple test-time scaling\" has already been cited and discussed in Lines 267-271. The main innovation and contribution of this paper is to analyze the underlying dynamic reasoning mechanisms of LRMs, discovering the MI Peaks phenomenon, and further analyzing the token distribution corresponding to MI Peaks based on information-theoretic tools. Furthermore, to demonstrate the potential real-world application of MI Peaks' theoretical insights, we show two case studies (RR and TTTS) for improving reasoning performance in a training-free manner. In this way, this paper aims to demystify and explain the reasoning mechanisms of LRMs rather than merely focusing on Test-time scaling.\n\n---\n\n**Q3**: \"There are some ambiguities in the MI calculation setup, particularly it is not clearly specified whether the “golden answer” representation used for computing MI includes the pre-question context or only the answer itself. This lack of clarity affects reproducibility and the interpretation of the MI peak dynamics, making it hard to understand in one go and keeping things ambiguous in general.\" \"... Was the pre-question context included in the target representation for MI computation? ... \"\n\n**A3**: Thank you.  As introduced in Lines 90-91, we describe that \"extract the representation of the gold answer by feeding y into the LLM.\" In this way, the representation used for computing MI only includes the answer itself.\n\n---\n\n**Q4**: \"Regarding the analysis of “thinking tokens”, the paper does not provide a clear, explicit identification method of the tokens termed as “thinking tokens.” (seems more like a heuristic). Moreover, many of these tokens seem like the sentence-initial markers, which could confound the interpretation that MI peaks correspond to reasoning steps rather than simply sentence boundaries. A systematic analysis of the overlap between sentence-start tokens and thinking tokens is missing, making it more difficult to come to a proper conclusion/finding about the thinking tokens.\" \"... how was temporal alignment handled between question and answer during MI estimation across timesteps? ...\" \"... Another question is regarding the thinking tokens; it seems many of the identified thinking tokens (e.g., “So,” “Therefore,” “Wait”) are commonly used to begin new sentences (i.e., the sentence always begins with these words/tokens). Did the authors analyze whether these tokens are simply serving as sentence-onset (SOS) markers, rather than uniquely contributing to reasoning via high-MI representations? ...\"\n\n**A4**:  Thank you for your suggestions. As introduced in Lines 51-53 and 212-216, we define the tokens at MI peaks as thinking tokens. \n\nWe have **followed your suggestion and extracted the words at the beginning of all sentences**, then sorted them by their occurrence frequency to get the top ten: \"So, Wait, Let, But, Therefore, The, Alternatively, Then, Hmm, First\". Interestingly, we found that there is a significant overlap between these sentence-onset (SOS) tokens and the thinking tokens at MI peaks in Figure 4. Here, we would like to make the following two statements:\n\n1. **The large overlap between the set of tokens at MI peaks and the set of SOS tokens is not surprising**. The beginning of a sentence usually affects the overall direction of the subsequent content, so it may contain more information, which precisely corresponds to those MI peaks. From another perspective, $I(X;Y) = H(X) - H(X|Y)$, where $H(X)$ measures the uncertainty of $X$. The tokens at the beginning of a sentence usually affect the subsequent trend and require transitions, choices, etc., thus having higher uncertainty; while for tokens in the middle of a sentence, since the semantics of the previous part of the sentence have gradually been determined, it is only necessary to continue generating along the established logic/semantics, thus having higher certainty (this point has also been mentioned in some concurrent works [c1]).\n2. **Considering the frequency of token occurrence further, the distribution of tokens at MI peaks and the distribution of SOS tokens are quite different**. This also indicates that not all SOS tokens have high mutual information.\n\n[c1] Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning\n\n---\n\n**Q5: Minor Comments**\n\n**A5**: Thank you for your constructive suggestions. We will follow your suggestions to make the following revisions:\n\n1. Cite and discuss the \"logit lens\" technique.\n2. Add more notations in Figure 2 to highlight the MI peaks phenomenon, and enrich the captions with descriptions and brief conclusions, so that each figure is more self-contained.\n3. Move the limitation section into the main paper."}, {"review_id": "cPMPYcqOJo", "reviewer": "Reviewer_WHTM", "summary": "This paper discovered that reasoning-relevant tokens seemed to surge during long Chain-of-Thought (CoT) reasoning in the mutual information (MI) perspective, which is an interesting phenomenon in distilled large reasoning language models (LRMs). Through the deep analysis, the authors demonstrated that reasoning tokens played an important role in math reasoning performance: the higher MI indicated better reasoning accuracy. To validate the observation, this paper leveraged the Representation Recycling and Think Token based test-time scaling methods to intervene the LRMs inference, and results were in line with the finding.", "questions": "1. I notice that during the investigation, the authors conduct experiments from 7B to 70B, which is very comprehensive, while the application experiments were conducted only on 7B. I understand that there may be computation resources limited, why did not complement a 14B experiments to make the validation more convincible?\n2. Intervening at the test-time has demonstrated the performance, have the authors considered for post-training phase methods? Just for personal curiosity.\n3. Except for the MI, are there other alternative approaches that can support the analysis results?\n4. This paper listed some frequent tokens such as: “Wait”, “Hmm” and “So”, have the authors statistical the “Alternatively” and how did it perform? As during my investigation, this word is also very common.", "limitations": "The authors put the limitation section in Supplementary Materials, where they have fully discussed the current deficiencies.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n1. This work investigated the reasoning procedures in the lens of MI and observe some phenomena that correct prediction of LRMs appears with significant MI values that bind with specific tokens, which is interesting.\n2. Combining extensive empirical validation and theoretical analysis in both the main body and supplementary materials, the authors revealed the MI peak patterns on both reasoning and non-reasoning LMs, they also explored the relation between token representations and MI peaks and built a set of correlations.\n3. This paper further proposed two inference-time methods inspired by the former findings, and experimental results demonstrated the effectiveness of the presented methods, thereby proving the correctness of the above analysis.\n\n**Weaknesses**\n1. Although this work discovered and demonstrated the positive correlation between the MI peaks and reasoning performance from both the empirical and theoretical perspectives, the investigation and evaluation were mainly across different math problem-solving benchmarks, involving no other domains that demand reasoning, such as GPQA and MedQA, undermining the effectiveness for all reasoning tasks.\n2. The token budgets of DeepSeek-R1-Distill-Qwen models were usually set the maximum as 32768, and the accuracy in Figure 7 seemed not converged, the authors might consider for more max_length to show the overall trends.\n3. Generally, the evaluation of reasoning models should consider the pass@k metric with more sampling times, as the LRMs perform unstably, merely the one-time inference is not that faithful.", "was_revised": true, "final_justification": "I have read all the replies from the authors, and my early minor questions and suggestions have been addressed. Hence, I maintain my positive suggestion for this submission.", "rebuttal": "Thank you for your great efforts in reviewing this paper. We will try our best to answer all your questions. **Please let us know if you still have further concerns, or if you are not satisfied with the current responses, so that we can further update the response ASAP.**\n\n---\n\n**Q1**: \"Although this work discovered and demonstrated the positive correlation between the MI peaks and reasoning performance from both the empirical and theoretical perspectives, the investigation and evaluation were mainly across different math problem-solving benchmarks, involving no other domains that demand reasoning, such as GPQA and MedQA, undermining the effectiveness for all reasoning tasks.\"\n\n**A1**: Thank you for your constructive comment. We have **followed your instructions to conduct new experiments on GPQA and MedQA** to examine whether the MI peaks phenomenon still exists. We used DeepSeek-R1-Distill-Llama-8B, DeepSeek-R1-Distill-Qwen-14B, and their corresponding base models.\n\n1. **On both GPQA and MedQA, the MI peaks phenomenon persists**. Since figures cannot be included here, we instead report the AOM metric (as defined in Lines 160-162, larger values indicate stronger outlier peaks) for reference.\n\n      |                  |           | **GPQA**   | **MedQA**  |\n      | ---------------- | --------- | ---------- | ---------- |\n      | **Llama-3.1-8B** | Origin    | 3.5193     | 3.6337     |\n      |                  | Reasoning | **4.0809** | **3.8436** |\n      | **Qwen2.5-14B**  | Origin    | 2.7671     | 2.4355     |\n      |                  | Reasoning | **2.9444** | **2.6281** |\n\n2. Regarding the **tokens at MI peaks**:\n\n   - For GPQA, the token set largely overlaps with the one obtained from the MATH dataset in the manuscript.\n\n   - For MedQA, while tokens like \"Let,\" \"So,\" and \"But\" still show high overlap, we also observe some new tokens such as \"Admin,\" \"Perform,\" \"She,\" and \"He,\" which may be related to the medical domain.\n\nIn summary, these experiments further validate that the MI peaks phenomenon generalizes to other reasoning domains.\n\n---\n\n**Q2**: \"The token budgets of DeepSeek-R1-Distill-Qwen models were usually set the maximum as 32768, and the accuracy in Figure 7 seemed not converged, the authors might consider for more max_length to show the overall trends.\"\n\n**A2**: Thank you. We follow your suggestions to further increase the token budgets on AIME24 in Figure 7. The results show that after accuracy saturates, **TTTS consistently outperforms the baseline**.\n\n| **Budget** | **Origin** | **TTTS**  |\n| ---------- | ---------- | --------- |\n| 16384      | 33.3       | **34.87** |\n| 20480      | 33.3       | **35.55** |\n| 24576      | 33.3       | **34.87** |\n| 28672      | 33.3       | **35.88** |\n\n---\n\n**Q3**: \"Generally, the evaluation of reasoning models should consider the pass@k metric with more sampling times, as the LRMs perform unstably, merely the one-time inference is not that faithful.\"\n\n**A3**: Thank you for your suggestion. We have followed your suggestions to **conduct new experiments by adding pass@3 as an evaluation metric for RR and TTTS**, following settings in Lines 248-260 and 277-280. The results on AIME24 with DeepSeek-R1-Distill-Llama-8B are as follows:\n\n| **Budget** | **Origin** | **TTTS**   |\n| ---------- | ---------- | ---------- |\n| 2048       | 0.3666     | **0.3866** |\n| 3072       | 0.3666     | **0.4266** |\n| 4096       | 0.3666     | **0.4444** |\n| 6144       | 0.4333     | **0.4733** |\n| 8192       | 0.4        | **0.4888** |\n\n| **Origin** | **RR**     |\n| ---------- | ---------- |\n| 0.5        | **0.5666** |\n\nThe results show that **when evaluated with pass@3, both TTTS and RR consistently outperform the baselines**.\n\n---\n\n**Q4**: \"I notice that during the investigation, the authors conduct experiments from 7B to 70B, which is very comprehensive, while the application experiments were conducted only on 7B. I understand that there may be computation resources limited, why did not complement a 14B experiments to make the validation more convincible?\"\n\n**A4**: Thank you for your constructive suggestions. We have followed your suggestions to **conduct new experiments** to validate the effectiveness of RR and TTTS using **DeepSeek-R1-Distill-Qwen-14B** on AIME24 following the settings in Lines 248-260, 277-280. The following experimental results indicate that 1) **TTTS** performs worse than the origin model at early stages of the token budget but continues to grow steadily later  (similar to the phenomenon described in Lines 184–190); 2) **RR** shows a clear improvement over the origin model.\n\n| **Budget** | **Origin** | **TTTS**  |\n| ---------- | ---------- | --------- |\n| 2048       | **40.0**   | 36.91     |\n| 3072       | **43.3**   | 42.55     |\n| 4096       | 46.7       | **47.01** |\n| 6144       | 53.3       | **54.15** |\n| 8192       | 53.3       | **55.28** |\n\n| **Origin** | **RR**     |\n| ---------- | ---------- |\n| 0.5670     | **0.6670** |\n\n---\n\n**Q5**: \"Intervening at the test-time has demonstrated the performance, have the authors considered for post-training phase methods? Just for personal curiosity.\"\n\n**A5**: Thank you for your valuable suggestions. For simplicity and ease of use, we only applied thinking tokens during the test-time phase. Meanwhile, it can also be developed into post-training phase methods, and here are two potential integration directions:\n\n1. During RL (Reinforcement Learning) and SFT (Supervised Fine-Tuning), higher weights can be assigned to these \"thinking tokens.\"\n2. During RL, multiple rollouts could be performed for thinking tokens to allow more exploration.\n\nSome more recent works are exploring similar ideas, and we are also happy to explore and verify these post-training applications in future work.\n\n---\n\n**Q6**: \"Except for the MI, are there other alternative approaches that can support the analysis results?\"\n\n**A6**: A good question. Some similarity and distance measuring metrics and tools, such as Center Kernel Alignment (CKA) and Kullback–Leibler (KL) divergence, can also be used to support the analysis results. We are happy to use different metrics in future work.\n\n---\n\n**Q7**: \"This paper listed some frequent tokens such as: “Wait”, “Hmm” and “So”, have the authors statistical the “Alternatively” and how did it perform? As during my investigation, this word is also very common.\"\n\n**A7**: A good question. \nYes, we did record the token “Alternatively”, which appears on the left of Figure 1(b) in the main text. Regarding its frequency, it ranks roughly within the 20th and 40th across DeepSeek-R1-Distill-Llama-8B, DeepSeek-R1-Distill-Qwen-14B, and QwQ-32B, while Figure 4 only demonstrates top-15 tokens."}, {"review_id": "s7WaCvdOwx", "reviewer": "Reviewer_Pihp", "summary": "The paper analyzes how the mutual information (MI) between intermediate representations and the ground truth answer evolves in reasoning models. The MI is estimated by computing the HSIC criterion between the internal representation of each step, and the LLM representation of the answer. It is observed that the MI sequence sharply peaks at specific steps throughout the reasoning process. Such peaks often correspond to thinking tokens (so,wait,hmm,therefore), aligning with the popular belief that self-correction or reflection during thinking boosts reasoning ability. Suppressing thinking tokens are demonstrated to harm reasoning performance, justifying their importance. Moreover, two methods (reusing representations and forcing thinking token generation) are proposed to improve reasoning performance at test time.", "questions": "See Weaknesses", "limitations": "Yes (Appendix C)", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 4, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\nThe paper enables a novel quantitative analysis to a very timely topic. The proposed MI experiments and RR/TTTS methods quantifies the crucial importance of thinking tokens, or transitions in reasoning paths. The methods not only make reasoning amenable to information theoretic analyses but also serve as a guideline to enable more effective inference time scaling.\n\n**Weaknesses/Questions**\n\nIt is intuitively not very clear to me what exactly MI is measuring. If MI is to be taken as how informative the hidden representation at each step is about the correct answer, should we not expect MI to increase throughout the reasoning process as the model arrives closer to the answer? (Perhaps this can be quantified instead as the MI between the answer and the concatenated representations up to time $T$.) Why does MI increase sharply at thinking tokens but fall back down afterwards? The paper does a great job at showing the importance of thinking tokens but could have more discussion on what this signifies or why this happens.\n\nIt should be mentioned that Theorem 1 basically follows from an application of Fano's inequality (Eq.6). Also, the result is stated with the conditional MI $I(y;h_j|h_{<j})$, so the precise connection with the experiments is unclear. Can we also compute the conditional MI in the experiments?\n\nSome results may not be statistically significant, in particular the gaps between the original and proposed models in Figure 7 are not very large and would benefit from adding deviations/error bars over a couple of trials.", "was_revised": true, "final_justification": "The authors have addressed my concerns on statistical significance and whether MI behaves as expected (measuring concatenated MI). Overall, this is a strong empirical paper and raises important points which merit further study from an information theory perspective.", "rebuttal": "Thank you for your great efforts in reviewing this paper. We will try our best to answer all your questions. **Please let us know if you still have further concerns, or if you are not satisfied with the current responses, so that we can further update the response ASAP.**\n\n---\n\n**Q1**: \"It is intuitively not very clear to me what exactly MI is measuring. If MI is to be taken as how informative the hidden representation at each step is about the correct answer, should we not expect MI to increase throughout the reasoning process as the model arrives closer to the answer? (Perhaps this can be quantified instead as the MI between the answer and the concatenated representations up to time .) Why does MI increase sharply at thinking tokens but fall back down afterwards? The paper does a great job at showing the importance of thinking tokens but could have more discussion on what this signifies or why this happens.\"\n\n**A1**: Thank you for your insightful comment!\n\n- **Intuitively, your understanding of \"what exactly MI is measuring\" is completely correct**, i.e., \"MI is to be taken as how informative the hidden representation at each step is about the correct answer.\" However, what we compute is the MI between the representation of **each generated token** and the answer. Since **the representation of a single token may not contain all information up to the current step**, the MI of the whole sequence is not necessarily monotonically increasing.\n\n- Following your suggestion, we **conduct new experiments** to quantify the MI between the answer and the **concatenated representations up to each time step**. Specifically, we follow the same experimental setting as in the paper and use the DeepSeek-R1-Distill-Qwen-7B model on the training split of the MATH dataset. We compute Spearman’s ρ and Kendall’s τ to measure the monotonicity of the MI sequence. Both metrics take values in [−1,1], with **values closer to +1 indicating stronger monotonic increasing trends**. \n\n   | **Spearman’s ρ** |         | **Kendall’s τ** |          |\n   | ---------------- | ------- | --------------- | -------- |\n   | value            | p-value | value           | p-value  |\n   | 0.9204           | 0.0051  | 0.8150          | $4.2e^{-11}$ |\n\n   The results show that when computing MI between the concatenated representations and the answer, **the MI sequence indeed increases as reasoning progresses, consistent with your insight**.\n\n- Regarding \"*Why does MI increase sharply at thinking tokens but fall back down afterwards\"*, we propose a possible explanation from the perspective of the definition of MI: $I(X;Y) = H(X) - H(X|Y)$, where $H(X)$ measures the uncertainty of $X$. During reasoning, when some tokens represent transitions or explore new reasoning paths (e.g., *“Wait”*, *“But”*), they may have higher uncertainty. In contrast, most other tokens simply continue along an already established reasoning path in a more deterministic manner (e.g., *“1+1=2”*, *“and”*).\n\n- Finally, thank you for your recognition of our paper. We fully agree with your point that it is interesting and important to further investigate why the MI peaking phenomenon occurs (which we also discussed in the limitations section). We hope our work can inspire future explorations in this direction.\n\n---\n\n**Q2**: \"It should be mentioned that Theorem 1 basically follows from an application of Fano's inequality (Eq.6). Also, the result is stated with the conditional MI , so the precise connection with the experiments is unclear. Can we also compute the conditional MI in the experiments?\"\n\n**A2**: Thank you for your comment.\n\n- We will make the connection to Fano’s inequality explicit in the revised manuscript.\n- Regarding the conditional MI $I(y; h_j | h_{<j})$ in Theorem 1, we follow your suggestion to improve clarity. By the chain rule and the definition of mutual information, we can rewrite: $I(y; h_j | h_{<j}) = I(y; h_j) + I(h_j; h_{<j} | y) -  I(h_j; h_{<j})$. Since in the setting of this paper, we assume that y is independent  to $h_j$, $h_{<j}$, respectively (only the input X affects $h_j$, $h_{<j}$. Therefore, $I(h_j; h_{<j} | y) = I(h_j; h_{<j})$. As a result, the conditional MI in Theorem 1 $I(y; h_j | h_{<j})$ can be directly replaced by $I(y; h_j)$。\n\n---\n\n**Q3**: \"Some results may not be statistically significant, in particular the gaps between the original and proposed models in Figure 7 are not very large and would benefit from adding deviations/error bars over a couple of trials.\"\n\n**A3**: Thank you for your suggestion. **We have followed your instructions to conduct new experiments**, running both the origin and TTTS settings in Figure 7 **three times and reporting the mean and standard deviation**. The results are as follows:\n\nMATH500:\n\n| **Budget** | **Origin**    | **TTTS**          |\n| ---------- | ------------- | ----------------- |\n| 512        | 0.5200±0.0214 | **0.5944±0.0076** |\n| 1024       | 0.5827±0.0155 | **0.6404±0.0083** |\n| 1536       | 0.6273±0.0057 | **0.6788±0.0071** |\n| 2048       | 0.6620±0.0085 | **0.7049±0.0071** |\n| 3072       | 0.6907±0.0203 | **0.7347±0.0072** |\n| 4096       | 0.7207±0.0077 | **0.7452±0.0115** |\n| 5120       | 0.7433±0.0098 | **0.7649±0.0069** |\n| 7168       | 0.7633±0.0125 | **0.7792±0.0094** |\n\nGSM8K：\n\n| **Budget** | **Origin**    | **TTTS**          |\n| ---------- | ------------- | ----------------- |\n| 256        | 0.6967±0.0054 | **0.7292±0.0076** |\n| 512        | 0.7627±0.0079 | **0.7837±0.0072** |\n| 1024       | 0.7988±0.0126 | **0.8195±0.0063** |\n| 1536       | 0.8029±0.0041 | **0.8310±0.0065** |\n| 2048       | 0.8057±0.0088 | **0.8394±0.0058** |\n| 2560       | 0.8150±0.0025 | **0.8404±0.0054** |\n| 3072       | 0.8175±0.0087 | **0.8441±0.0044** |\n\nAIME24：\n\n| **Budget** | **Origin**    | **TTTS**          |\n| ---------- | ------------- | ----------------- |\n| 1024       | 0.1889±0.0157 | **0.3081±0.0154** |\n| 2048       | 0.2222±0.0416 | **0.3244±0.0208** |\n| 4096       | 0.2667±0.0471 | **0.3674±0.0332** |\n| 6144       | 0.3222±0.0416 | **0.3896±0.0348** |\n| 8192       | 0.3222±0.0416 | **0.3933±0.0357** |\n| 12288      | 0.3667±0.0720 | **0.4044±0.0354** |\n\nThe results show that across three runs, **TTTS consistently outperforms the original model**, demonstrating stable performance improvements."}]

Author rebuttal:

Meta-review: This paper presents an interesting analysis showing that there are critical tokens in thinking based LLMs where the mutual information (MI) between the token representation and the answer representation spikes – these are identified as “thinking tokens”. On further analysis, the top thinking tokens look like “Hmm”, “Wait” and “Therefore,” that express reflection or transition and are shown to be crucial for the LLM’s performance since suppressing them leads to large drops in accuracy. These findings inspire some inference-time ideas that practically improves the accuracy of LLMs by looping on representations of thinking tokens and inserting more thinking tokens.

---

The reviewers liked the overall idea and execution of the paper. There were some concerns about baselines, evaluation metrics and the significance of the theoretical result. The author's response addressed most of the concerns.

R1: MI is not comparable across scales, better baseline for token suppression, different baselines for the proposed inference-time methods —- *Addressed by showing MI spikes at many scales and adding more baselines*

R2: Theory follows from Fano’s inequality, possibly misrepresentative and loose, issues with how MI was calculated —- *Partially addressed through relaxed theory and more details on MI calculation*

R3: Not enough benchmarks. Larger token budget and pass@k —- *Addressed by adding 14B results and pass@k evaluations*

R4: More details on what MI signifies, results follows from Fano —- *Promised to address*


The theoretical results, as multiple reviewers pointed out, basically follow from known results. So that’s not the most significant contribution of the paper but simply justifies the existence of such critical tokens with large MI.

---


Overall the paper makes an interesting observation and shows some practical utility for it. The authors also did a fair job of addressing most reviewer concerns. On the whole, the paper makes a positive contribution towards better understanding the thinking mechanism in LLMs. Thus the recommendation is accept, assuming the authors incorporate the reviewer suggestions (especially about the use of Fano’s inequality in the theory)
Decision: Accept (poster)