Reasoning Models Know When They’re Right: Probing Hidden States for Self-Verification
Authors: Anqi Zhang
Yulin Chen
Jane Pan
Chen Zhao
Aurojit Panda
Jinyang Li
He He
Venue: COLM 2025 (2025)
Keywords: Reasoning models; Chain-of-thought reasoning (CoT); Intermediate answers; Overthinking
TL;DR: Reasoning models with long chain-of-thought encode strong signals about the correctness of intermediate answers in model's hidden states, and we can use it for early exit.

Abstract: Reasoning models have achieved remarkable performance on tasks like math and logical reasoning thanks to their ability to search during reasoning. However, they still suffer from \textit{overthinking}, often performing unnecessary reasoning steps even after reaching the correct answer. This raises the question: \textit{can models evaluate the correctness of their intermediate answers during reasoning?}
In this work, we study whether reasoning models  encode information about answer correctness through probing the model's hidden states. The resulting probe can verify intermediate answers with high accuracy and produces highly calibrated scores. Additionally, we find models' hidden states encode correctness of future answers, enabling ealy prediction of the correctness before the intermediate answer is fully formulated.
We then use the probe as a verifier to decide whether to exit reasoning at intermediate answers during inference, reducing the number of inference tokens by 24\% without compromising performance. These findings confirm that reasoning models do encode a notion of correctness yet fail to exploit it, revealing substantial untapped potential to enhance their efficiency.

Published as a conference paper at COLM 2025
Reasoning Models Know When They're Right: Probing Hid- den States for Self-Verification
Anqi Zhang', Yulin Chen Jane Pan', Chen Zhao Aurojit Panda', Jiinanng Li', He He NNww York University NYU Shanghai
Abstract
Reasoning models have achieved remarkable performance tasks like math and logical reasoning thanks to their ability to search during rea- soning- However, they still suffer from overthinking, often performing unnecessary reasoning steps even after reaching the correct answer. This raises the question: can models evaluate the correctness of their intermediate answers during reasoning? In this work, we study whether reasoning models encode information about answer correctness through probing the model's hidden states. The resulting probe can verify intermediate answers with high accuracy and produces highly calibrated scores. Additionally, we find models' hidden states encode correctness of future answers, enabling early prediction of the correctness before the intermediate answer is fully formulated. We then use the probe
verifier to decide whether to exit reasoning at intermediate answers during inference, reducing the number of inference tokens by 24% without compromising performance. These findings confirm that reasoning models do encode notion of correctness yet fail to exploit it, revealing substantial untapped potential enhance their efficiency.
1 Introduction
Recent advances in reasoning models, such as OpenAl's (OpenAI, 2024) and DeepSeek- R1 (DeepSeek-Aled et al., 2025), have demonstrated significant progress in complex reasoning capabilities, particularly in domains such as mathematical problem solving (DeepMind, 2024; Zhou et al., 2023) and logical reasoning (Feng et al., 2023; Liu et al., 2025; Lam et al., 2024). A key advantage of reasoning models lies in their ability to search: they often explore multiple reasoning paths leading to different intermediate answers to the original problem before arriving at a final solution (Figure 1, left). While this search-based reasoning is beneficial, it also introduces inefficiencies. Previous studies (Chen al., 2025; Sui et al., 2025) show that reasoning models tend to overthink by exploring additional reasoning paths even after reaching a correct answer.
This observation prompts the question: to what extent can models evaluate the correctness of their intermediate answers during reasoning? Existing works that interpret hidden states primarily focuses on non-reasoning models with short Chain-of-Ti Chain-of-Thoughtts (CoT) (Zhu et al., 2024; Xie et al., 2024; Li et al., 2024b; Kadavath et al., 2022; Orgad et al., 2024). which cannot address our central research question of whether reasoning models know when they're re right during their unique search behavior, especially given the observed overthinking phenomenon. The answer to this question is also crucial to preventing overthinking, either through more targeted design of the training strategy or a better elicitation method.
We investigate this question by probing the model's 's hidden states for answer correctness. Specifically, we segment the long Chain-of- ThoThht (CoT) into chunks containing interme- diate answers, and train binary classifier to predict answer correctness from the model's hidden states at the answer positions (Figure 1).
Correspondence to:
hhe@nyu.edu
thub. 'The code and probes are available in: https:/ httpp:///gitubb..o////ggggge wwt.://wwa/////ee/eo... /tap://///eesoonnmmmmeee....
probing

---

Published as a conference paper at COLM 2025
Question: John plans to sell all his toys and use the money to buy video games. He has 13 lego sets and he sells them for $15 each. ends up buying video games for $20 each and has left. How many lego sets does he still have?
Long CoT Reasoning
Chunks in Long CoT Reasoning
Probability of Answer Being Correct
Okay, John has tays So, he sold toys. Walt... So, he sold all he should have none left, right?
Wait, let me back... So, sold 11 LEGO sets and got 165, so he has none left. But the problom....
Let me make sure... Ah! Maybe the problem is asking how many he still has... So, he sold 11, has 2 left. The answer is
Probe
x
x
Altomatively, me verify this... So, the answer 2. *"Fnnll Answer" still LEGO sets left.
Figure 1: An illustration of the probing method. On the left side, long CoT is parsed into multiple chunks, each corresponding to a reasoning path and contains an intermediate answer as termination. On the right side, representations for each chunk are obtained and probe is used to predict the probability of answer being correct.
We find that information about answer correctness is readily encoded in the model's internal representations. A simple probe can reliably extract this information, performing accurately on both in-distribution and nat-of-distribbutoo examples. Moreover, the probe is highly calibrated, with an expected calibration error (ECE) below 0.1. Our analysis also reveals that the model's hidden states contain "look-ahead" information: correctness can be predicted even before the intermediate answer is fully articulated. Notably, when applying the same probing method to traditional short CoT models, we observe significant degradation in both accuracy and calibration performance, suggesting that the encoded correctness information is likely acquired during reasoning-specific training. We hypothesize that exposure to long CoT patterns with both correct and incorrect intermediate steps enables reasoning models to implicitly learn answer correctness, even without explicit supervised signals.
We also investigate whether reasoning models effectively use this information on answer correctness during inference. Because the trained probe is well-calibrated, we use the output score to measure the model's 's confidence in the current intermediate answer. Ideally, the model should reason at an optimal length if it is taking advantage of the well-encoded correctness information, i.e. it should stop reasoning when the confidence about an intermediate answer is high enough. We adopt the probe as a verifier and implement confidence-based early- exit strategy by thresholding confidence scores from the probe. The strategy achieves up to 24% red oddutiion in inference tokens without compromising accuracy. The improvement in efficiency with our verifier reveals that while reasoning models encode information about answer correctness, they do not efficiently use this internal knowledge during inference.
2 Related work
Uncertainty estimation in LLMs. Black-box techniques for estimating LLM uncertainty over their response have primarily focused on prompting the model to verbalize its con- fidence directly, often aggregating self-reported confidence scores over multiple samples (Lin et al., 2022; Tian et al., 2023). However, Xiong et al. (2024); Kapoor et al. (2024) find that white-box methods, including those that depend on internal model representations (Mielke et al., 2022), tend to perform better than black-box methods on confidence estimation. For in- stance, Azaria & Mitchell (2023); Burns et al. (2024) show that an LLM's representation after processing statement is highly predictive of the statement's 's correctness; moreover, linear probes trained on these representations can classify correctness, even without ground-trun -tuth labels. We extend this work to long CoT generated by reasoning models, demonstrating that
2

---

Published as a conference paper at COLM 2025
the representations at intermediate stages of the CoT also capture key information about the correctness of each intermediate stage. Especially, we show reasoning models exhibit substantially better calibration, which is inline with more recent work (Yoon et al. (2025)) showing that reasoning models better express their confidence.
Efficient reasoning during inference. Reasoning models demonstrate improved per- formance on many tasks thanks to their ability to search while generating reasoning chains, which often demand additional test-time compute in comparison to standard CoT (DeepSeek-AL al., 2025). Additionally, reasoning models often suffer from repeated and unnecessary reasoning steps-de or wwerthinkingg-eerr after a correct answer has been reached (Chen et al., 2025). Recent work has explored training methods to make reasoning more concise or to reduce the frequency of overthinking (Chen et al., 2025; Munkhbat et al., 2025). Other inference-ting be successful (Zhao et al., 2025; Manvi et al., 2024; Li et al., 2024a) or dynamically adjusting the test-time compute budget based on input difficulty or other properties of the prompt (Damani et al., 2024; Wang et al., 2025; Xu et al., 2024; Fu et al., 2024). We find that while the model encodes information about answer correctness, it fails to use it efficiently, which may contribute to overthinking. We leverage this to perform threshold-based early-exiting at inference time, reducing test-time compute while preserving performance.
techniques focus on curtailing generations that are unlikely to
Learned verifiers. The ability to verify intermediate answers is also related to the line of works on verifiers, which is an important technique used regulate test-time scaling. Previous work has focused on training verifiers to classify the correctness of a model- generated solution or select which of two model-ggnerated responses is preferred (Bai et al., 2022; Cobbe et al., 2021; Lightman et al., 2023; Zhang et al., 2025; Zheng et al., 2023; Creswell & Shanahan, 2022; Paul et al., 2024). However, recent improvements in reasoning capabilities have enabled LLMs to critique and refine their outputs without the aid of external verifiers, often using natural language prompt templates to guide self-critique of model- -eeee--geeerttdd output (Ling et al., 2023; Zhang et al., 2024; Madaan et al., 2023; Weng et al., 2023; Shinn et al., 2023). In contrast, we focus on leveraging information about correctness which is encoded in the model representations of the reasoning chain.
3 Probing for intermediate answer correctness
The long CoT output from a reasoning model often contains multiple mentions of infer- mediate answers. We aim to explore whether the notion of "correctness" is encoded in the representation of each intermediate answer by probing. This section describes how we identify intermediate answers, obtain their representations, and train a two-layer multilayer perceptron (MLP) probe.
3.1 Data collection
We first collect responses from reasoning models for each problem in the task dataset. The reasoning trace, which is encapsulated in <think> tokens, is extracted and split into para- graphs with "\n',n" delimiter. We identify the start of a new reasoning path by detecting keywords like "wait", "double-check" and " "alternatively" in each paragraph. Á complete list of the keywords is shown in Table 3 in the appendix. We merge paragraphs in the same reasoning path to form chunk. Then we use Gemini 2.0 Flash (Gemini-Teaae 2024) to extract the intermediate answer in each chunk one exists, and judge its correctness against the true answer. Finally, adjacent chunks that do not contain an intermediate answer are merged with the nearest chunk that contains an answer. Each merged chunk now has an interme- diate answer and label generated by Gemini, represented as {C1,y))) (C2, (C2.1)),....e (t), where each c; is part of the reasoning trace that contains an answer to the original problem, and y; is binary label indicating the correctness of the answer. $ A.3 presents results the sensitivity of LLM-based annotation with different advanced LLMs. The high consistency shows that this task is relatively straightforward for state-of-the-art L.LMs.
3

---

Published as a conference paper at COLM 2025
The next step is to obtain the model representation for each chunk. For each chunk C₁ take the last-layer hidden states at the last token position as its representation ej. Finally, for each task dataset, we collect set of reasoning representations and their corresponding labels, formulating the probing dataset DD y;) that will be finally used to train probes. Note that the construction of probing dataset D depends on both the original task dataset and the reasoning model we use to generate representations.
3.2 Training the probe
After obtaining the probing dataset, train two-layer multilayer perceptron on D. Since the datasets are often highly imbalanced, where most intermediate answers from strong reasoning models are correct (see Table 4 in Appendix A.1 for detailed label statistics), use weighted binary cross-entropy loss:
= o (ReLU(e,,, + b,) + b2)
1 C(W, -- N Σ tway; p: + (1 yı) log(1 - pı))
(1)
where o is the sigmoid function, w is the ratio of negative to positive samples in the training data, and a is a hyperparameter to scale the imbalance weight. The model parameters are
W1 E Raxd W2 E Rdx1, b E RA, and by E R, where m is the hidden size of the language model and is the hidden size of the MLP.
Experiments
We first describe the basic experimental setup ($ 4.1). Then, we explore whether information about answer correctness is encoded in reasoning models (S 4.2) and if it generalizes across datasets (S 4.3), how such information is related to long CoTl reasoning abilities (S 4.4), and is the information also well-encoded even before explicit answer is formulated (5 4.5).
4.1 Experimental setup
Task datasets. We select mathematical reasoning and logical reasoning tasks as their answers are automatically verifiable. For mathematical reasoning, we use three datasets: GSM8K (Cobbe et al., 2021), MATH (Hendrycks et al., 2021), and AIME. For logical reasoning, we use Logw (Zhan et al., 2025), logical reasoning benchmark of 5.4k multiple-choice questions synthesized with knowledge-driven methods. For expert-level scientific QA, we use GPQA (Rein et al., 2023). To ensure the reliability of intermediate answer extraction, filter the KnowLogic dataset to only retain examples with single correct answer. For ease of training, all training sets are down-sampled to include no more than 1000 examples, which did not affect performance according our pilot experiment. See Appendix A.1 for more details regarding data processing.
Reasoning models. We use the open-source DeepSeek-R1-Dise series mod- els (DeepSeek-Al et al., 2025), including R1-Distill-Llama-BB R1-Distill-Llama-700 R1- Distill-Qwen-1.5BB 11-Distill-Qwen--BB and Rt--Ditill--wen-333 supervised fine-tuned with reasoning data generated by DeepSeek-RR R1 model. We also use QwQ-32B (Team, 2025; Yang et al., 2024), an open-source reasoning language model trained with reinforcement learning.
All the distilled models
Implementation details. For probing data collection, we enumerate each combination of task dataset and model to collect model representation and answer labels. The statistics of the collected data can be found in Appendix A.1. For training, each dataset D is randomly split into a training set and a validation set Drrain and Duat, with a train-to-validation ratio of 8:2. The Adam optimizer (Kingma & Ba, 2017) is used for training, and perform grid search for hyperparameter tuning. The hyperparameters for search include learning rate, scaling factor for imbalance weight a, weight decay, and ML.P hidden size d. Each
4

---

Published as a conference paper at COLM 2025
1.00
D1-DistillLllamaaae
ROCALC
0.75
HA-Diistil-Qwen--
Qwen-7B
0.50
RI Distill Qwen-
RI-DistillLLlmma-78
0.25
QwQ-328
0.00
GSM8K
MATH
AIME
KnowLogic
GPQA
Figure 2: ROC-AUC scores for each probe trained on hidden states from different reasoning models and datasets. We train separate probe on each probing dataset and evaluate it on in-distribution test set.
model is trained for at most 200 epochs with a batch size of 64; the validation loss is used as the criterion for early stopping. Following grid search, the probing models are first ranked based their validation accuracy. From the top 10 performing models, we select the probe with the least number of parameters, specifically the model with the smallest hidden dimension d. Details regarding the grid search setting and search results for each probing dataset can found in Appendix A.4. Note that most resulting models achieve non-trivial performance when d (see Appendix A.4), which means that correctness of the intermediate answer can be easily extracted with a linear probe.
4.2 Reasoning models encode answer correctness
We first test in-distribution performance of trained probes by evaluating each probe the test set from the same dataset as the training set. Figure 2 reports the ROC-AUC scores each dataset, and Table presents the corresponding Expected Calibration Error (ECE) (Naeini et al., 2015) and Brier score (Brier, 1950). Other metrics including accuracy, precision, recall, and macro F1 are reported in Appendix A.6.
Overall, all probes perform satisfactorily in in-distribution setting, achieving ROC-AUC scores above 0.7 and remarkably low Expected Calibration Error (ECE) scores below 0.1. This indicates the reasoning models inherently encode information about answer correctness that be extracted with simple probe. Moreover, many of the probes converge to linear probe after grid search (hidden size = 0), suggesting that correctness information is linearly encoded the hidden states of the reasoning model (Appendix A.4).
Across task datasets, probes trained on mathematical reasoning data perform better than those trained on logical reasoning data. This may correlate with the training data distributions of the reasoning models, where math problems presumably play a larger role. Meanwhile, probes extracted from larger reasoning models work better, with R1-Distill-Qwen- achieving over 0.9 ROC-AUC score on AIME. The Qwen family models' representations also exhibit stronger correctness signals, with Qwen- generally surpassing Llama-8B model in the mathematical domain, potentially reflecting differences in the base model training data distribution.
Reasoning Model
GSM8K
MATH
AIME
KnowLogic
GPQA
ECE 1
Brier
ECE J
Brier
ECE 1
Brier
ECE
Brier
ECE
Brier
Distill-Llama
0.05
0.09
0.19
R1 RI-Distill-Ilama Jalma-00
0.03
0.07
0.07
0.10
0.10
0.18
0.03
0.19
0.03
0.18
R1-Distill-Qwee
0.04
0.16
0.04
0.12
0.14
0.12
0.09
0.20
0.05
0.14
Di-Ditill-Qwen--2
0.02
0.11
0.03
0.10
0.09
0.15
0.06
0.21
0.05
0.18
R1-Distill-- Distilll-Qwen-22 Qwen-32B
QwQ-32B
0.03
0.13
0.13
0.10
0.08
0.13
0.03
0.15
0.09
0.20
Table Expected Calibration Error (ECE) and Brier score for the in-distribution periifroatioonpeffrrmnccc of each probe trained on each probing dataset.

---

Published as a conference paper at COLM 2025
4.3 Probes generalize to some out-of-distribution datasets
Past studies have shown that probe performance can deteriorate significantly when applied to out-of-distribution data (Belinkov, 2021; Kapoor et al., 2024). Since strong in-distribution results may not necessarily indicate reliable generalization, we examine how well the probes trained in $ 4.2 perform across different domains and datasets.
Table shows the ROC-AUC and ECE scores for probes evaluated on outtof-distributtio data, compared to those trained and tested on in-distribution data, using representations from Ri-Distill-Llama-B1 We find that probes exhibit generalizability across mathematical reasoning datasets. The probes trained MATH and GSM8K transfer well between the two datasets, demonstrating both high discriminative performance (ROC-AUC) and satisfactory calibration (ECE). In contrast, for AIME, more difficult dataset, the probes trained on GSM8K and MATH are less calibrated. However, the probe does not stably generalize to out-of-domain data (e.g,, from logical reasoning to mathematical reasoning), perhaps due to the difference in distribution of the two domains (Figure 6). More generalization results on other reasoning models can be found in Appendix A.6.
Training Data
GSMSK
MATH
AIME
KnowLogic
AUC
ECE 4
AUC
ECE 4
AUC
ECE L
AUC 1
ECE 1
0.80
0.08
0.69
0.25
0.56
0.10
GSM8K
0.82
0.05
(-0.04)
(+0.05)
(-0.11)
(+0.15)
(-0.11)
(+0.03)
0.83
0.04
0.76
0.28
0.63
0.08
MATH
0.84
0.03
(+0.01)
(40.01)
(-0.04)
(+0.18)
(40.04)
(+0.01)
0.77
0.17
0.74
0.19
0.81
0.31
Koowloggic (-0.05)
0.67
0.07
(+0.12)
(-0.10)
(+0.16)
(+0.01)
(+0.21)
Table 2: ROC-AUC scores and ECE of trained probes on out-of-distribution test set. The numbers in red and green denote performance decrease and increase relative the probe trained on in-distribution training set, respectively. R1-Distill- is used as the reasoning model.
4.4 Encoding of correctness is related to long CoT reasoning abilities
We have shown information on answer correctness is encoded in reasoning model's hidden states; to what extent this encoding is related to the model's ability to perform long CoT reasoning? To that end, we train a probe with the non-reasoning counterpart of the reasoning model. Specifically, representations of reasoning chunks using the MATH dataset. As instruct models do not have long CoT reasoning abilities, each chunk is just the full model output for one problem (i.e., including the short CoT and final answer), and the representation is simply the hidden state of the last token for each problem output. To account for this, we add an additional setting for reasoning model probes, where the probe is evaluated on the correctness of the final answers (rather than the intermediate answers) of each reasoning chain.
use 1annaa3.--B-IInttteee
(Grattafiori & Others, 2024) to obtain
As shown in Figure 3, the probe trained on non-reasoning model representations performs much worse than its reasoning counterpart, with lower classification scores and higher calibration errors. The fact that the encoded information on answer correctness is more prominent in reasoning models may suggest that the self-verification ability enhanced during long CoT supervised training. The enhanced correctness encoding might have to do with training data difference. Non-reasoning models are trained with question-responee pairs that contain single correct answer, whereas reasoning models usually undergo long CoT supervision which contains both correct and incorrect intermediate answers, often with self-correction or backtracking behaviors following the incorrect ones. This exposure may enable reasoning models to implicitly leam the correctness of intermediate answers (which is useful in predicting the reasoning chain), even without explicit supervised signals.

---

Published as a conference paper at COLM 2025
1.00
0.92
0.90
0.85
0.86
0.82
Reasoning model (Intermediate)
0.75
0.66
Reasoning, model (Final)
Scores
Non-reasoning model (Final)
0.50
0.25
0.23
0.14
0.03 0.03
0.06
0.00
Accuracy
ROC-AUC
ECE
Brier Score
Comparison on the performance on reasoning models (i.e., RI-Distill-Llam--B fine-tuned on the base Llama-3.1 model using long CoT data) and non-reasoning models (i.e., 1aama-3.1-8B-Isstii on predicting the correctness of intermediate answers (blue) and the final answers (green).
Figure 3:
on MATH. For reasoning models, we show both the performance
For non-reasoning models, the data only contains the final answers (red).
4.5 Correctness can be detected before the answer is generated
Section 4.2 shows that the hidden states at the end of reasoning chunks encode information about intermediate answer correctness, we now investigate a further question: do hidden states from earlier positions within the chunk also encode such signals? Specifically, we analyze hidden states from varying positions midwway through reasoning chunk-before an intermediate answer is fully generated-to determine these earlier representations already encode predictive signals about the forthcoming answer's correctness.
As described in $ 3.1, each reasoning trace is initially split into k chunks with correspond- ing correctness labels (1), (62, (c2, (C, yz)) Each chunk c; can subdivided into paragraphs. We obtain the representation of each paragraph-level sequence, and assign each sequence within chunk c; the label 1/i, corresponding to the correctness of the nearest upcoming intermediate answer. We train probe to predict the future answer correctness for R1-Distill-Llama- on MATH (following $ 3.2). We use hidden states at the end of dif- ferent paragraphs to predict chunk correctness. We report probing performance at different percentages of all paragraphs within a chunk.
We observe that the reasoning model's hidden states encode information about correctness even before an intermediate answer has been explicitly generated. Moreover, the probe performance is positively correlated with the paragraph's proximity to the upcoming intermediate answer. As shown in Figure 4, the probe's classification accuracy improves primarily during two critical phases: an initial steep increase in the 0-10% range, followed by minimal gains until second noticeable improvement near the chunk's end (90-100%). Compared to the peak accuracy of 79%, performance at the 10%, 50%, and 95% positions shows decrements of 14%, 10%, and 5% respectively. This highlights that early positions contain significant correctness signals, while the most predictive information emerges just before answer generation. On the other hand, calibration error is highest at the initial paragraph and then undergoes a sharp decline. ECE reaches its minimum (0.03) relatively early- around the 60% dosition-whhte the Brier score continues improving until the final positions of the reasoning chunk.
Probe as a verifier for early-exit
While reasoning models are able to encode well-calibrated and accurate information about intermediate answer correctness, do they fully utilize it during inference? We investigate this by checking whether early exiting based on the probe's confidence score on answer correctness can improve reasoning efficiency. This approach allows us to determine whether models continue reasoning unnecessarily after the probe is highly confident that the answer is correct (i.e., overthinking).
7

---

Published as a conference paper at COLM 2025
0:15
ECE
0.25
0.85
ROC-AUC
0.10
Beker Scare
0.75
Accuracy
0.20
0.65
0.55
20%
Peroentage (middle reasoning chunk position)
Percentage (middle reasoming chunk position)
(a) ECE and Brier Score decrease as the paragraph position approaches the answer at the end of the reasoning chunk
(b) Accuracy and ROC-AUC increase as the para- graph position approaches the generated answer at the end of the reasoning chunk
Figure Performance on predicting the correctness of the upcoming intermediate answers midway through reasoning chunk. The results are obtained at different percentages of all paragraphs within each chunk. The task dataset and reasoning model used are MATH dataset and RI-Distil--Llam---
5.1 Experimental setup
Following $ 3, we obtain classifier that takes reasoning chunk c/'s representation e; as input and outputs the probability p; of the intermediate answer y; being correct. Since the estimated pi highly calibrated (S 4.2), we directly use it to guide confidence-based early-exit during inference. Specifically, we first set threshold Thr for model confidence. Then, we sequentially evaluate each intermediate answer in the full reasoning trace, using the probe to compute confidence scores the answer's 's correctness. Once encounter an intermediate answer whose probed pj exceeds the threshold Thr, we truncate the reasoning trace at this chunk and take the intermediate answer the final answer.
We compare the intermediate answer selected by early exiting with the question's ground- truth answer to compute accuracy. Additionally, we record the inference token length at the point of truncation to evaluate computational efficiency. We run R1-Distill-Llama-- MATH dataset. In this experiment, the maximum token generation limit is set to be 10K across all test examples.
on
For comparison, we implement static early-exit, where we predetermine fixed number of intermediate answers !! and terminate the reasoning process after m chunks, taking the m-th chunk's ss intermediate answer Ym as the final answer?.
We also compare the probed confidence score with verbalized confidence (details in A.5), where we prompt the same reasoning model to output its evaluation of the current answer for each chunk.
5.2 Results
As shown in Figure 5, using the probe perform confidence-based early exiting can improve reasoning efficiency without accuracy degradation. When setting Thr to 0.85, our strategy achieves roughly the same reasoning accuracy (88.2%) as no early-exit, while reducing the number of generated tokens by approximately 24%. Setting Thr to 0.9 (or higher) can achieve identical reasoning accuracy (88.6%) as no early-exit and reduces the number of generated tokens by 19%. In other words, without early exiting, the reasoning model continues to generate excess tokens even when the probe indicates high confidence; this failure to fully utilize internal information answer correctness empirically leads to overthinking behavior.
Additionally, when saving equivalent numbers of tokens, our approach outperforms the static exit strategy by achieving up to a 5% accuracy improvement. For instance,
Note that the static early-exit strategy degrades to no early-exit if the total number of chunks k <
8

---

Published as a conference paper at COLM 2025
0.90
The 0.95
The: 0.98
The 0.8
m 10
0.85
mt:
Accuracy
0.80
Thr 0.6
Thr 0.5
Thr 0.4
Confidence- baaadence-based early-exit
0.75
Thr: 0.2
Thr
Static early-exit
No early-exit
0.6
1.2
1.4
1.6
Total Tokens Length
1e6
Figure 5: Final answer accuracy versus inference token cost with different early-exit strate- gies. For confidence-based early-exit, the curve is obtained by varying the confidence threshold for answer correctness. For static early-exit, the curve is generated by varying the chunk number m!
confidence-basee early exiting has 87.4% accuracy (Thr = 0.8), whereas the static early-exit strategy has approximately 82.5% accuracy with similar total token usage. Controlling for the same accuracy score (e-g- above 85%), confidence-based early-exit strategy (Thr 0.8)
consumes significantly fewer tokens than static strategy (with = 6). This demonstrates that leveraging the internal encoded information of answer correctness as an exit strategy can lead to more efficient reasoning.
Figure 7 and Figure 8 in S A.5 show that the probed confidence outperforms the verbalized confidence score in terms of both accuracy and token efficiency.
Overall, the improvements suggest that reasoning models fail to fully leverage this internal encoded information of answer correctness during inference, and that more effective usage of the information can reduce overthinking and enhance reasoning efficiency.
Discussion
In this study, we explore the existence of answer correctness information in reasoning models' inner representation. With probing, we show that such information is readily accessible in models' hidden states. The trained probe demonstrate strong calibration performance, and can be adopted as lightweight verifier to improve reasoning efficiency. The significant reduction in inference tokens suggest that reasoning models' hidden states probably contain rich information that are underexplored. Our findings contribute to the growing body of research on model interpretability and open up several intriguing avenues for future investigation.
Self-verification ability of language models. Our study reveals that answer correctness is encoded in reasoning models' hidden states. The information can be easily extracted with a probe and used abilities can be elicited from reasoning models. Notably, these abilities are less pronounced in non-reasoning models. However, given the intricate training processes and the diversity of training data these models are exposed to, the precise origins of this ability remains unclear, suggesting a promising avenue for future research into how and when such self-veriffiation abilities emerge during model training.
verifier during inference. This indicates that strong self-verification
Internal mechanisms of reasoning models. We uncover surprisingly well-calibrated hid- den verifier that enables models to autonomously assess intermediate reasoning correctness. This finding suggests that models possess an ability to self-verify, which is an important step toward understanding their internal decision-making processes. However, we still

---

Published as a conference paper at COLM 2025
observe "overthinking" phenomenon, where models perform unnecessary re-checks even after generating correct answers with high confidence, as demonstrated our early-exit experiment. This suggests that while models can self-verify, they do not yet efficiently leverage this intrinsic capability. Further study is needed to explore how reasoning models internally utilize the information encoded in their representations, and how can guide them to use this information more efficiently during training or inference.
On-policy control of reasoning models. In contrast previous LLM-based verifiers (Zhang et al., 2025; Cobbe et al., 2021; Zhang et al., 2024), the hidden verifier extracted in our work much more lightweight. Our approach leverages the hidden states of the reasoning model directly during inference, which not only improves token efficiency but also makes the verifier more integrated with the model's existing architecture. Our finding highlights the potential of an on-policy perspective in model inference control. We believe this opens new avenues for future research in designing more efficient and adaptive control modules for reasoning models.
In summary, our study highlights the encoded answer correctness information reasoning models, indicating the latent capability of reasoning models to verify their own answers. Leveraging this information through lightweight probing techniques, we show reasoning efficiency can be further enhanced, implying an inadequate use of the information by reasoning models during inference. Our findings underscore the potential of on-policy control for reasoning models, offering a novel direction for more efficient and adaptive inference strategies. Future research should further investigate the origins of the self- verification abilities and develop methods to better harness them, ultimately improving reasoning efficiency and reliability.
References
Amos Azaria and Tom Mitchell. The internal state of an Ilm knows when it's lying, 2023. URL https://arxix wwtrrxivv.org/aae wwt.oogg//////// arggabs/2304. 13734.
Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, Carol Chen, Catherine Olsson, Christopher Olah, Danny Hernandez, Dawn Drain, Deep Ganguli, Dustin Li, Eli Tran-Johnson, Ethan Perez, Jamie Kerr, Jared Mueller, Jeffrey Ladish, Joshua Landau, Kamal Ndousse, Kamile Lukosuite, Liane Lovitt, Michael Sellitto, Nelson Elhage, Nicholas Schiefer, Noemi Mercado, Nova DasSarma, Robert Lasenby, Robin Larson, Sam Ringer, Scott Johnston, Shauna Kravec, Sheer El Showk, Stanislav Fort, Tamera Lanham, Timothy Telleen-Lawton Tom Conerly, Tom Henighan, Tristan Hume, Samuel R. Bowman, Zac Hatfield-Dodds, Ben Mann, Dario Amodei, Nicholas Joseph, Sam McCandlish, Tom Brown, and Jared Kaplan. Constitutional ai: Harmlessness from ai feedback, 2022. URL https://arxtv.orr/bbb//// 08073.
Yonatan Belinkov. Probing classifiers: Promises, shortcomings, and advances, 2021. URL https://arriv..or//a///11... 12452.
Glenn W Brier. Verification of forecasts expressed in terms of probability. Monthly weather review, 78(1):1-3, 1950.
Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering latent knowledge in language models without supervision, 2024. URL http:///axxxv...//bbbbss// 03827.
Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Quuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Do not think that much for 2+3=? on the overthinking of ol-like Ilms, 2025. URL https://arxiv.. wwwpriiiv......eeeeeeeoeeeoo argg/abs/2112.111111
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichio Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021. URL http://aaxxi..ooo///////// 14168.
10

---

Published as a conference paper at COLM 2025
Antonia Creswell and Murray Shanahan. Faithful reasoning using large language models, 2022. URL https: http://www...orrb///////////// 14271.
Mehul Damani, Idan Shenfeld, Andi Peng, Andreea Bobu, and Jacob Andreas. Learning how hard to think: Input-adaptive allocation of Im computation, 2024. URL https: //arxiv. hwwp:/orggbb//////// 04707.
DeepMind. Ai solves imo problems at silver medal level, 2024. UtLhttss: boogle/discover/bbeee ai solves- imo peto--probeesss-eee -t--emsssstteete at- wwwwssileewlelleeeeeeeeeeeeeeeee 2025-03-24.
Accessed:
DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjee Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guannbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang, Jingchang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang, Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li, Miaojun Wang, Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li, Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, T. Wang, Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang, Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, 2. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. URL Incentivizing reasoning capability in llms via reinforcement learning, 2025. 12948.
Deepseek-rl:
Jiazhan Feng, Ruochen Xu, Junheng Hao, Hiteshi Sharma, Yelong Shen, Dongyan Zhao, and Weizhu Chen. Language models can be logical solvers, 2023. URL httptttss///axvv.oom
0as//211..61155
Yichao Fu, Junda Chen, Siqi Zhu, Zheyu Fu, Zhongdongming Dai, Aurick Qiao, and Hao Zhang. Efficiently serving IIm reasoning programs with certaindex, 2024. URL https://arxiv..or//altt
20993.
Gemini-Team. Gemini: A family of highly capable multimodal models, 2024. URL https: //arxiv.org/abs//311.. 11805.
Aaron Grattafiori and Others. The llama 3 herd of models, 2024. URL https:///rxxv..org/ abs/2407.21783.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problle solving with the math dataset, 2021. URL https://arxxvv
http:////aaxxxv.o////////// orgg/bs////3. 03874.
11

---

Published as a conference paper at COLM 2025
Saurav Kadavath, Tom Conerly, Amanda Askell, Tom Henighan, Dawn Drain, Ethan Perez, Nicholas Schiefer, Zac HatfieldDoodss, Nova DasSarma, Eli Tran-Johnson, Scott Johnston, Sheer El-Showk, Andy Jones, Nelson Elhage, Tristan Hume, Anna Chen, Yuntao Bai, Sam Bowman, Stanislav Fort, Deep Ganguli, Danny Hernandez, Josh Jacobson, Jackson Kernion, Shauna Kravec, Liane Lovitt, Kamal Ndousse, Catherine Olsson, Sam Ringer, Dario Amodei, Tom Brown, Jack Clark, Nicholas Joseph, Ben Mann, Sam McCandlish, Chris Olah, and Jared Kaplan. Language models (mostly) know what they know, 2022. / http/v.xvvvvooo//ee org. /t/p//bbb/222222 05221.
URL https:
Sanyam Kapoor, Nate Gruver, Manley Roberts, Katherine Collins, Arka Pal, Umang Bhatt, Adrian Weller, Samuel Dooley, Micah Goldblum, and Andrew Gordon Wilson. Large language models must be taught to know what they don't know, 2024. URL https: //arxiv.org abs/2406. 08391.
Diederik P. Kingma and Jimmy Ba. Adam: method for stochastic optimization, 2017. URL nttps://arxiv.orge
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention, 2023. URL 06180.
Long Hei Matthew Lam, Ramya Keerthy Thatikonda, and Ehsan Shareghi. A closer look at logical reasoning with 1lms: The choice of tool matters, 2024. URL https::/arxxi...ee abs/2406. 00284..
Yiwei Li, Peiwen Yuan, Shaoxiong Feng, Boyuan Pan, Xinglin Wang, Bin Sun, Heda Wang, and Kan Li. Escape sky-high sky-hiig cost: Early-stopping self-consistency for multi-step reasoning, 2024a. URL httpts///arxxi wwwpaaaiiiv.ooag////e//eeee/seee arrg/abb//200.11 10480.
Zhaoyi Li, Gangwei Jiang, Hong Xie, Linqi Song, Defu Lian, and Ying Wei. Under- standing and patching compositional reasoning in LLMs. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Lin- guistics: ACL 2024, pp. 9668-9688, Bangkok, Thailand, August 2024b. Association for Computational Linguistics. doi: 10.18653/v1 /22411/222.22222 2222..indinns--ccl57766 URL https: //aclanthology org/2024. findings-acl. 576/.
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step, 2023. URL //arxiv.///ii/////// UUR.:tttp///axvvvorr///////2/ 20050.
Stephanie Lin, Jacob Hilton, and Owain Evans. Teaching models to express their uncertainty in words, 2022. URL https:///axiv http:///aaxii.oorraa///2ee. orggabs//2055 14334.
Zhan Ling, Yunhao Fang, Xuanlin Li, Zhiao Huang, Mingu Lee, Roland Memisevic, and Hao Su. Deductive verification of chain-of-thought reasoning, 2023. URL httpp::/arxe org/abs/ 2306. 03872.
Hanmeng Liu, Zhizhang Fu, Mengru Ding, Ruoxi Ning, Chaoli Zhang, Xiaozhang Liu, and Yue Zhang. Logical reasoning in large language models: A survey, 2025, URL https://arxiw.org/abb///2/
09100.
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegr- effe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bod- hisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refinement with self-feedback, 2023. URL
https://arriv..or/aa//23333
www.:/o/g/bbb////////// 17651.
Rohin Manvi, Anikait Singh, and Stefano Ermon. Adaptive inference-time compute: Lims can predict if they can do better, even mid-generation, 2024. URL https: //tpxxx.xxxvo/////// abs/2410. 02725. 227.2.222....
12

---

Published as a conference paper at COLM 2025
Sabrina J. Mielke, Arthur Szlam, Emily Dinan, and Y-Lan Boureau. Reducing conversational agents' overconfidence through linguistic calibration. Transactions of the Association for Computational Linguistics, 10:857-872, 2022. doi: 10.1162 tacl a 00494. URL https: // aclanthology. wttp:e....rg///// 2022. ta01-1.50/.
Tergel Munkhbat, Namgyu Ho, Seo Hyun Kim, Yongjnn Yang, Yujin Kim, and Se-Young Yun. Self-training elicits concise reasoning in large language models, 2025. URL https: www.iii.org/ab////// 20122.
Mahdi Pakdaman Naeini, Gregory Cooper, and Milos Hauskrecht. Obtaining well calibrated probabilities using bayesian binning. In Proceedings of the AAAI conference on artificial intelligence, volume 29, 2015.
OpenAl. Openai ol system card, 2024. URL https://arrdi..og//bb///111
Hadas Orgad, Michael Toker, Zorik Gekhman, Roi Reichart, Idan Szpektor, Hadas Kotek, and Yonatan Belinkov. Llms know more than they show: On the intrinsic representation of Ilm hallucinations. Xiv preprint ar a<Xto:2410.02707, 2024.
Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beatriz Borges, Antoine Bosselut, Robert West, and Boi Faltings. Refiner: Reasoning feedback intermediate representations, 2024. URL https: hoops://arge /awp/vaaavvv///a////a///////aaa/ orggabs//304. 01904.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. Gpqai A graduate-level google- proof q&a benchmark, 2023. URL https::/arxist http://axxvvoorgrrbbrrs/a/a org/abs/2311 12022.
Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning, 2023. URL https::/arxiv.org/bbb////
1666.
Yang Sui, Yu-Neng Chuang, Guanchu Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, and Xia Hu. Stop overthinking: A survey on efficient reasoning for large language models, 2025. URL https: abs/2503. 16419.
Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URL https://qwenlmm gi thub. hoobllo//wqq-3/0
Katherine Tian, Eric Mitchell, Allan Zhou, Archit Sharma, Rafael Rafailov, Huaxiu Yao, Chelsea Finn, and Christopher D. Manning. Just ask for calibration: Strategies for eliciting calibrated confidence scores from language models fine-tuned with human feedback, 2023. URL https://arxiv. org/abs/23053 14975.
Xinglin Wang, Shaoxiong Feng, Yiwei Li, Peiwen Yuan, Yueqi Zhang, Chuyi Tan, Boyuan Pan, Yao Hu, and Kan Li. Make every penny count: Difficulty-daaptiee self-consistency for cost-efficient reasoning, 2025. URL https: org abs////bs/// 13.47....44.. 13457.
Yixuan Weng, Minjun Zhu, Fei Xia, Bin Li, Shizhu He, Shengping Liu, Bin Sun, Kang Liu, and Jun Zhao. Large language models are better reasoners with self-verification, 2023. URL htttttsss///riii www.oogg///b//ee
09561.
Zhihui Xie, Jizhou Guo, Tong Yu, and Shuai Li. Calibrating reasoning in language models with internal consistency. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL httpp::/ http:////oooeeeeeeeeeeeeeeerrroo tttt/foorummddddudxWWFFFFF
Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. Can Iims express their uncertainty? an empirical evaluation of confidence elicitation llms, 2024. URL https: wtt.v/ororr/////////// 13863.
13

---

Published as a conference paper at COLM 2025
Mayi Xu, Yongqi Li, Ke Sun, and Tieyun Qian. Adaptio--ofthhought Learning question difficulty improves large language models for reasoning. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference Empirical Methods in Natural Language Processing, pp. 5468-548 5495, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10. (0.18653/vv //066//11/2...mmmmmmm..... Smnnp-main.3.. 313/.
URL
https://aclanthollgg.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. Xiv preprint Xrio:2412..55115, 2024.
Dongkeun Yoon, Seungone Kim, Sohee Yang, Sunkyoung Kim, Soyeon Kim, Yongil Kim, Eunbi Choi, Yireun Kim, and Minjoon Seo. Reasoning models better express their confi- dence, 2025. URL https::/aart /arxiv. www.aaawwwwww..///...//a..o.a.ae orggabs//50g/ 14489.
Weidong Zhan, Yue Wang, Nan Hu, Liming Xiao, Jingyuan Ma, Yuhang Qin, Zheng Li, Yixin Yang, Sirui Deng, Jinkun Ding, Wenhan Ma, Rui Li, Weilin Luo, Qun Liu, and Zhifang Sui. Knowlogic: A benchmark for commonsense reasoning via knowledge-driven data synthesis, 2025. URL https: wrwpi///..r//bb/2233... 06218.
Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction, 2025. URL
https://arxiv. wttp://o.wg///b////////// 15240.
Yunxiang Zhang, Muhammad Khalifa, Lajanugen Logeswaran, Jaekyeom Kim, Moontae Lee, Honglak Lee, and Lu Wang. Small language models need strong verifiers to self- correct reasoning, 2024. URL https: //arxiv http://aaw.v.ooab//////eaa/ org/abs//444/ 17140.
Zirui Zhao, Hanze Dong, Amrita Saha, Caiming Xiong, and Doyen Sahoo. Automatic curriculum expert iteration for reliable Ilm reasoning, 2025. URL https://arxiv.. 0ts/22410.07227
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and lon Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena. In Proceedings of the 37th International Conference on Neural Information Processing Systems, 2023.
Aojun Zhou, Ke Wang, Zimu Lu, Weikang Shi, Sichun Luo, Zipeng Qin, Shaoqing Lu, Anya Jia, Linqi Song, Mingjie Zhan, and Hongsheng Li. Solving challenging math word problems using gpt-4 code interpreter with code-based self-verification, 2023. URL https://arxiv htte:///axxi.oor//ab/23000. 07921.
Tinghui Zhu, Kai Zhang, Jian Xie, and Yu Su. Deductive beam search: Decoding deducible rationale for chain-of-thougee reasoning. In First Conference on Language Modeling, 2024. URL
14

---

Published as a conference paper at COLM 2025
A
Additional details
A.1 Data collection details
Table shows the keywords we use to identify beginning of new reasoning paths to help segmenting reasoning trace into chunks.
Keywords for chunk segmentation
"wait", "double-check", "alternatively", "make sure", "another way", "verify", "to confirm"
For AIME, we use AIME-1983.20 2024 3 for training and AIME 2025 for testing. For MATH, we use the original training set and the 500-example test set released by Hug- gingFace For Know Logic dataset, we ran- domly split the dataset into training and test set by 80% and 20%, and collect probing data separately. For GPQA dataset 6 we use the
Table 3: Keywords we use for identifying rea- soning path switch and segmenting reasoning trace into chunks.
subset as test data and the rest of Geteeext as training data.
Table 4 shows the statistics of collected chunks for each dataset. We use vLLM (Kwon et al., 2023) for inference and set maximum output length to 30K. Examples whose model completion goes over the maximum output length are discarded.
Figure 6 a visualization chunk represen- tations obtained for different datasets with RI-Distill-Llama-8B (DeepSeek-Al et al., 2025). The domain difference between log- ical reasoning and mathematical problems is evident.
AIME
GSM8K
KnowLogic
MATH
A.2 Prompts
Table 5 shows the prompt we used to elicit reasoning trace from all reasoning models. Note that for Qwen models, the prompt we use is slightly different from its original prompt. We observe the performance benchmark does degrade little but within reasonable range. To ensure the extracted feature is on-policy, we also keep the same prompt when extracting representations for each reasoning chunk.
the
Figure 6: T-SNE visualization of chunk repre- sentations for different datasets. 1000 chunks are randomly sampled from each training set and R1-Distill-Llama-- is used to obtain the representation.
Table is the evaluation prompt we use for Gemini 2.0 Flash (Gemini- Team, 2024) for answer extraction and evaluation based on given reasoning chunks.
hhtts://huugiingface.oo http:/ggiiingaacc..//aatttttttse datasetsddizhanggddu/AMMM
"nttps:/hhuggiigffac.co datasets /atnnssts/yenninglinnnaaaa 2025
"httss//hhuggingfaac.c. /dttasett /ttage/////ggggggffaaaa/aa/AA///
Phtps://huugingfface.oo /atp://eeeeeeetttettee////vneene gpqa
15

---

Published as a conference paper at COLM 2025
#Train Examples
#Test Examples
#Train Chunks
# #Test Chunks
Avg. Chunk Len.
Positive Chunks (%)
Reasoning Model
GSMBK
RI-Distill Llamil---
1000
1317
7379
11228
328.0
70.97
R1-Distill-Llame -lamma70B
998
1318
9030
6116
272.4
84.36
R1-Distill-Qwent
1308
8599
11730
379.0
63.57
KI-Distill-Owen=
1000
1316
5615
7568
302.8
75.87
R1-Distill 2t-Distill-Qwen-322 QwQ-32B
1317
4393
6381
293.1
84.25
1000
1319
4786
6302
338.2
82.78
MATH
RI-Distill-Llam--S1
1000
491
6259
3380
615.1
76.91
-Lamaa70BB
996
499
4865
2559
701.5
82.88
RI-Distill
PDDisill-Qwen--le -Wwen-1.5B
988
495
7388
4089
996.7
68.46
R1-Distill -wattll---ee--BB
983
494
5062
2764
713.3
79.77
I1-Distill-wwen--20
991
497
4732
2460
678.5
84.40
QwQ-32B
1000
500
5619
2826
795.0
86.12
AIME
R1-Distill PenttillLllam---- Llama-8B
922
30
7158
323
1652.0
35.24
R1-Distill -laaml-0-
923
30
5443
318
1528.0
50.78
R1-Distill- -wenn-.5BBBB
892
29
8358
314
1809.4
26.33
R1-Distill Qwen-7B
RI-Distill-Qwen-77
922
29
3501
179
1841.8
42.50
R1-Distill-Qwen-B0
868
25
4181
104
1244.1
55.03
QwQ-32B
933
30
5037
142
2079.1
68.14
Knowloogic
R1-Distill- RI-Distil-Llama-B8
Llama-8B
986
320
7620
2596
1079.6
44.27
R1-Distill-L -l0m
297
6529
2000
639.7
57.71
RI-Distill-Qwent -wel---5eenBB
245
6879
2036
1070.0
20.56
R1-Distill Seetill-ween--B
938
306
7169
2430
1072.7
42.25
R1-Distill- Seatill-Qwen-321
315
6131
1827
818.8
57.40
QwQ-32B
1000
326
6256
2093
1043.3
70.78
GPQA
R1-Distill-Llan
Satiil--Llma--B3
348
198
2891
1662
794.6
38.89
-00
348
198
1477
1151
737.5
51.46
R1-Distill 101----een1155
348
198
2618
1745
803.9
17.61
I-Distiil-Qween
348
198
2547
1601
852.2
33.73
R1-Distill R1-Distill-Qwen-33 Qwen-32B
348
198
1796
1188
747.0
41.76
QwQ-32B
348
198
1477
1220
1067.7
45.09
Table 4: Statistics for obtained probing dataset across task datasets and reasoning models. The inconsistency in training examples and test examples number comes from discard of examples with truncated model completion. The average chunk length is calculated by sampling 1000 chunks from each training dataset and measured by number of tokens. The positive chunk ratio is calculated based on the training set.
Inference Prompt
<BOS_TOKENT 
 (instruction)
Please reason step by step, and put your final answer within \boxed().
Assistant|>>
Table Prompt used for inference with reasoning models.
A.3 Sensitivity of LLM annotation
To test the sensitivity of LLM annotation, compare annotation results from Gemini with GPT-4o-mini with randomly sampled 100 examples from MATH training data. As shown in the Table. 7 below, the two models well agree with each other with 80% annotations
16

---

Published as a conference paper at COLM 2025
Evaluation Prompt
Given several chunks a reasoning trace, along with a ground-truth answer, independently evaluate each chunk. If a chunk reaches result at the end, return the intermediate result; otherwise, return None if the chunk does not contain an intermediate result (e.g,, pure reflections).
Then, if an intermediate answer exists, compare it to the ground-truth answer. If the intermediate result the chunk equals the ground-truth answer, return True; if the intermediate result in the chunk does not equal the ground-truth answer, return False; no intermediate answer exists, return None. Output in JSON format:
{"id": "1", "result": "6 9i" / None, "correctness": True / False None],
Input chunks: {reasoning.traas Ground-truth answer: (answer)
Table 6: Prompt used for answer extraction and evaluation with Gemini 2.0 Flash.
exactly the same. The results show that the annotation task is relatively straightforward for state-of-the-art LLMs.
CoT Data
Cohen's kappa coefficient
% consistent annotation
# segments
Swenl.5BBMMAT Qwen7B+MATT
0.58
85
3416
0.62
80
1772
Qwen22B+MATH
0.60
81
1513
Table 7: Annotation consistency and agreement for Gemini and GPT-4o-mini models.
A.4
Grid search
We perform grid search over hyparameters include learning rate, loss weight scaling factor a, weight decay for optimizer, and classifier hidden size d. The specific search range for each hyperparameter can be found in Table 8, and the resulting optimal hyperparameter settings for each probing dataset are shown Table 9.
Hyperparameter
Search Space
Learning rate Scaling factor a Weight decay Hidden size
1e-3, 1e-4, 1e-5 0.3, 0.5, 0.7, 0.9, 1.0, 1.5, 2.0, 3.0 0.001, 0.01, 0.1 0, 16, 32
Table 8: Hyperparameter search space for classifier training.
17

---

Published as a conference paper at COLM 2025
Model
Dataset
Learning rate
Loss weight
Weight decay
Hidden size
GSM8K
le-4
3.0
0.1
R1-Distill
MATH
1e-5
2.0
0.001
-LJlma-8B
AIME
1e-5
0.3
0.1
Lnoic
le-5
0.7
0.1
GPQA
le-5
0.3
0.001
GSM8K
2.0
R1-Distill
MATH
le-3
2.0
-Qwen-1.5B
AIME
le-5
0.5
0.01
16
KnowLogic
le-4
0.3
0.001
GPQA
le-5
0.3
0.001
GSM8K
le-4
3.0
0.1
RI-Distill -Qwen-7B
MATH
3.0
0.1
AIME
0,9
0.1
KnowL.gie
le-5
0.9
GPQA
le-5
0.5
0.01
GSMBK
le-3
3.0
0.001
16
R1-Distill
MATH
le-4
2.0
0.1
-wwen-32B
AIME
1e-5
1.0
0.01
16
KnowLogic
le-5
0.9
0.1
GPQA
le-5
1.5
0.01
GSM8K
le-4
2.0
0.001
R1-Distill
MATH
le4
3.0
0.001
-llam-70B
AIME
le-4
2.0
0.001
Loow
le-3
1.0
0.01
32
GPQA
le-4
0.9
0.001
GSM8K
le4
3.0
0.1
MATH
le-3
2.0
0.001
16
Q-333B
AIME
le-3
3,0
0.01
16
KnowLogic
le-4
1.5
0.1
GPQA
1e-5
0.7
0.01
Table 9: Results of grid search across reasoning models and datasets.
A.5 More results about prompting for confidence verbalization
Prompting the model to verbalize its confidence a straightforward method to test its notion of correctness. However, in real implementation, we find it extremely difficult to get reasoning models to follow user instruction. Experimenting with different prompting strategies, we only manage get the model to produce a verbalized confidence for 66% of the chunks. On these chunks, we conducted the same experiment as in $ 5 and find that probing significantly outperforms self-reported confidence both accuracy and token efficiency. Figure 7 illustrates the results.
We also use prompting with designed verbalizer. Specifically, we append "The answer is to each chunk, then obtain the probabilities of the next token being exactly "correct" or "incorrect", and re-normalize them to produce confidence score. However, the confidence obtained in this way is still less calibrated with the ECE score equal to 0.25. Using this forced self-reported confidence for early exit also performs much worse than our probe method in terms of both accuracy and token efficiency. Figure 8 illustrates the results.
A.6 Further results
Table 10 and Table 11 show in-distribution probing performance measured by accuracy, precision, recall, and macro F1 across reasoning models and datasets.
Table 12 to Table 16 show out-of-distribution probing performance trained and test on representations from RI-Distill-Qwen-11.. R1-Distill-Qwen-7B RI-Distill-Qwee--22, and QwQ-32B, respectively.
18

---

Published as a conference paper at COLM 2025
0,90
The 0.9
The: 0.95
The 0.8
Thr: 0.85
0.8
0.95
0.98
0.85
t:
Accuracy
0.80
1:0.7
0.885
t: 0.3
Thr 0.5
0.0.4
Thr:044
PU3
0.75
Thr
Confidence-based early-exit (Probe)
Model's self-reported confidence and early-exit
0.6
0.8
1.2
14
Total Tokens Length
1e6
Figure 7: Final answer accuracy versus inference token cost with different early-exit strate- gies. For confidence-based exrly-exit, the curve is obtained by varying the probe confidence threshold (Thr) for answer correctness. For model's self-reported confidence and early-exit, the curve is generated by varying the model's self-reported confidence threshold (t).
0.90
Thr:
The 0.95
Thr: 0.98
Thr: 0.85
D. 0.99
Thr: 0&8
0.5 0.85
0.9
0.85
Thr:
0.7.
Accuracy
0.66
0.80
Thr
0.5
Thr:
Thr:
t 0.3
0.75
Thr:
Thr:
Confidence-band early-exit (Probe)
Forced self reported confidence and early-exit
0.6
0.8
1.0
1.2
1.4
1.6
Total Tokens Length
1e6
Figure 8: Final answer accuracy versus inference token cost with different early-exit strate- gies. For forced self-reported confidence and early-exit, we force the model to output confidence scores, and the curve is generated by varying the forced self-reported confidence threshold (t).
GSMSK
MATH
Reasoning Model
Accuracy
Precision
Recall
Macro
Accuracy
Precision
Recall
Macro
R1-Distill-L -alma-8B
0.77
0.85
0.82
0.73
0.80
0.84
0.88
0.75
E1-Distill-Llama-s
0.91
0.92
0.97
0.82
0.89
0.92
0.93
0.83
RI-Distill-Qweet -
0.76
0.81
0.81
0.74
0.84
0.84
0.88
0.83
R1-Distill 11-DistillQween-7B
0.84
0.88
0.92
0.77
0.87
0.89
0.94
0.82
R1- -Disiill--wen---1
0.91
0.95
0.89
0.94
QwQ-32B
0.83
0.99
0.49
0.87
0.95
0.79
Table 10: Accuracy, precision, recall, and macro F1 score for probes trained and test on GSM8K and MATH datasets in in-distribution setting.
19

---

Published as a conference paper at COLM 2025
AIME
KnowLogic
Reasoning Model
Accuracy
Precision
Recall
Macro F1
Accuracy
Precision
Recall
Macro F1
R1-Distill- Jamm--8
0.85
0.37
0.38
0.64
0.62
0.62
0.41
0.60
RI-Distll-Lllmm--00
0.75
0.80
0.54
0.73
0.67
0.79
0.62
0.67
H--Disill-Qwen-ll
0.83
0.45
0.62
0.71
0.72
0.23
0.53
0.42
D-Distill-Owen--B
0.65
0.64
0.69
RI-Distil-Qween222B
0.91
0.88
0.96
0.91
0.70
0.80
0.67
0.69
QwQ-32B
0.82
0.84
0.85
0.82
0.78
0.82
0.88
0.74
Table 11: Accuracy, precision, recall, and macro Fl score for probes trained and test AIME and KnowLogic datasets in in-distribution setting.
Training Data
GSMSK
MATH
AIME
KnowLogic
AUC
ECE 4
AUC1
ECE 1
AUC
ECE
AUC
ECE |
0.90
0.07
0.75
0.14
0.62
0.08
GSM8K
0.82
0.04
(+0.06)
(+0.04)
(-0.05)
(+0.04)
(-0.05)
(+0.01)
0.82
0.10
0.84
0.18
0.63
0.14
MATH
0.84
0.03
(-0.01)
(+0.06)
(+0.04)
(+0.08)
(-0.04)
(+0.08)
0.67
0.36
0.73
0.34
0.68
0.05
Logi
0.67
0.07
(-0.16)
(+0.32)
(-0.11)
(+0.32)
(-0.12)
(-0.05)
Table 12: ROC-AUC scores and ECE of trained probes out-of-distribution test set. The numbers in red and green denote performance decrease and increase relative to the probe trained on in-distribution training set, respectively. R1-Distill-Qwen--15t is used the reasoning model.
Training Data
GSMSK
MATH
AIME
KnowLogic
AUC
ECE +
AUC
ECE +
AUC
ECE 4
AUC
ECE +
0.86
0.06
0.76
0.15
0.60
0.17
GSMSK
0.82
0.04
(+0.02)
(+0.03)
(-0.04)
(+0.05)
(-0.07)
(+0.10)
0.86
0.06
0.73
0.18
0.68
0.17
MATH
0.84
0.03
(+0.04)
(+0.02)
(-0.07)
(+0.08)
(+0.02)
(+0.10)
0.81
0.07
0.83
0.10
0.72
0.16
KnowLogic
0.67
0.07
(-0.02)
(+0.03)
(-0.01)
(+0.07)
(-0.08)
(+0.06)
Table 13: ROC-AUC scores and ECE of trained probes on out-of-distribution test The numbers in red and green denote performance decrease and increase relative to the probe trained on in-distribution training set, respectively. R1-Distill-Qwen is used as the reasoning model.
Training Data
GSMSK
MATH
AIME
KnowLogic
AUC 1
ECE 1
AUC1
ECE 4
AUC
ECE L
AUC
ECE J
0.87
0.04
0.98
0.17
0.73
0.06
GSMSK
0.82
0.04
(+0.03)
(+0.01)
(+0.17)
(+0.07)
(+0.06)
(-0.01)
0.89
0.03
0.97
0.10
0.72
0.15
MATH
0.84
0.03
(+0.06)
(-0.01)
(+0.16)
(+0.00)
(+0.05)
(+0.08)
0.83
0.09
0.89
0.10
0.91
0.22
KnowLogic
0.67
0.07
(+0.00)
(+0.05)
(+0.05)
(+0.07)
(+0.10)
(+0.12)
Table 14: ROC-AUC scores and ECE of trained probes on out-of-distribution test set. The numbers in red and green denote performance decrease and increase relative to the probe trained on in-distribution training set, respectively. RI-Distill-Qwen--22 used as the reasoning model.
20

---

Published as a conference paper at COLM 2025
Training Data
GSMSK
MATH
AIME
KnowLogic
AUC 1
ECE +
AUC 1
ECE 4
AUC f
ECE +
AUC
ECE +
0.88
0.09
0.71
0.17
0.62
0.25
GSM8K
0.82
0.04
(+0.04)
(+0.06)
(-0.09)
(+0.07)
(-0.04)
(+0.18)
0.87
0.06
0.75
0.16
0.73
0.20
MATH
0.84
0.03
(+0.05)
(+0.02)
(-0.05)
(+0.06)
(+0.06)
(+0.13)
0.84
0.10
0.87
0.13
0.70
0.12
KnowLogic
0.67
0.07
(+0.01)
(+0.06)
(+0.03)
(+0.10)
(-0.10)
(+0.02)
Table 15: ROC-AUC scores and ECE of trained probes on out-of-distribution test set. The numbers in red and green denote performance decrease and increase relative to the probe trained on in-distribution training set, respectively. R1-Distill-Llamar is used as the reasoning model.
Training Data
GSMSK
MATH
AIME
KnowLogic
AUC 1
ECE 4
AUC
ECE
AUC
ECE
AUC
ECE |
0.74
0.14
0.73
0.23
0.61
0.29
GSM8K
0.82
0.04
(-0.10)
(+0.12)
(-0.07)
(+0.13)
(-0.06)
(+0.23)
0.55
0.22
0.87
0.07
0.76
0.11
MATH
0.84
0.03
(-0.27)
(+0.18)
(+0.07)
(-0.03)
(+0.09)
(+0.04)
0.61
0.14
0.81
0.05
0.84
0.07
KnowLogic
0.67
0.07
(-0.22)
(+0.11)
(-0.03)
(+0.02)
(+0.04)
(-0.03)
Table 16: ROC-AUC scores and ECE of trained probes on out-of-distribution test set. The numbers in red and green denote performance decrease and increase relative to the probe trained on in-distribution training set, respectively. QwQ-32B is used as the reasoning model.
21


Reviews:
[{"review_id": "ImeV9SAz4o", "reviewer": "Reviewer_KxZk", "summary": "This paper studies the role of the hidden state in reasoning models in verifying the correctness of their intermediate reasoning process.\nThey find that using a simple layer of MLP can probe the correctness from the hidden states and can generalize to OOD data.\nThey conduct an early-exit experiment on MATH dataset.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. This paper shows that the hidden state in reasoning models can serve as logit lens with a single MLP.\n2. The experimental results show that the token usage can be reduced by 24% given same reasoning accuracy.", "reasons_to_reject": "1. There are existing works interpreting the hidden states in the reasoning process of the LLMs[1]. I think the differences between RMs and LLMs are not big enough to review the role of the hidden states again. I would suggest a more thorough investigation into logit lens employed in LLMs.\n2. Lack of compared baselines. There are lots of work focusing on either external verifiers or internal verifiers. For example, [2] exploits an external verifier for enhancing reasoning performance and [3] utilizes hidden states for internal consistency. At least both of them can be compared to your method regarding accuracy and token efficiency.\n\n[1] Li, Zhaoyi, et al. \"Understanding and patching compositional reasoning in llms.\" ACL 2024.\n\n[2] Zhu, Tinghui, et al. \"Deductive beam search: Decoding deducible rationale for chain-of-thought reasoning.\" COLM 2024.\n\n[3] Xie, Zhihui, et al. \"Calibrating reasoning in language models with internal consistency.\" NeurIPS 2024.", "questions_to_authors": "1. In Sec 4.4, the encoding of correctness is related to long CoT reasoning abilities. Does this suggest that using internal hidden states does not generalize to smaller/weaker models?", "extra_scores": {}}, {"review_id": "j3Nfb4lL0X", "reviewer": "Reviewer_Etou", "summary": "This paper studies early exit for reasoning model, and finds that the hidden state of the intermediate answer can show the model's confidence to an answer based on the current generation. The authors then introduce a light verifier as a probe to enable the model exit reasoning step when the confidence is high. Results show that the probe could reduce output token by 24%, which accordingly reduces the model generation cost and latency.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The idea of using hidden states to verify confidence is novel and clean;\n- The writing is clear to understand and easy to follow;", "reasons_to_reject": "- The paper could benefit from comparing prediction accuracy/efficiency with other methods. For example, for self-reported confidence, the model can generate a few tokens to estimate the current confidence at the end of each paragraph / chunk.\n- Comparing to other reasoning tasks, math has a shorter and unique answer. This may reduce the difficulty to verify the answer confidence. The paper could benefit from testing the verifier accuracy on other tasks such as coding or formal proving.\n- The authors mention that another language model is employed to extract the answer from a chunk. In real world deployment, using this method could be very costly and slow.", "questions_to_authors": "- The paper mentioned that using hidden states to evaluate confidence of non-reasoning model has a low performance. It could be more convincing to discuss the reason of such a difference.", "extra_scores": {}}, {"review_id": "cDjRpNL6YA", "reviewer": "Reviewer_5iSN", "summary": "The paper presents a method for on-policy control of reasoning language models. Specifically, the authors show that by using a simple (even linear) classifier on the hidden state it is possible to predict correctness of the intermediate answers during reasoning. The latter can be used to stop generation early without compromising the accuracy.\nThe finding is interesting and the authors provide experimental validation of their idea.", "questions": "", "limitations": "", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The author's method is simple and interesting for broad audience. \n2. The experimental validation is sufficient to support the idea. \n3. The language of the paper is clear.", "reasons_to_reject": "1. While the idea of the work is interesting, the authors do not go anywhere beyond simple empirical observation of the effect. \nNo attempt was made to explain why the hidden states of reasoning LLMs predict correctness of intermediate answers. Also, it is hard to tell how novel is the idea comparing with the existing state of the art, as hidden states of LLMs were already used to detect hallucinations and to estimate answer uncertainty in previous works, for example in https://openreview.net/forum?id=LYx4w3CAgy. The authors should clearly explain the novelty of their approach.\n\n2. Considering an empirical nature of the work, additional testing may be necessary. Specifically, the datasets used by the authors are mostly math-related. It is necessary to check the findings of the authors on non-math related datasets., for example NPHardEval. \n\n3. The procedure for data preparation requires further validation. The authors claim that reasoning content contains multiple intermediate answers. This assumption is used to segment the reasoning trace into chunks with the help of an external LLM. Further, intermediate answers are extracted with an external LLM from each chunk. It would be nice to see some validation of the assumption that full intermediate answers exist in the reasoning data, since human reasoning usually does not contain complete intermediate answers. Also, the dependence of the chunking procedure on the external LLM should be studied.", "questions_to_authors": "1. Does the effect exist when using non-math datasets? \n2. How often intermediate answers present in the reasoning traces? The statistic on the number of chunks / intermediate answers depending on the dataset would be interesting. \n3. How sensitive is the reasoning data segmentation / captioning to the external LLM choice?", "extra_scores": {}}]

Author rebuttal:

Meta-review: Overall quality: This paper presents an empirical study on leveraging hidden states in reasoning language models to enable early-exit generation through a lightweight MLP-based verifier. The empirical setup is sound, and the findings are clearly demonstrated, with a promising application for reducing generation cost and latency for math reasoning tasks. However, the scope of experimentation is somewhat narrow (mostly focused on math reasoning), and the novelty relative to existing literature is a bit unclear. As a result, reviewers consistently point out the need for more baselines, more datasets, and clearer positioning relative to prior work on internal verifiers and logit lens methods. 

Presentation clarity: The paper is clearly written and easy to follow. All three reviewers appreciated the presentation quality. The methodology and results are presented with sufficient detail for replication. 

Originality: While the application of probing hidden states with a simple verifier is not entirely new, the specific investigation on reasoning models is relatively underexplored.

Significance: The idea has practical implications for making reasoning models more efficient, especially in scenarios where latency or token cost is critical. However, the current evaluation is mostly limited to math problems, and thus the generality and robustness of the approach remain unclear. Its broader significance would be enhanced by extending the study to other domains like coding, theorem proving and more.

Pros:
- Clear and well-written presentation.
- Simple and effective method for early exit in reasoning models.
- Demonstrates tangible gains in token efficiency (24% reduction).
- Offers a practical, lightweight solution applicable to existing models.

Cons:
- Unclear position compared to existing probing methods on LLMs, thus raising concerns about the limited comparison with relevant baselines from recent literature.
- Evaluations are confined to math reasoning tasks, limiting generalizability.
- Relies on external LLMs for data segmentation and annotation, raising scalability and robustness concerns.
- Lack of deeper analysis on why hidden states are predictive of reasoning correctness.
Decision: Accept