On the Self-awareness of Large Reasoning Models' Capability Boundaries

## Abstract

Abstract Large Reasoning Models (LRMs) have shown impressive performance on complex reasoning tasks such as mathematics, yet they also display misbehaviors that expose their limitations.
In particular, when faced with hard questions, LRMs often engage in unproductive reasoning until context limit, producing wrong answers while wasting substantial computation.
This phenomenon reflects a fundamental issue: current answering paradigms overlook the relationship between questions and LRMs’ capability boundaries .
In this paper, we investigate whether LRMs possess self-awareness of capability boundaries .
We begin by an observation that LRMs may know what they cannot solve through expressed reasoning confidence.
For black-box models, we find that reasoning expressions reveal boundary signals, with accelerated growing confidence trajectory for solvable problems but convergent uncertainty trajectory for unsolvable ones. For white-box models, we show that hidden states of the last input token encode boundary information, with solvable and unsolvable problems linearly separable even before reasoning begins.
Building on these findings, we propose two simple yet effective optimization strategies: reasoning expression monitoring and hidden states monitoring. Experiments demonstrate that these boundary-aware strategies enable LRMs to avoid unproductive reasoning without sacrificing accuracy, significantly improving reliability and efficiency by cutting token usage up to 62.7 62.7 – 93.6 % 93.6\% .
We open-source at https://github.com/qingjiesjtu/CapBound .

## 1 Introduction

Large Reasoning Models (LRMs) have demonstrated remarkable capabilities on complex reasoning tasks such as mathematics (Guo et al., 2025; Jaech et al., 2024; Ahn et al., 2024). However, they also exhibit a range of misbehaviors that reveal the limitations of their capabilities (Kalai et al., 2025; Yao et al., 2025b; Sun et al., 2025; Zhang et al., 2024). In particular, when confronted with hard questions, LRMs may fall into repetitive looping until the context length is exhausted (Zhou et al., 2025; Chen et al., 2025b). Such a response paradigm not only fails to produce a correct answer but also wastes computational resources (Peng et al., 2025; Pu et al., 2025; Chen et al., 2024b), leaving users waiting a long time with no meaningful outcome.

This phenomenon highlights a fundamental issue: current answering paradigms of LRMs overlook the relationship between questions and their capability boundaries. For questions beyond their capability boundary, LRMs still struggle to produce a wrong answer, often engaging in unproductive long reasoning manifested as repetitive looping or error accumulation (Yao et al., 2025a; Mukherjee et al., 2025) (shown in [Appendix E](https://arxiv.org/html/2509.24711v2#A5)), which undermines both response efficiency and reliability. Therefore, it is crucial to identify questions beyond capability boundaries, and to explicitly acknowledge the inability to solve them. This importance has also been recognized by the broader AI community, as exemplified by the praise for GPT-5’s ability to appropriately say “I don’t know.”(^1^11[https://x.com/koltregaskes/status/1957474061153436094](https://x.com/koltregaskes/status/1957474061153436094))

Existing approaches to exploring capability boundaries are mainly conducted from an external perspective, for example, by constructing increasingly challenging benchmarks such as Humanity’s Last Exam (HLE) (Phan et al., 2025), or by designing real-time routers that quickly estimate question difficulty and assign harder questions to more capable models (OpenAI, 2025). However, little work has investigated capability boundaries from the model’s internal perspective, namely self-awareness of capability boundaries, which is particularly meaningful for understanding reasoning mechanism and for further improving reliability and efficiency.

Figure: Figure 1: Capability boundary is encoded in reasoning trajectories as expression density patterns, and in the last token hidden states even before reasoning begins. Reasoning with capability boundary enables LRMs to identify unsolvable questions and provide more reliable and efficient answers.
Refer to caption: x1.png

Thus, this paper focuses on answering the question: “Do LRMs know their capability boundaries?” [Figure 1](https://arxiv.org/html/2509.24711v2#S1.F1) shows the overview of our work. We begin with the observation that LRMs often exhibit awareness of what they cannot solve ([Section 3](https://arxiv.org/html/2509.24711v2#S3)), and we provide evidence for such self-awareness for both black-box and white-box scenarios. For black-box models, we show that reasoning expressions contain clear signals of capability boundaries ([Section 4](https://arxiv.org/html/2509.24711v2#S4)). For white-box models, we find that hidden states encode information that also delineates these boundaries ([Section 5](https://arxiv.org/html/2509.24711v2#S5)). Taken together, these findings suggest that LRMs can assess, on the fly, whether a given question is beyond capability boundary, thereby enabling more efficient and reliable reasoning ([Section 6](https://arxiv.org/html/2509.24711v2#S6)). We summarize our contributions as follows.

- $\bullet$
We observe indications of LRMs’ self-awareness of capability boundaries by comparing their reasoning confidence, articulated in anthropomorphic tones, with the correctness of their final answers. Notably, LRMs know what they cannot solve.
- $\bullet$
From a black-box perspective, we reveal LRMs’ capability boundaries through reasoning trajectories. Remarkably, their reasoning shows an accelerated growing confidence trajectory for questions within the boundary, but a convergent uncertainty trajectory for questions beyond it.
- $\bullet$
From a white-box perspective, we reveal LRMs’ capability boundaries through the hidden states of the last input token. Remarkably, solvable and unsolvable questions are almost linearly separable in the hidden space, even before the reasoning begins.
- $\bullet$
To leverage capability boundaries during reasoning, we design two simple yet effective optimization strategies: Reasoning Expression Monitoring (black-box) and Hidden States Monitoring (white-box). Experiments show that these boundary-aware strategies substantially improve efficiency and reliability without sacrificing accuracy, reducing token usage up to $62.7$ – $93.6\%$.

## 2 Related Work

From knowledge boundary to capability boundary. Recent studies have examined the knowledge boundaries of large language models (LLMs), focusing on the limits of factual knowledge encoded in parameters (Yin et al., 2024; Li et al., 2024). For instance, Wen et al. (2024) investigate LLMs’ perception of knowledge boundary through semi-open-ended question answering. Deng et al. (2025) unveil knowledge boundary of LLMs for trustworthy information access. Kale & Nadadur (2025) evaluate LLMs’ self-knowledge via consistency. These works primarily emphasize factual coverage, with retrieval-augmented generation (RAG) being a common solution to extend the accessible knowledge space (Gao et al., 2023; Lewis et al., 2020).

By contrast, the capability boundary concerns what reasoning tasks LLMs can reliably solve (Chen et al., 2025a; Xue et al., 2025; Chen et al., 2024a). This notion is broader yet rarely formalized. Existing efforts typically approximate it externally: either by constructing more challenging benchmarks such as Humanity’s Last Exam (HLE) (Phan et al., 2025), or by designing routing mechanisms such as the GPT-5 router that delegate harder questions to more capable models (OpenAI, 2025). In contrast, this paper investigates capability boundaries from the internal perspective of LLMs, focusing on their potential self-awareness.

Interpretability of reasoning expressions and hidden states. Interpretability aims to uncover how LLMs understand and execute tasks, providing insights that enhance their reliability and efficiency. Recent studies have approached from both reasoning expressions and hidden states. On the reasoning expressions side, Guo et al. (2025) highlight the anthropomorphic cues, “aha moments”, as a key role in reasoning. Qian et al. (2025) tracks mutual information along the generation and finds sharp surges at specific thinking tokens (e.g., “Hmm”, “Wait”, “Therefore”). Bogdan et al. (2025) frame sentence-level importance as a counterfactual influence problem, showing that only a few critical utterances steer the reasoning trajectory.

On the hidden states side, Wendler et al. (2024) probe multilingual Llama models and reveal English as the pivot language in the internal structure. Fan et al. (2024) demonstrate that certain intermediate layers already encode sufficient predictive features, implying redundancy in the full forward pass. Zhang et al. (2025) train linear probes on hidden states to verify intermediate answers, anticipating future correctness. Our work builds on both perspectives by analyzing reasoning expressions and hidden states to reveal self-awareness of capability boundaries.

Efficient reasoning: from solvable questions to unsolvable questions. A range of recent works examine the phenomenon of overthinking (Sui et al., 2025). Chen et al. (2024b) show that LRMs often allocate excessive computation to easy questions, while Zhang et al. (2024) report that multi-turn self-correction can lead to unnecessary cost.
To mitigate such inefficiency, Wang et al. (2025) demonstrate that suppressing frequent reflective tokens such as “Wait”, “Hmm”, or “Alternatively” improves reasoning efficiency without harming accuracy. Similarly, Yang et al. (2025a) introduce mechanisms to decide during reasoning whether further steps are unnecessary, allowing the model to terminate early and output the final answer directly.

However, these works target the case where LRMs can solve a question but waste resources by reasoning too long. In contrast, our work addresses the complementary scenario: when a question is beyond the model’s capability boundary, LRMs still continue reasoning unproductively despite having no chance to get a correct answer.

## 3 Large Reasoning Models Know What They Cannot Solve

We begin our exploration by evaluating commonly used LRMs including GPT-oss-20B (Agarwal et al., 2025), DeepSeek-R1-0528-Qwen3-8B (abbreviated as R1-Distill-Qwen3-8B) (Guo et al., 2025), DeepSeek-R1-Distill-Qwen-32B (abbreviated as R1-Distill-Qwen-32B) (Guo et al., 2025), and QwQ-32B (Qwen Team, 2024) on several standard mathematical benchmarks, including AIME’25 (MAA, 2025), AIME’24 (MAA, 2024), HMMT (HMMT, 2025), and AMC’23 (MAA, 2023) (see [Appendix A](https://arxiv.org/html/2509.24711v2#A1) for detailed descriptions of datasets and models).

Figure: Figure 2: Confident vs. uncertain expressions in reasoning.
Refer to caption: x2.png

Inspired by works which monitors LRMs’ reasoning (Baker et al., 2025; He et al., 2025; Luo et al., 2024), we examine the reasoning processes (e.g., the segments wrapped within <think> in DeepSeek-like models) to look for signs of self-awareness regarding whether the input question can be correctly solved. As illustrated in [Figure 2](https://arxiv.org/html/2509.24711v2#S3.F2), we observe that such signals, expressed as anthropomorphic tones (Guo et al., 2025; Yang et al., 2025b), emerge throughout the reasoning process, and become especially evident in its final stages. Toward the end of reasoning, models often produce explicit statements that reflect their confidence level. Strikingly, these expressed beliefs tend to align with the correctness of their final answers. For example, when the reasoning contains more confident expressions such as “So, no mistake” or “I think it’s correct”, the final answers are typically correct. Conversely, when uncertain expressions like “My initial assumption is wrong” or “I’m not 100% sure” appear, the final answers are incorrect.

This observation motivates us to quantify the alignment between expressed belief and answer correctness, which we formalize through a pair of metrics:

$$ Can % $\displaystyle=\mathbb{P}(\text{Correct answer}\mid\text{Model expresses more confidence}),$ (1) Cannot % $\displaystyle=\mathbb{P}(\text{Wrong answer}\mid\text{Model expresses more uncertainty}).$ $$

Specifically, Can % is the proportion of questions for which the model expresses more confidence during reasoning and indeed produces a correct answer, while Cannot % is the proportion of questions for which the model expresses more uncertainty and indeed fails to provide the correct answer.

**Table 1: LRMs’ final answers aligns with their expressed belief, especially for incorrect final answers (Cannot % = 100).**
|  | Can % | Cannot % |
| --- | --- | --- |
| GPT-oss-20B | $80.2$ | $100$ |
| R1-Distill-Qwen3-8B | $97.6$ | $100$ |
| R1-Distill-Qwen-32B | $90.6$ | $100$ |
| QwQ-32B | $86.0$ | $100$ |

[Table 1](https://arxiv.org/html/2509.24711v2#S3.T1) shows the results. We observe that LRMs’ expressed beliefs align with the correctness of their final answers. In particular, for questions where LRMs express more uncertainty during reasoning, the final answers are consistently incorrect (Cannot % = 100). This phenomenon is observed across multiple models, suggesting that LRMs are aware of what they cannot solve. Taken together, the reasoning expressions indicate a form of self-awareness of capability boundary.

## 4 Capability Boundary Revealed by Reasoning Expressions

Motivated by the observations in the previous section, we hypothesize that it is feasible to reveal LRMs’ capability boundaries through their reasoning expressions. Therefore, we systematically investigate the dynamics of confident and uncertain expressions throughout the reasoning process.

We first construct two sets of expressions, confident and uncertain expressions, that frequently appear in the reasoning process (the full list is provided in [Appendix C](https://arxiv.org/html/2509.24711v2#A3)). These anthropomorphic expressions are commonly observed and have been recognized in prior work as critical for reasoning (Yang et al., 2025b; Guo et al., 2025; Qian et al., 2025). Second, we divide the reasoning process into multiple stages (e.g., 50 stages) and compute the density of confident and uncertain expressions at each stage. Finally, concatenating these values yields the expression density trajectories, denoted as
$\mathcal{D}_{C}(t)$ for confident expressions and $\mathcal{D}_{U}(t)$ for uncertain expressions. These trajectories capture how the density of each type of expression evolves over the course of reasoning.

Figure: Figure 3: Capability boundaries revealed by reasoning expressions. Questions within the boundary exhibit concave trajectory for confident expressions, whereas questions beyond the boundary exhibit convex trajectory for uncertain expressions.
Refer to caption: x3.png

We plot $\mathcal{D}_{C}(t)$ and $\mathcal{D}_{U}(t)$ in [Figure 3](https://arxiv.org/html/2509.24711v2#S4.F3), separately for cases with correct and wrong final answers. We observe that LRMs exhibit clear differences in their reasoning expression trajectories depending on final answer correctness. When the final answer is correct, the confident trajectory lies above the uncertain trajectory and shows an accelerating increasing trend. In contrast, when the final answer is incorrect, the uncertain trajectory dominates over the confident trajectory and tends to converge. Based on this distinction, the capability boundary can be defined as a linearly increasing trajectory (blue line in [Figure 3](https://arxiv.org/html/2509.24711v2#S4.F3)). A concave trajectory indicates that the question lies within the LRM’s capability boundary, while a convex trajectory indicates that the question lies beyond it.

To formally characterize these patterns, we design two indicators as criteria for determining whether a question is within or beyond the capability boundary:

- $\bullet$
Confidence Differential (ConfDiff) measures the sign of gap between uncertain and confident trajectory along the reasoning process. Specifically, we accumulate the sign up to a given reasoning stage percentage $s\in[0,100]$, and treat the question as beyond the capability boundary if the cumulative value exceeds a threshold $\alpha_{s}$:
$\text{ConfDiff}(s)=\mathbb{P}\!\left(\mathcal{D}_{U}(t)>\mathcal{D}_{C}(t)\ \bigg|\ t\in(0,s)\right)>\alpha_{s}.$
(2)
- $\bullet$
Confidence Curvature (ConfCurv) computes the second derivative of the expression density trajectory to capture its convexity (Boyd & Vandenberghe, 2004). A concave trajectory corresponds to questions within the capability boundary, while a convex trajectory indicates questions beyond it. Also, we accumulate the sign of convexity up to a reasoning stage percentage $s$ and treat the question as beyond the capability boundary if the value exceeds a threshold $\beta_{s}$:
$\text{ConfCurv}(s)=\mathbb{P}\!\left(\frac{d^{2}}{dt^{2}}\,\big(\mathcal{D}_{U}(t)-\mathcal{D}_{C}(t)\big)<0\ \bigg|\ t\in(0,s)\right)>\beta_{s}.$
(3)

[Figure 4](https://arxiv.org/html/2509.24711v2#S4.F4) shows that the two proposed indicators can reliably distinguish unsolvable from solvable questions, achieving high accuracy across different LRMs throughout the reasoning stage. Between the two indicators, ConfDiff is more reliable as it achieves higher accuracy and exhibits smaller fluctuations during the reasoning process. Moreover, the high accuracy emerges early, e.g., as early as 2% for GPT-oss-20B. Interestingly, the gradually declining yellow curve of this model suggests that boundary signals are more salient at the beginning than at the end of reasoning. This is likely due to repetitive looping and error accumulation in later stages (Yao et al., 2025a; Mukherjee et al., 2025). Therefore, we demonstrate that the capability boundary can be revealed through reasoning expressions, even at an early stage.

Figure: Figure 4: Accuracy (%) of ConfDiff and ConfCurv to separate unsolvable and solvable questions through out the reasoning stage. ConfDiff is more reliable as it achieves higher accuracy and exhibits smaller fluctuations. Boundary awareness signal appears at an early stage (e.g., 2%).
Refer to caption: x4.png

## 5 Capability Boundary Revealed by Hidden States

Following the observation that capability boundaries can be revealed at an early stage of reasoning, it is worth examining whether such signals may emerge even earlier, potentially before the model begins reasoning. Therefore, we investigate whether capability boundaries are already encoded in the hidden states of input questions.

Since hidden states are high-dimensional (e.g., 4096), limited samples may lead to sparsity issues that obscure statistical patterns (Vishwakarma, 2024; Köppen, 2000). To mitigate this, we first expand our dataset by including GSM8K (Cobbe et al., 2021) and Humanity’s Last Exam (HLE) (Phan et al., 2025). These two benchmarks provide complementary levels of difficulty (see [Appendix A](https://arxiv.org/html/2509.24711v2#A1) for full accuracy results): GSM8K is relatively easy, with an accuracy exceeding 90%; HLE is extremely difficult, with an accuracy below 10%. This contrast helps amplify the distinction between solvable and unsolvable questions, making capability boundaries more salient.

Next, we extract hidden states from the prefilling stage as it processes the input question (Yuan et al., 2024; Pope et al., 2023), using the hook method (Vig, 2019). Specifically, we focus on the hidden state of the last input token, which encapsulates the semantic content of the entire question (Radford et al., 2019). Finally, we apply simple linear classifiers to these hidden states, including Linear Discriminant Analysis (LDA) and Logistic Regression (LR) (Hastie, 2009). Using final answer correctness as labels, we test whether solvable and unsolvable questions can be linearly separated, thereby revealing the presence of capability boundaries in hidden states.

Figure: Figure 5: Left: Capability boundary revealed by linear classifier clearly separates solvable and unsolvable questions in hidden states. Right: Among solvable questions, those close to capability boundary requires more token usage ($1.5$ – $2\times$) to arrive at the correct answer.
Refer to caption: x5.png

**Table 2: Accuracy (%) of separating solvable and unsolvable questions in hidden states using linear classifiers (robust across hyperparameter choices).**
|  | LDA | LR |  |
| --- | --- | --- | --- |
|  | $C=0.1$ | $C=1$ |  |
| GPT-oss-20B | $98.0$ | $98.9$ | $98.2$ |
| R1-Distill-Qwen3-8B | $96.7$ | $96.7$ | $97.0$ |
| R1-Distill-Qwen-32B | $97.8$ | $97.8$ | $98.0$ |
| QwQ-32B | $98.9$ | $98.2$ | $98.2$ |

The left subfigure of [Figure 5](https://arxiv.org/html/2509.24711v2#S5.F5) visualizes the classification of solvable and unsolvable questions (see [Appendix D](https://arxiv.org/html/2509.24711v2#A4) for results of all LRMs), along with the capability boundary (blue line) determined by the decision boundary of linear classifier (e.g., LDA). We observe that the two classes of questions are clearly separated, under random train – test splits. To quantify this separation, [Table 2](https://arxiv.org/html/2509.24711v2#S5.T2) shows the classification accuracy, which is consistently close to $100\%$ and robust to the choice of classifier and hyperparameters (e.g., the regularization coefficient $C$ of LR). These results demonstrate that the features of capability boundaries are strongly encoded in the hidden states of the last input token.

Interestingly, we also observe that among solvable questions, some lie very close to the capability boundary, while others are far away. We compare these two cases by measuring their token usage during inference. Although LRMs are able to produce correct answers, the average token usage of the questions close to the boundary is $1.5$ – $2$ times longer than that far from the boundary (right subfigure of [Figure 5](https://arxiv.org/html/2509.24711v2#S5.F5)). This further supports the validity of the identified capability boundary: solvable questions closer to the boundary require longer reasoning to reach the correct answer.

Therefore, the capability boundary is encoded in the hidden states of the last input token, and it also reflects the reasoning effort required to solve the question.

## 6 Reason with Capability Boundary

The analysis in the previous sections reveal that LRMs exhibit clear signs of self-awareness on capability boundaries: reasoning expressions expose confidence patterns that distinguish solvable from unsolvable problems, and hidden states encode boundary information even before reasoning begins. These findings suggest that capability boundaries are not only detectable but can also be harnessed to improve efficiency and reliability of LRMs’ response. In this section, we propose two strategies to reason with capability boundary and demonstrate their effectiveness.

### 6.1 Experimental Design

Strategies. Building on the evidence that self-awareness of capability boundaries emerges during test time, we propose two test-time monitoring strategies that leverage boundary-awareness to guide the reasoning process. The key idea is to monitor signals of capability boundaries on the fly, either from reasoning trajectories (black-box) or hidden states (white-box), and to adjust reasoning when questions are beyond the model’s capability. This approach allows LRMs to avoid unproductive reasoning and give more reliable response.

- $\bullet$
Reasoning Expression Monitoring (Monitor${}_{\text{express}}$) is a black-box strategy that leverages expressions density during reasoning. The pipeline is as follows:
1.
Construct expression trajectories: continuously track the densities of confident and uncertain expressions (e.g., “I think that’s it” vs. “I might be wrong”) across reasoning steps to form expression density trajectories $\mathcal{D}_{C}(t)$ and $\mathcal{D}_{U}(t)$.
2.
Detect unsolvable question: analyze these trajectories on the fly using quantitative indicators such as Confidence Differential (ConfDiff) and Confidence Curvature (ConfCurv) to determine whether a question lies within or beyond the capability boundary.
3.
Early stop unproductive reasoning and reprompt with suffix: if the trajectory indicates the question is beyond the capability boundary, we early terminate reasoning and reprompt the model to output a concise approach. This is implemented by appending a self-awareness prompt suffix (shown below) to the end of the question; otherwise, it continues with the standard reasoning process.
Self-awareness prompt suffix
The above question is beyond your capability boundary. Do not solve the question, just provide a concise potential approach of less than 5 steps:
- $\bullet$
Hidden States Monitoring (Monitor${}_{\text{hidden}}$) is a white-box strategy that leverages the hidden states of the last input token. The pipeline is as follows:
1.
Extract hidden states: capture the hidden states of the last input token, which encodes the semantic representation of the entire question during the prefilling stage.
2.
Detect unsolvable question: apply simple linear classifiers (e.g., LDA or LR) trained to distinguish unsolvable from solvable questions. This is performed before reasoning begins.
3.
Avoid unproductive reasoning via constrained output prefix: if the classifier predicts that the question is beyond the capability boundary, the model abstains from full reasoning and instead outputs a concise possible approach. This is implemented by constraining the response to begin with a self-awareness output prefix (shown below); otherwise, the model proceeds with the standard reasoning process.
Self-awareness output prefix
<think>
I think this question is beyond my capability boundary. I cannot fully solve it, but I can outline a concise potential approach. I must give a concise outline to the user (less than 10 steps)!
</think>
This question is beyond my capability boundary, but I can outline a concise potential approach:

Notably, both strategies preserve the model’s performance on solvable questions while providing more efficient and reliable responses on questions beyond its capability boundary.

Metrics. We adopt four metrics to systematically evaluate the two proposed strategies:

- $\bullet$
Accuracy (ACC) measures the proportion of correctly solved problems, serving as the standard indicator of reasoning quality (Xia et al., 2025).
- $\bullet$
Hard Abstention (HA) measures the model’s ability to recognize problems beyond its capability boundary. Different to abstention which focus on underspecification, outdated information, or noncompliance (Kirichenko et al., 2025; Madhusudhan et al., 2024; Brahman et al., 2024), HA specifically evaluates whether LRMs can explicitly abstain on unsolvable problems instead of producing incorrect answers.
- $\bullet$
Token usage measures the number of tokens consumed during reasoning and answering (Snell et al., 2024). This metric directly reflects the efficiency of the reasoning process.
- $\bullet$
Overflow measures the proportion of cases where the model’s output exceeds the context window, which identifies incomplete or abnormal responses (Ben-Kish et al., 2025).

Notably, ACC is evaluated on the full dataset, whereas the remaining three metrics are computed only on questions for which LRMs produce incorrect answers. For token usage and overflow, we vary the models’ context length to observe how our strategies scale in longer context scenarios.

Baseline. Prior work has explored improving reasoning performance by self-verification after completing the full Chain-of-Thought (Madhusudhan et al., 2024) or by monitoring undesirable behaviors during training (Baker et al., 2025). However, these approaches are orthogonal to our scenario: the former introduces additional computation after reasoning, while the latter does not address hard questions beyond the capability boundary.

Therefore, we adopt a baseline that improve efficiency by enabling the model to abstain. In particular, we consider BoostAbstention (Kirichenko et al., 2025) which crafts a system prompt that encourages the model to abstain in designated scenarios, showing effectiveness for both reasoning and standard LLMs. For our experiments, we modify the system prompt to better match the abstention scenarios of questions beyond capability boundary (see [Appendix B](https://arxiv.org/html/2509.24711v2#A2) for the full prompt).

### 6.2 Results

[Table 3](https://arxiv.org/html/2509.24711v2#S6.T3) compares the performance of the original models, the baseline BoostAbstention, and our proposed monitoring strategies. We highlight the following key observation.

**Table 3: Monitor${}_{\text{express}}$ and Monitor${}_{\text{hidden}}$ compared to baseline BoostAbstention. Our strategies enable LRMs to recognize questions beyond capability boundary (HA) without compromising accuracy (ACC), and provide more efficient and reliable response (Token usage and Overflow).**
| Context Length |  |  | 2048 | 4096 |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| Metric | ACC (%) ↑ | HA (%) ↑ | Token ↓ | Overflow (%) ↓ | Token ↓ | Overflow (%) ↓ |
| GPT-oss-20B | $74.6$ | 0 | $2029$ | $97.0$ | $4015$ | $97.0$ |
| +BoostAbstention | $74.6$$\blacktriangledown$$0\%$ | 0$\blacktriangle$$0\%$ | $2023$$\blacktriangledown$$0.3\%$ | $97.0$$\blacktriangledown$$0\%$ | $3933$$\blacktriangledown$$2.0\%$ | $84.8$$\blacktriangledown$$12.2\%$ |
| +Monitor${}_{\text{express}}$ | $74.6$$\blacktriangledown$$0\%$ | $100$$\blacktriangle$$100\%$ | $1135$$\blacktriangledown$$44.1\%$ | $21.2$$\blacktriangledown$$75.8\%$ | $1499$$\blacktriangledown$$62.7\%$ | $15.2$$\blacktriangledown$$81.8\%$ |
| +Monitor${}_{\text{hidden}}$ | $73.7$$\blacktriangledown$$0.9\%$ | $98.9$$\blacktriangle$$98.9\%$ | $1784$$\blacktriangledown$$12.1\%$ | $78.4$$\blacktriangledown$$18.6\%$ | $3317$$\blacktriangledown$$17.4\%$ | $5.4$$\blacktriangledown$$91.6\%$ |
| R1-Distill-Qwen3-8B | $76.9$ | 0 | $2048$ | $100$ | $4096$ | $100$ |
| +BoostAbstention | $76.9$$\blacktriangledown$$0\%$ | 0$\blacktriangle$$0\%$ | $2048$$\blacktriangledown$$0\%$ | $100$$\blacktriangledown$$0\%$ | $4096$$\blacktriangledown$$0\%$ | $100$$\blacktriangledown$$0\%$ |
| +Monitor${}_{\text{express}}$ | $74.6$$\blacktriangledown$$2.3\%$ | $63.3$$\blacktriangle$$63.3\%$ | $1455$$\blacktriangledown$$29.0\%$ | $40.0$$\blacktriangledown$$60.0\%$ | $2551$$\blacktriangledown$$37.7\%$ | $30.0$$\blacktriangledown$$70.0\%$ |
| +Monitor${}_{\text{hidden}}$ | $73.8$$\blacktriangledown$$3.1\%$ | $97.9$$\blacktriangle$$97.9\%$ | $787$$\blacktriangledown$$61.6\%$ | $22.2$$\blacktriangledown$$77.8\%$ | $1243$$\blacktriangledown$$69.7\%$ | 0$\blacktriangledown$$100\%$ |
| R1-Distill-Qwen-32B | $56.2$ | 0 | $2048$ | $100$ | $4095$ | $97.4$ |
| +BoostAbstention | $56.2$$\blacktriangledown$$0\%$ | 0$\blacktriangle$$0\%$ | $2005$$\blacktriangledown$$2.1\%$ | $94.7$$\blacktriangledown$$5.3\%$ | $3903$$\blacktriangledown$$4.7\%$ | $91.2$$\blacktriangledown$$6.2\%$ |
| +Monitor${}_{\text{express}}$ | $56.2$$\blacktriangledown$$0\%$ | $100$$\blacktriangle$$100\%$ | $801\raisebox{0.5pt}{{{\color[rgb]{0,0.45,0.24}\definecolor[named]{pgfstrokecolor}{rgb}{0,0.45,0.24}{$\blacktriangledown${$60.9\%$}}}}}$ | 0$\blacktriangledown$$100\%$ | $801$$\blacktriangledown$$80.4\%$ | 0$\blacktriangledown$$97.4\%$ |
| +Monitor${}_{\text{hidden}}$ | $55.3$$\blacktriangledown$$0.9\%$ | $97.0$$\blacktriangle$$97.0\%$ | $457$$\blacktriangledown$$77.7\%$ | $5.3$$\blacktriangledown$$94.7\%$ | $565$$\blacktriangledown$$86.2\%$ | $5.3$$\blacktriangledown$$92.1\%$ |
| QwQ-32B | $71.5$ | 0 | $2048$ | $100$ | $4096$ | $100$ |
| +BoostAbstention | $71.5$$\blacktriangledown$$0\%$ | 0$\blacktriangle$$0\%$ | $2048$$\blacktriangledown$$0\%$ | $100$$\blacktriangledown$$0\%$ | $4096$$\blacktriangledown$$0\%$ | $100$$\blacktriangledown$$0\%$ |
| +Monitor${}_{\text{express}}$ | $71.5$$\blacktriangledown$$0\%$ | $100$$\blacktriangle$$100\%$ | $1164$$\blacktriangledown$$43.2\%$ | $32.4$$\blacktriangledown$$67.6\%$ | $1828$$\blacktriangledown$$55.4\%$ | $32.4$$\blacktriangledown$$67.6\%$ |
| +Monitor${}_{\text{hidden}}$ | $71.5$$\blacktriangledown$$0\%$ | $96.9$$\blacktriangle$$96.9\%$ | $262$$\blacktriangledown$$87.2\%$ | 0$\blacktriangledown$$100\%$ | $262$$\blacktriangledown$$93.6\%$ | 0$\blacktriangledown$$100\%$ |

Observation 1: our strategies enable LRMs to recognize unsolvable questions without compromising their accuracy on solvable ones.

Although the original models achieve acceptable Accuracy (ACC), their Hard Abstention (HA) is zero, indicating a complete lack of ability to abstain on questions beyond the capability boundary. Moreover, almost all Overflow is 100%, suggesting that most incorrect cases arise because LRMs struggling to solve a hard question until exceeding the context length.

Promisingly, both Monitor${}_{\text{express}}$ and Monitor${}_{\text{hidden}}$ achieve strong performance. First, their ACC remains nearly unchanged compared to the original models, showing that good reasoning performance is preserved. Meanwhile, the substantially high HA (almost 100%) demonstrates that LRMs can now effectively identify problems beyond the capability boundary.

Surprisingly, and contrary to the observation from Kirichenko et al. (2025), the baseline BoostAbstention has almost no effect in our scenario. This reveals that mathematical reasoning problems differ from outdated or unsafe requests: abstention cannot be induced simply by system prompts, but instead requires boundary signals encoded in hidden states or reasoning expressions.

Observation 2: our strategies enable LRMs to provide more efficient and reliable responses to unsolvable questions by offering concise solution approaches.

Once questions beyond the capability boundary are recognized, our strategies enable LRMs to improve efficiency and reliability by offering concise solution approaches. [Figure 6](https://arxiv.org/html/2509.24711v2#S6.F6) illustrates the responses produced by our strategies compared to the original models. The original models struggle with unsolvable questions and often fail to complete the reasoning process even after exhausting the context length. Such responses provide neither a correct answer nor valuable insights from the lengthy reasoning trace, while also wasting user time.

Figure: Figure 6: Our strategies avoid unproductive reasoning by acknowledging unsolvable questions, improving efficiency and reliability of response.
Refer to caption: x6.png

In contrast, our strategies detect these unsolvable questions and guide the model to explicitly acknowledge that they are beyond capability boundary, while offering a concise possible approach. This response paradigm provides users with possible hints for problem solving without wasting time on meaningless output. And importantly, it does not compromise the model’s ability to produce correct answers for solvable questions.

This explains why we observe large reductions in both token usage (up to 93.6%) and overflow (up to 100%) in [Table 3](https://arxiv.org/html/2509.24711v2#S6.T3), indicating that reasoning with capability boundaries greatly improves efficiency and mitigates incomplete outputs. Besides, this reduction becomes more evident with larger context length. For example, overflow reduces from 78.4% to 5.4% on GPT-oss-20B when the context length is extended from 2048 to 4096. This suggests that our strategies holds promise for optimizing long-context reasoning tasks (Ling et al., 2025; Kuratov et al., 2024).

Observation 3: LRMs vary in reasoning behavior correction, with some good at instruction following and others more prone to output guidance.

We compare the performance of Monitor${}_{\text{express}}$ and Monitor${}_{\text{hidden}}$ across different LRMs. For GPT-oss-20B, Monitor${}_{\text{express}}$ achieves lower token usage than Monitor${}_{\text{hidden}}$. We find this model is resistant to output guidance via constrained output prefix: although it sometimes begins by generating a solution approach, it tends to later negate this and restart the reasoning process, resulting in long and unproductive outputs. In contrast, its instruction-following ability is relatively strong, and thus reprompting with the self-awareness prompt suffix yields lower token usage.

Conversely, the other three LRMs show weaker ability of instruction following but more prone to output guidance, making Monitor${}_{\text{hidden}}$ more effective than Monitor${}_{\text{express}}$. This suggest that, in addition to the black-box versus white-box distinction, the choice of optimization strategy in practice should also consider the behavioral tendencies of the specific LRM.

## 7 Conclusion

In this work, we investigate whether Large Reasoning Models (LRMs) possess self-awareness of their capability boundaries. Through both black-box and white-box analyses, we found consistent evidence that LRMs can distinguish unsolvable from solvable questions: reasoning expressions reveal separable expression trajectories, while hidden states of the last input token encode boundary information even before reasoning begins. Building on these insights, we proposed two simple yet effective test-time monitoring strategies that leverage boundary signals to prevent unproductive reasoning for unsolvable questions. Our experiments show that these strategies preserve accuracy on solvable problems while substantially improving efficiency and reliability on unsolvable ones.

## References

- Agarwal et al. (2025)
Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applebaum, Edwin Arbus, Rahul K Arora, Yu Bai, Bowen Baker, Haiming Bao, et al.
gpt-oss-120b & gpt-oss-20b model card.
*arXiv preprint arXiv:2508.10925*, 2025.
- Ahn et al. (2024)
Janice Ahn, Rishu Verma, Renze Lou, Di Liu, Rui Zhang, and Wenpeng Yin.
Large language models for mathematical reasoning: Progresses and challenges.
*arXiv preprint arXiv:2402.00157*, 2024.
- Baker et al. (2025)
Bowen Baker, Joost Huizinga, Leo Gao, Zehao Dou, Melody Y Guan, Aleksander Madry, Wojciech Zaremba, Jakub Pachocki, and David Farhi.
Monitoring reasoning models for misbehavior and the risks of promoting obfuscation.
*arXiv preprint arXiv:2503.11926*, 2025.
- Ben-Kish et al. (2025)
Assaf Ben-Kish, Itamar Zimerman, M Jehanzeb Mirza, James Glass, Leonid Karlinsky, and Raja Giryes.
Overflow prevention enhances long-context recurrent llms.
*arXiv preprint arXiv:2505.07793*, 2025.
- Bogdan et al. (2025)
Paul C Bogdan, Uzay Macar, Neel Nanda, and Arthur Conmy.
Thought anchors: Which llm reasoning steps matter?
*arXiv preprint arXiv:2506.19143*, 2025.
- Boyd & Vandenberghe (2004)
Stephen Boyd and Lieven Vandenberghe.
*Convex Optimization*.
Cambridge University Press, 2004.
- Brahman et al. (2024)
Faeze Brahman, Sachin Kumar, Vidhisha Balachandran, Pradeep Dasigi, Valentina Pyatkin, Abhilasha Ravichander, Sarah Wiegreffe, Nouha Dziri, Khyathi Chandu, Jack Hessel, et al.
The art of saying no: Contextual noncompliance in language models.
*Advances in Neural Information Processing Systems*, 37:49706–49748, 2024.
- Chen et al. (2024a)
Qiguang Chen, Libo Qin, Jiaqi Wang, Jingxuan Zhou, and Wanxiang Che.
Unlocking the capabilities of thought: A reasoning boundary framework to quantify and optimize chain-of-thought.
*Advances in Neural Information Processing Systems*, 37:54872–54904, 2024a.
- Chen et al. (2025a)
Qiguang Chen, Libo Qin, Jinhao Liu, Yue Liao, Jiaqi Wang, Jingxuan Zhou, and Wanxiang Che.
Rbf++: Quantifying and optimizing reasoning boundaries across measurable and unmeasurable capabilities for chain-of-thought reasoning.
*arXiv preprint arXiv:2505.13307*, 2025a.
- Chen et al. (2025b)
Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che.
Towards reasoning era: A survey of long chain-of-thought for reasoning large language models.
*arXiv preprint arXiv:2503.09567*, 2025b.
- Chen et al. (2024b)
Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et al.
Do not think that much for 2+ 3=? on the overthinking of o1-like llms.
*arXiv preprint arXiv:2412.21187*, 2024b.
- Cobbe et al. (2021)
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al.
Training verifiers to solve math word problems.
*arXiv preprint arXiv:2110.14168*, 2021.
- Deng et al. (2025)
Yang Deng, Moxin Li, Liang Pang, Wenxuan Zhang, and Wai Lam.
Unveiling knowledge boundary of large language models for trustworthy information access.
In *Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval*, pp. 4086–4089, 2025.
- Fan et al. (2024)
Siqi Fan, Xin Jiang, Xiang Li, Xuying Meng, Peng Han, Shuo Shang, Aixin Sun, Yequan Wang, and Zhongyuan Wang.
Not all layers of llms are necessary during inference.
*arXiv preprint arXiv:2403.02181*, 2024.
- Gao et al. (2023)
Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yixin Dai, Jiawei Sun, Haofen Wang, and Haofen Wang.
Retrieval-augmented generation for large language models: A survey.
*arXiv preprint arXiv:2312.10997*, 2(1), 2023.
- Guo et al. (2025)
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al.
Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.
*arXiv preprint arXiv:2501.12948*, 2025.
- Hastie (2009)
Trevor Hastie.
The elements of statistical learning: data mining, inference, and prediction, 2009.
- He et al. (2025)
Yancheng He, Shilong Li, Jiaheng Liu, Weixun Wang, Xingyuan Bu, Ge Zhang, Zhongyuan Peng, Zhaoxiang Zhang, Zhicheng Zheng, Wenbo Su, et al.
Can large language models detect errors in long chain-of-thought reasoning?
*arXiv preprint arXiv:2502.19361*, 2025.
- HMMT (2025)
HMMT.
Harvard–mit mathematics tournament: February 2025 problems archive.
[https://www.hmmt.org/www/archive/282](https://www.hmmt.org/www/archive/282), 2025.
- Jaech et al. (2024)
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al.
Openai o1 system card.
*arXiv preprint arXiv:2412.16720*, 2024.
- Kalai et al. (2025)
Adam Tauman Kalai, Ofir Nachum, Santosh S. Vempala, and Edwin Zhang.
Why language models hallucinate, 2025.
URL [https://arxiv.org/abs/2509.04664](https://arxiv.org/abs/2509.04664).
- Kale & Nadadur (2025)
Sahil Kale and Vijaykant Nadadur.
Line of duty: Evaluating llm self-knowledge via consistency in feasibility boundaries.
*arXiv preprint arXiv:2503.11256*, 2025.
- Kirichenko et al. (2025)
Polina Kirichenko, Mark Ibrahim, Kamalika Chaudhuri, and Samuel J Bell.
Abstentionbench: Reasoning llms fail on unanswerable questions.
*arXiv preprint arXiv:2506.09038*, 2025.
- Köppen (2000)
Mario Köppen.
The curse of dimensionality.
In *5th online world conference on soft computing in industrial applications (WSC5)*, volume 1, pp. 4–8, 2000.
- Kuratov et al. (2024)
Yury Kuratov, Aydar Bulatov, Petr Anokhin, Ivan Rodkin, Dmitry Sorokin, Artyom Sorokin, and Mikhail Burtsev.
Babilong: Testing the limits of llms with long context reasoning-in-a-haystack.
*Advances in Neural Information Processing Systems*, 37:106519–106554, 2024.
- Lewis et al. (2020)
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al.
Retrieval-augmented generation for knowledge-intensive nlp tasks.
*Advances in neural information processing systems*, 33:9459–9474, 2020.
- Li et al. (2024)
Moxin Li, Yong Zhao, Yang Deng, Wenxuan Zhang, Shuaiyi Li, Wenya Xie, See-Kiong Ng, and Tat-Seng Chua.
Knowledge boundary of large language models: A survey.
*arXiv preprint arXiv:2412.12472*, 2024.
- Ling et al. (2025)
Zhan Ling, Kang Liu, Kai Yan, Yifan Yang, Weijian Lin, Ting-Han Fan, Lingfeng Shen, Zhengyin Du, and Jiecao Chen.
Longreason: A synthetic long-context reasoning benchmark via context expansion.
*arXiv preprint arXiv:2501.15089*, 2025.
- Luo et al. (2024)
Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Meiqi Guo, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, et al.
Improve mathematical reasoning in language models by automated process supervision.
*arXiv preprint arXiv:2406.06592*, 2024.
- MAA (2023)
MAA.
Amc 2023 10a problems and solutions.
[https://artofproblemsolving.com/wiki/index.php/2023_AMC_10A](https://artofproblemsolving.com/wiki/index.php/2023_AMC_10A), 2023.
- MAA (2024)
MAA.
Aime 2024 i problems and solutions.
[https://artofproblemsolving.com/wiki/index.php/2024_AIME_I](https://artofproblemsolving.com/wiki/index.php/2024_AIME_I), 2024.
- MAA (2025)
MAA.
AIME 2025 I Problems and Solutions.
[https://artofproblemsolving.com/wiki/index.php/2025_AIME_I](https://artofproblemsolving.com/wiki/index.php/2025_AIME_I), 2025.
- Madhusudhan et al. (2024)
Nishanth Madhusudhan, Sathwik Tejaswi Madhusudhan, Vikas Yadav, and Masoud Hashemi.
Do llms know when to not answer? investigating abstention abilities of large language models.
*arXiv preprint arXiv:2407.16221*, 2024.
- Mukherjee et al. (2025)
Sagnik Mukherjee, Abhinav Chinta, Takyoung Kim, Tarun Anoop Sharma, and Dilek Hakkani-Tür.
Premise-augmented reasoning chains improve error identification in math reasoning with llms.
*arXiv preprint arXiv:2502.02362*, 2025.
- OpenAI (2025)
OpenAI.
Gpt-5 system card.
[https://openai.com/index/gpt-5-system-card/](https://openai.com/index/gpt-5-system-card/), 2025.
- Peng et al. (2025)
Keqin Peng, Liang Ding, Yuanxin Ouyang, Meng Fang, and Dacheng Tao.
Revisiting overthinking in long chain-of-thought from the perspective of self-doubt.
*arXiv preprint arXiv:2505.23480*, 2025.
- Phan et al. (2025)
Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Chen Bo Calvin Zhang, Mohamed Shaaban, John Ling, Sean Shi, et al.
Humanity’s last exam.
*arXiv preprint arXiv:2501.14249*, 2025.
- Pope et al. (2023)
Reiner Pope, Sholto Douglas, Aakanksha Chowdhery, Jacob Devlin, James Bradbury, Jonathan Heek, Kefan Xiao, Shivani Agrawal, and Jeff Dean.
Efficiently scaling transformer inference.
*Proceedings of machine learning and systems*, 5:606–624, 2023.
- Pu et al. (2025)
Xiao Pu, Michael Saxon, Wenyue Hua, and William Yang Wang.
Thoughtterminator: Benchmarking, calibrating, and mitigating overthinking in reasoning models.
*arXiv preprint arXiv:2504.13367*, 2025.
- Qian et al. (2025)
Chen Qian, Dongrui Liu, Haochen Wen, Zhen Bai, Yong Liu, and Jing Shao.
Demystifying reasoning dynamics with mutual information: Thinking tokens are information peaks in llm reasoning.
*arXiv preprint arXiv:2506.02867*, 2025.
- Qwen Team (2024)
Qwen Team.
Qwq-32b: A medium-sized reasoning model.
[https://huggingface.co/Qwen/QwQ-32B](https://huggingface.co/Qwen/QwQ-32B), 2024.
- Radford et al. (2019)
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al.
Language models are unsupervised multitask learners.
*OpenAI blog*, 1(8):9, 2019.
- Snell et al. (2024)
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar.
Scaling llm test-time compute optimally can be more effective than scaling model parameters.
*arXiv preprint arXiv:2408.03314*, 2024.
- Sui et al. (2025)
Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, et al.
Stop overthinking: A survey on efficient reasoning for large language models.
*arXiv preprint arXiv:2503.16419*, 2025.
- Sun et al. (2025)
Zhongxiang Sun, Qipeng Wang, Haoyu Wang, Xiao Zhang, and Jun Xu.
Detection and mitigation of hallucination in large reasoning models: A mechanistic perspective.
*arXiv preprint arXiv:2505.12886*, 2025.
- Vig (2019)
Jesse Vig.
A multiscale visualization of attention in the transformer model.
*arXiv preprint arXiv:1906.05714*, 2019.
- Vishwakarma (2024)
Swapnil Vishwakarma.
The curse of dimensionality in machine learning!
*Analytics Vidhya*, 2024.
- Wang et al. (2025)
Chenlong Wang, Yuanning Feng, Dongping Chen, Zhaoyang Chu, Ranjay Krishna, and Tianyi Zhou.
Wait, we don’t need to” wait”! removing thinking tokens improves reasoning efficiency.
*arXiv preprint arXiv:2506.08343*, 2025.
- Wen et al. (2024)
Zhihua Wen, Zhiliang Tian, Zexin Jian, Zhen Huang, Pei Ke, Yifu Gao, Minlie Huang, and Dongsheng Li.
Perception of knowledge boundary for large language models through semi-open-ended question answering.
*Advances in Neural Information Processing Systems*, 37:88906–88931, 2024.
- Wendler et al. (2024)
Chris Wendler, Veniamin Veselovsky, Giovanni Monea, and Robert West.
Do llamas work in english? on the latent language of multilingual transformers.
In *Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*, pp. 15366–15394, 2024.
- Xia et al. (2025)
Shijie Xia, Xuefeng Li, Yixin Liu, Tongshuang Wu, and Pengfei Liu.
Evaluating mathematical reasoning beyond accuracy.
In *Proceedings of the AAAI Conference on Artificial Intelligence*, volume 39, pp. 27723–27730, 2025.
- Xue et al. (2025)
Boyang Xue, Qi Zhu, Rui Wang, Sheng Wang, Hongru Wang, Fei Mi, Yasheng Wang, Lifeng Shang, Qun Liu, and Kam-Fai Wong.
Reliablemath: Benchmark of reliable mathematical reasoning on large language models.
*arXiv preprint arXiv:2507.03133*, 2025.
- Yang et al. (2025a)
Chenxu Yang, Qingyi Si, Yongjie Duan, Zheliang Zhu, Chenyu Zhu, Qiaowei Li, Zheng Lin, Li Cao, and Weiping Wang.
Dynamic early exit in reasoning models.
*arXiv preprint arXiv:2504.15895*, 2025a.
- Yang et al. (2025b)
Shu Yang, Junchao Wu, Xin Chen, Yunze Xiao, Xinyi Yang, Derek F Wong, and Di Wang.
Understanding aha moments: from external observations to internal mechanisms.
*arXiv preprint arXiv:2504.02956*, 2025b.
- Yao et al. (2025a)
Junchi Yao, Shu Yang, Jianhua Xu, Lijie Hu, Mengdi Li, and Di Wang.
Understanding the repeat curse in large language models from a feature perspective.
*arXiv preprint arXiv:2504.14218*, 2025a.
- Yao et al. (2025b)
Zijun Yao, Yantao Liu, Yanxu Chen, Jianhui Chen, Junfeng Fang, Lei Hou, Juanzi Li, and Tat-Seng Chua.
Are reasoning models more prone to hallucination?, 2025b.
URL [https://arxiv.org/abs/2505.23646](https://arxiv.org/abs/2505.23646).
- Yin et al. (2024)
Xunjian Yin, Xu Zhang, Jie Ruan, and Xiaojun Wan.
Benchmarking knowledge boundary for large language models: A different perspective on model evaluation.
*arXiv preprint arXiv:2402.11493*, 2024.
- Yuan et al. (2024)
Zhihang Yuan, Yuzhang Shang, Yang Zhou, Zhen Dong, Zhe Zhou, Chenhao Xue, Bingzhe Wu, Zhikai Li, Qingyi Gu, Yong Jae Lee, et al.
Llm inference unveiled: Survey and roofline model insights.
*arXiv preprint arXiv:2402.16363*, 2024.
- Zhang et al. (2025)
Anqi Zhang, Yulin Chen, Jane Pan, Chen Zhao, Aurojit Panda, Jinyang Li, and He He.
Reasoning models know when they’re right: Probing hidden states for self-verification.
*arXiv preprint arXiv:2504.05419*, 2025.
- Zhang et al. (2024)
Qingjie Zhang, Han Qiu, Di Wang, Haoting Qian, Yiming Li, Tianwei Zhang, and Minlie Huang.
Understanding the dark side of llms’ intrinsic self-correction.
*arXiv preprint arXiv:2412.14959*, 2024.
- Zhou et al. (2025)
Yang Zhou, Hongyi Liu, Zhuoming Chen, Yuandong Tian, and Beidi Chen.
Gsm-infinite: How do your llms behave over infinitely increasing context length and reasoning complexity?
*arXiv preprint arXiv:2502.05252*, 2025.

## Appendix A Datasets, Models, and Performance

In this section, we provide the specific description of models and datasets evaluated in our work. And we provide the performance of each model on each dataset.

We evaluate the following LRMs:

- $\bullet$
GPT-oss-20B (Agarwal et al., 2025): An open-source large reasoning model with 20B parameters, designed as a general-purpose backbone for multi-step reasoning tasks.
- $\bullet$
DeepSeek-R1-0528-Qwen3-8B (Guo et al., 2025): A distilled variant of DeepSeek-R1 based on Qwen3-8B, post-trained with chain-of-thought supervision. It achieves state-of-the-art performance among open-source models on mathematical reasoning benchmarks.
- $\bullet$
DeepSeek-R1-Distill-Qwen-32B (Guo et al., 2025): A 32B-parameter distilled model obtained by transferring reasoning abilities from DeepSeek-R1 into the Qwen2.5-32B backbone, yielding strong performance across multiple benchmarks.
- $\bullet$
QwQ-32B (Qwen Team, 2024): A medium-sized reasoning model from the Qwen series, competitive with other state-of-the-art reasoning-focused models on math and logical reasoning tasks.

We evaluate the following Math datasets:

- $\bullet$
AIME’25 (MAA, 2025): The 2025 American Invitational Mathematics Examination. It consists of 30 integer-answer questions with increasing difficulty, used as a challenging benchmark for mathematical reasoning.
- $\bullet$
AIME’24 (MAA, 2024): The 2024 American Invitational Mathematics Examination. Similar in format to AIME’25, it consists of 30 problems and provides a prior-year benchmark for evaluating model robustness across exam editions.
- $\bullet$
HMMT (HMMT, 2025): The Harvard–MIT Mathematics Tournament (February 2025 session). The dataset contains 30 questions sourced from the original competition, which were extracted, converted to LaTeX, and verified. These problems feature complex multi-step reasoning across various mathematical domains.
- $\bullet$
AMC’23 (MAA, 2023): The 2023 American Mathematics Competition (AMC 10A). It consists of 30 questions, providing a lighter but diverse benchmark compared to AIME and HMMT.
- $\bullet$
GSM8K (Cobbe et al., 2021): A large-scale dataset of grade-school math word problems requiring multi-step arithmetic reasoning. We evaluate on approximately 1.5k problems from the test set. It is widely used to evaluate the basic reasoning competence of LRMs, with accuracy typically exceeding 90%.
- $\bullet$
Humanity’s Last Exam (HLE) (Phan et al., 2025): A multi-modal benchmark at the frontier of human knowledge, designed with broad subject coverage across mathematics, humanities, and natural sciences. We use approximately 1.5k text-only questions (excluding image-based problems) from this dataset. Model accuracy on this dataset is typically below 10%, making it a strong stress test for reasoning beyond capability boundaries.

The performance of LRMs on these Math benchmarks are shown in the following [Table 4](https://arxiv.org/html/2509.24711v2#A1.T4):

**Table 4: Accuracy (%) of different LRMs on different Math benchmarks.**
|  | AIME’25 | AIME’24 | AMC’23 | HMMT | GSM8K | HLE |
| --- | --- | --- | --- | --- | --- | --- |
| GPT-oss-20B | $76.7$ | $70.0$ | $95.0$ | $46.7$ | $89.2$ | $6.1$ |
| R1-Distill-Qwen3-8B | $63.3$ | $76.7$ | $100.0$ | $60.0$ | $72.0$ | $3.5$ |
| R1-Distill-Qwen-32B | $40.0$ | $56.7$ | $82.5$ | $36.7$ | $90.8$ | $5.0$ |
| QwQ-32B | $70.0$ | $76.7$ | $95.0$ | $36.7$ | $92.7$ | $3.8$ |

## Appendix B System Prompt of BoostAbstention

We consider BoostAbstention (Kirichenko et al., 2025) which crafts a system prompt that encourages the model to abstain in designated scenarios, showing effectiveness for both reasoning and standard LLMs. For our experiments, we modify the system prompt to better match the abstention scenarios of questions beyond capability boundary:

## Appendix C Confident and Uncertain Expressions

[Figure 7](https://arxiv.org/html/2509.24711v2#A3.F7) shows the confident and uncertain expression that frequently appear in the reasoning process. The expressions are summarized manually and checked by LLM. In the experiment, when matching these expressions, the Uncertain Expression is matched first, followed by the Confident Expression. Additionally, both the position of the first token of an expression and the token length of the entire expression are considered.

Figure: Figure 7: Confident vs. uncertain expressions. These expressions are used to identify the confidence level in problem-solving processes, with manual summarization and LLMs verification.
Refer to caption: x7.png

## Appendix D Capability Boundary Revealed by Hidden States

[Figure 8](https://arxiv.org/html/2509.24711v2#A4.F8) visualizes the classification of solvable and unsolvable questions of all LRMs, along with the capability boundary (blue line) determined by the decision boundary of linear classifier (e.g., LDA). We observe that the two classes of questions are clearly separated, under random train–test splits. These results demonstrate that the features of capability boundaries are strongly encoded in the hidden states of the last input token, and is valid across different LRMs.

Figure: Figure 8: Capability boundary revealed by linear classifier (e.g., LDA) clearly separates solvable and unsolvable questions in hidden states.
Refer to caption: x8.png

## Appendix E Unproductive Reasoning

[Figure 9](https://arxiv.org/html/2509.24711v2#A5.F9) and [Figure 10](https://arxiv.org/html/2509.24711v2#A5.F10), demonstrate representative unproductive reasoning phenomena in large language models, including repetitive looping and error accumulation, which impair their capacity for sound and consistent reasoning in complex problem-solving scenarios. They respectively show how models get stuck in repetitive deductions ([Figure 9](https://arxiv.org/html/2509.24711v2#A5.F9)) and how errors compound during formula application and result derivation ([Figure 10](https://arxiv.org/html/2509.24711v2#A5.F10)). In mathematical problem reasoning, the former presents repeated deduction of the same uncertain conclusion, while the latter sees continuous error accumulation due to flawed initial methods.

Figure: Figure 9: An example of unproductive reasoning manifested as repetitive looping in LRMs when attempting to solve a mathematical problem, where the model repeatedly cycles through the same uncertain deductions without making progress.
Refer to caption: x9.png

Figure: Figure 10: An example of error accumulation in LRMs when tackling a problem on the expected number of regions in a disk, where the model accumulates errors due to an incorrect initial formula, leading to inconsistent results and a forced answer guess despite unresolved contradictions.
Refer to caption: x10.png

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 4
presentation: 2
questions: See “weaknesses” section
soundness: 2
strengths: - The paper is **well-structured, with all analyses and experiments clearly motivated and building upon each other naturally**. The initial study of reasoning expressions segues neatly into the analysis of input hidden states through the observation that capability boundaries are revealed already at an early stage of reasoning. Each of the two theoretical analyses are then subsequently utilized in practical test-time monitoring strategies.

- The paper **addresses a timely and important topic**: Do large reasoning models (LRMs) know their own capability boundaries?

- The **proposed test-time monitoring methods improve both reliability and token efficiency of open-source reasoning models** by having the model either outright abstain from providing a solution or cut short its reasoning process when the question is deemed outside its capability boundary. For example, the biggest reasoning model the authors studied, QwQ-32B, showed perfect abstention rate with “expression monitoring” and ~50% reduction in token usage on unsolvable problems, without any loss in accuracy (Table 3, last row).
summary: The paper investigates whether open-source reasoning LRMs (Large Reasoning Models) possess self-awareness of their own capabilities - in particular, whether there’s some signal either in their reasoning trajectories or hidden states that would correlate with answer accuracy. 

The authors find that by comparing the incidence rate of confident expressions  (e.g. “I am confident”, “this is solid”, “seems okay”, etc.) to those of unconfident expressions (e.g. “I’m stuck”, “not entirely confident”, “problematic”, etc.) in the models' reasoning trace, they’re able to predict with almost perfect accuracy whether the final answer is correct or incorrect on math benchmarks.

Motivated by their success of this approach even by analyzing only the beginning of the reasoning trace, the authors further investigate whether the LLM hidden states of the last input token are predictive of the correct answer. A logistic regression model is able to predict answer accuracy nearly perfectly across all the tested models.

Finally, the authors leverage both of these findings and propose two test-time monitoring strategies to either early-terminate the model’s reasoning based on the expression monitoring (black-box method), or have the model abstain from answering at all based on a classifier trained on the hidden state of the last input token (white-box method). Both strategies increase reliability and reduce token usage without sacrificing accuracy on the tested benchmarks (competition math, GSM8k, Humanity’s Last Exam).
weaknesses: - Throughout, the paper **lacks sufficient level of detail about the experimental setup**
  - The fundamental section of the paper is section 4, wherein the authors analyse the correlation between answer accuracy and incidence rate of various “reasoning expressions” in the reasoning trace. The paper doesn’t elaborate on how the specific expressions (provided in Appendix C) were chosen, nor provides any quantitative analysis of relative importance of individual reasoning expressions. Finally, the exact method by which the reasoning expressions are used to estimate confidence levels is left ambiguous (the authors do mention “manual summarization and LLMs verification”, but no further detail is provided).
  - Section 4 figure 4 - what sampling settings were used, what max sequence length for the output?
  - Section 5 figure 5 - how is  “close / far from the decision boundary” quantified?
  - In Section 6.1, the authors define the Hard Abstention (HA) metric, which “evaluates whether LRMs can explicitly abstain on unsolvable problems instead of producing incorrect answers”. However, the paper doesn’t specify how abstention is detected.

- In Section 6.1, the proposed test-time monitoring strategies are evaluated against a single, relatively weak baseline—BoostAbstention, a system prompt encouraging the model to abstain on difficult problems. This baseline yields a 0% Hard Abstention (HA) rate across all settings, providing little insight into how the proposed methods compare to realistic alternatives.
**The paper would benefit from the inclusion of stronger baselines to help better contextualize the proposed improvements**, such as model-as-judge or uncertainty-estimation approaches (e.g., Wen et al., 2025 for an overview).

- In section 6, table 3 reports that the 2k and 4k sequence length limits are exceeded by all analysed models in almost 100% of incorrect cases. If I’m reading this right, then the vanilla QwQ-32B produces no outright incorrect completions - in every case it didn’t arrive at a correct answer, it simply ran out of context. This suggests that the proposed monitoring strategies might primarily identify context-limited or under-budgeted reasoning chains rather than truly unsolvable problems - **the paper would benefit from extending the analyses to longer sequence lengths.**

- The evaluation results in Section 6 aggregate across six benchmarks, but Appendix A shows that roughly 96% of all evaluation problems (~3k / 3.12k) come from GSM8K and Humanity’s Last Exam (HLE)—two datasets that sit at opposite ends of the accuracy spectrum (≈ 90% vs < 10%).
Consequently, many reported metrics—particularly the Hard Abstention (HA) scores in Table 3—likely conflate benchmark difficulty with model “self-awareness.” In effect, the monitoring strategies may simply learn to distinguish which dataset a question originates from, rather than whether a question is intrinsically within or beyond the model’s capability boundary.
This same imbalance likely inflates the near-perfect linear-probe accuracies reported in Section 5, since solvable (GSM8K) and unsolvable (HLE) examples are trivially separable.
**The paper would benefit from reporting per-benchmark metrics (HA, token usage, overflow, etc.) to disambiguate the effects of benchmark distribution from genuine boundary detection.**

- In all experiments in the paper (e.g. section 5), the authors train and test on different subsets of the same datasets. **The paper could benefit from cross-dataset generalization tests (e.g., train on GSM8K, test on AIME).**

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1. Generalizability on coding tasks. The experiment mostly focuses on multiple choice questions or questions with numerical answers. Does it generalizes to coding questions? How would the methodology adapt to coding questions (and possibly more open-ended questions)?
2. Ablation on the self-aware output prefix. Does changing the output prefix alter the result?
soundness: 3
strengths: 1. Novelty. The analysis and methodology is novel and interesting. 
2. The paper writing is clear and concise. The evidence is presented very cleanly, and the logic flows very naturally.
3. Methodology and experiment. The methodology proposed is sound, intuitive, effective in practice, and backed by a lot of experiments and ablations to show its effectiveness.
summary: The paper first study the effect that LRM is self-aware of its capability boundary at reasoning process. The paper then propose two methods, reasoning expression monitoring and hidden states monitoring, to monitor black-box and white-box LLM's reasoning trajectory to determine if the model has hit its capability boundary for the given task. The evaluation shows impressive result of the two proposed methods to reduce token usage for hard (impossible) questions.
weaknesses: Related works. Entropy-based early-exit methods (especially training-free methods) has been quite extensively studied in prior works such as the following. The paper may want to expand a bit more to make a clearer distinction between the proposed methodology and the prior works (in the related work). Just listing a few as addition to the current related work section:

- https://arxiv.org/abs/2504.01296
- https://arxiv.org/abs/2508.15260
- https://arxiv.org/abs/2412.20993
- https://arxiv.org/abs/2412.18547
- https://arxiv.org/abs/2207.05221

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: N/A
soundness: 3
strengths: First, this paper addresses an important research problem whose significance is widely recognized. Its biggest strength in my opinion is the clarity of presentation along with supporting results. I would recommend it to anyone interested in understanding the current state of research on this topic.
summary: This paper mainly focuses on a widely acknowledged problem: whether LLMs know they do not know. Specifically, it proposes solutions for both black-box LLMs and open-source LLMs. Downstream applications indicate that the proposed approach can improve the LRM efficiency without sacrificing accuracy.
weaknesses: I am concerned that this paper does not offer substantial additional value beyond several existing works. For example, [1] (and lots of other work including some surveys) presents similar approaches to probing the hidden states of LRMs, also supported by strong empirical results. For this reason, although I like the paper, I do not believe we need to have another paper with similar contributions.

In addition, I think it would be interesting to focus on scenarios where LLM has reasonable accuracy. As the paper mentioned, GSM8K is too easy, and HLE is too difficult. It's not that surprising LLMs have "self-awareness". 


[1] Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification COLM 2025

---

confidence: 2
contribution: 3
initial_rating: 6
presentation: 3
questions: - Can you show the hidden-state separability within a single dataset and difficulty-controlled splits, with train and test drawn from the same distribution, and report how the accuracy/AUC changes?

- How sensitive are the black-box signals to prompt style, language, and removal of common cue phrases, and do the effects hold under alternative chain-of-thought formats or shorter reasoning budgets?

- Under matched compute, how do your methods compare to stronger uncertainty and verifier-guided strategies, and can the black-box-only variant deliver similar gains when white-box access to hidden states is unavailable?
soundness: 3
strengths: - The paper frames "capability boundaries" clearly and turns that idea into two simple, test-time strategies that are easy to implement.

- Evidence comes from both black-box signals (confidence/uncertainty trajectories) and white-box signals (last-token hidden states), which makes the story feel cohesive.

- Reported token savings are large without hurting accuracy, suggesting real efficiency wins for math-style workloads.
summary: This paper claims that LRMs can excel at math but often overthink hard questions, wasting tokens and still getting them wrong.
This paper tests whether models know their own limits: in black-box use, confidence/uncertainty trajectories in the reasoning text signal solvable vs. unsolvable cases; in white-box use, the last-input hidden state linearly separates them before any reasoning.
Leveraging these signals, the authors monitor reasoning expressions or hidden states to stop unproductive reasoning early.
They report large token savings (62.7–93.6%) without hurting accuracy.
weaknesses: - The striking linear separability in hidden states may reflect dataset or difficulty splits rather than genuine boundary awareness, so the claim risks a distribution confound.

- The black-box indicators depend on specific reasoning styles and cue phrases, which may not hold under different prompts, languages, or chain-of-thought formats.

- The evaluation is narrow and misses stronger uncertainty/abstention baselines and cross-model tests, and the white-box path raises deployability concerns where internal states are inaccessible.

---

confidence: 4
contribution: 3
initial_rating: 2
presentation: 2
questions: - The authors use Figure 3 to support a relation between expression density curve's concavity and the answer correctness However, it seems that in both correct and wrong answers, the confident expression density is always concave and the uncertain expression density is always convex.
- What is the exact number of y-axis? Is it #expression per token of the stage, or something alike?
soundness: 2
strengths: - The perspective of understaing model capacity through certainty trajectory's concavity is novel;
- The authors design comprehensive experiments on multiple models and benchmarks, supporting a universality of their findings.
summary: This paper studies if reasoning models is able to be aware of its capability boundary. The authors provide two methods: a black-box confidence keywords detection, and a white-box hidden states classification.
weaknesses: - The paper is in lack of some important details, making its phrasing not precise:
  - What is the definition of capability boundary? For example, it is confusing in Figure 3 that the capability boundary is the straight line connecting reasoning stage 0% and reasoning stage 100%.
  - How are the confident expressions and uncertain expressions defined? Although Appendix B provides a list of words, but is there any rule to construct such a list? e.g. assume a multilingual language model, how to construct the list for other languages?
  - In eq.1, how is the "model expresses more confidence/uncertainty defined"? For example, if a model solves a problems by several steps, and is only confident at the end of the reasoning process, is it considered more confident, or more uncertain?
  - In section 4, the detail how reasoning process is divided into stages is unclear. 
  - What is the training set of the LDA and LR? For a given token budget, will the training set also be constrained under that budget, or is free to use as much token as possible?
- Table 3 shows the primary end-to-end experiment result for the two methods. However, it will be more supportive if the authors can provide an average answer length without any overflow constraint. As the models almost always overflow, 2K/4K-token budget may not be a reasonable choice, because the baseline method (no monitor), due to the lack of enough tokens in reasoning, could show a poor performance.
- The paper can benefit from comparing with some other confidence based early existing methods to show its speedup. Since the main metric reported is to reduce the overflow ratio, comparison with existing approaches solving overthinking is important.


Decision: Reject