$\textbf{Re}^{2}$: Unlocking LLM Reasoning via Reinforcement Learning with Re-solving
Authors: Pinzheng Wang
ShuliXu
Juntao Li
Yu Luo
Dong Li
Jianye HAO
Min Zhang
Venue: ICLR 2026 Poster (2026)
Keywords: LLM Reasoning
Reinforcement learning
Re-solving Mechanism
Test-time Scaling
Primary area: generative models
TL;DR: We enable LLMs to restart the solution process via reinforcement learning when reasoning paths are unproductive, leading to improved performance.

Abstract: Reinforcement learning with verifiable rewards (RLVR) has shown promise in enhancing the reasoning performance of large language models (LLMs) by increasing test-time compute. 
However, even after extensive RLVR training, such models still tend to generate unnecessary and low-quality steps in their chain-of-thought (CoT), leading to inefficient overthinking and lower answer quality.
We show that when the initial direction or quality of the CoT is suboptimal, the model often fails to reach the correct answer, even after generating several times more tokens than when the initial CoT is well-initialized.
To this end, we introduce $\textit{\textbf{Re}inforcement Learning with \textbf{Re}-solving}$ (Re$^2$), in which LLMs learn to flexibly abandon unproductive reasoning paths and restart the solution process when necessary, rather than always committing to a final answer.
Re$^2$ applies pure reinforcement learning without any preliminary supervised fine-tuning, successfully amplifying the rare redo behavior in vanilla models from only 0.5\% to over 30\%.
This leads to substantial performance gains over standard RLVR under the same training compute budget, and also demonstrates notable improvements in test-time performance as the number of samples increases.

Published as a conference paper at ICLR 2026
Re2: UNLOCKING LLM REASONING VIA REINFORCE- MENT LEARNING WITH RE-SOLVING
Pinzheng Wang' Shuli Xu Juntao Li Yu Luo? Dong Li Jianye Hao Min Zhang! "School of Computer Science and Technology, Soochow University *Department of Foundation Model. 2012 Labs, Huawei
Cooleeee of Intelligence and Computing, Tianjin University (pzwangl, s slxu) estu.ssusu.. .edd.eddu.cm (1jt, minzhang) é sudaa edu. cn
(luoyu81, 1 lidong100 06) @huawei. com jianye. hao@t ju. edu. en
ABSTRACT
Reinforcement learning with verifiable rewards (RLVR) has shown promise in en- hancing the reasoning performance of large language models (LLMs) by increasing test-time compute. However, even after extensive RLVR training, such models still tend generate unnecessary and low-quality steps in their chain-of-thought (CoT), leading to inefficient overthinking and lower answer quality. We show that when the initial direction or quality of the CoT suboptimal, the model often fails reach the correct answer, even after generating several times more tokens than when the initial CoT is well-initialized. To this end, we introduce Reinforcement Learning with Re-solving (Re2), in which LLMs learn to flexibly abandon unproductive reasoning paths and restart the solution process when necessary, rather than always committing to final answer. Re² applies pure reinforcement learning without any preliminary supervised fine-tuning, successfully amplifying the rare redo behavior in vanilla models from only 0.5% over 30%. This leads to substantial perfor- mance gains over standard RLVR under the same training compute budget, and also demonstrates notable improvements in test-time performance as the number of samples increases.
(a) Train-Time Scaling AIMEZ5
(b) Test-Time Scaling on AIME25
0.200
DAPO
0.30
DAPO (Majority voting)
Rx2
0.25
Re²
Accuracy
0.150
0.125
0.15
0.100
0.10
2'
27
21
2
21
Train Stops
Number of Samples
Figure 1: (a) Accuracy improvements of DAPO and Re Qwenn2.5-7B-Instruu each training step. with comparable numbers of generated and trained tokens per step. (b) Test-time scaling of DAPO and Re under the same training budget, as the number of samples increases.
INTRODUCTION
Recent studies have shown that large language models (LLMs) can achieve strong reasoning abilities through scaling test-time compute (Snell et al. 2024 Wu et al. 2025). By generating longer chains of thought (CoT's) that incorporate planning, reflection, and self-correction, LLLMs attain higher accuracy on complex reasoning tasks such as coding and mathematics (Yang et al. 2025a Bercovich et al.)
"Corresponding author

---

Published as a conference paper at ICLR 2026
Low quality or moddracted iniaal seps
Answer
final
(c) Example Trace DAPO and Re
Hiah-cualiy intiial will coneet dieeciinn
Ausver
firel
AIME 24
Protdcm: Laid te remeuate hour wit oxtie vakuso Let
df othrr conserord df returga lar bases the mndeenated S The vake
Stepi
S
Stepp
Stepe
Stepd
Stepe
and colatttatlyy
Stept
Step
Sugp
Steps
Steps
Atswer
Staning Reasoring Trase of DAPO
Priblem
The a 4 (a-
SS 23
of oe
20gh
Step1
2
Step3
Asswer
54 imambte.
-0044
a Laing AM--G innquality 2.2 INE A opailty SVE5.
Step1
3
Step4
Ansver
fuca (3 237 50y/4. srenitiing leo the famslee.
wilie beta Ausrcc shoucs . (50) (Meong Arnowr)
(o) Standard Reasoning
Overall accurncy: 0.5
[foran Tradia AM. cCM wae bondls
Heasoning Truee of it?" Stating
Step1
Step2
Skep3
Reatking
Restarting from scratch
Firt AMS-GM
SVS5
conditerr fee the insuecttyy only drier sushilty oimeeress vakor ats
Stepl
Reselong
Restarting from scratch
Saveu corplicee
apprrdrates
Problam
better
Stepl
Step2
3
Amswer
Restarting scrudds
Janasing 1. alc B-axte S. Hugdre laso ar- 27
22
Step1
Step2
Step3
Step4
Auswer
Found insgrrment o Then a 2 anteett bohhsrffon are and edure cactly.
(b) Reasoning with Re-solving
Overall sccuracy:
Fis inoerr pig 657 6e
(Conrct Arswer)
Figure 2: Illustration of reasoning with re-solving. (a) In standard reasoning, when the initial steps are suboptimal, the LLM struggles to reach the correct answer even after generating more reasoning steps and tokens. (b) Reasoning with re-solving, allowing the model to abandon an unpromising path and restart at any point, leads to a higher overall accuracy. (c) Example trace of DAPO and Re²: Both models initially attempt incorrect approach using the AM-GM inequality; however, Re² detects the failure, restarts, and arrives at the correct answer.
2025 Team et al. 2025 Wu et al., 2024). To this end, state-of-the-art models adopt reinforcement
learning (RL) post-training, which has proven effective in producing longer CoTs and strengthening deep-thinking capabilities (Guo et al. 2025 Shao et al. 2024 Yu et al. 2025
However, even with extensive RL training, LLMs still suffer from issues such as overthinking (Chen
it al.] [024 Cuadron et al. (025) and underthinking (Wang et al. 2025| Raasire et al. 2025 Ding et al. 2025p, generating unnecessary or low low-quality reasoning steps that degrade both efficieccy and overall performance. In this paper, we investigate the limitations of test-time scaling in existing LLMs by analyzing the correlation between CoT length and accuracy, together with the impact of early reasoning quality on final outcomes. Our analysis reveals that when the initial reasoning steps are suboptimal, LLMs struggle to reach the correct answer, even after generating significantly more reasoning steps and tokens (Figure 2 aa). However, solving complex problems often requires early exploration, during which the model may pursue directions that initially appear promising but ultimately lead to dead ends or errors. Motivated by these findings, we aim enable models restart the reasoning process when the current path proves unproductive, thereby improving overall accuracy by escaping unpromising paths (Figure 2 b)).
To equip models with this capability, we introduce Reinforcement Learning with Re-solving (Re?), a novel framework that equips models with the ability to either finalize an answer or re-solve the problem. During training, the model extends partial reasoning trajectories and learns to decide whether to continue or to restart from its current progress. If the model chooses re-solve, its trajectory receives a reward equal to the expected success rate of solving the problem from scratch. If it instead produces a final answer, the reward is assigned as for correct solution and O otherwise. Under this scheme, when the current reasoning prefix is in the wrong direction or of low quality, abandoning and re-solving the problem more likely to yield higher accuracy and thus a larger reward. Conversely, when the reasoning trajectory is promising, directly producing the final answer leads to a higher expected reward. Notably, Re² requires no preliminary supervised fine-tuning. Through pure reinforcement learning alone, it amplifies the rare redo behavior in vanilla models from just 0.5% to over 30%, thereby enabling models to flexibly decide when to re-solve and ultimately leading to more accurate reasoning, illustrated in Figure Cc).
We evaluate Re² on a diverse set of reasoning benchmarks, including AIME 2024, AIME 2025 (MAA Committees). AMC 2023 (AL-MO) 2024), GSM8K (Cobbe et al. 2021), and GPQA-
2

---

Published as a conference paper at ICLR 2026
Diamond (Rein et al.) 2021b, covering wide range of domains and difficulty levels. Our evaluation covers five models ranging from 3B to 14B parameters, including pre-trained, instruction-tumm and reasoning models. Experimental results demonstrate that our approach achieves significant improvements in reasoning performance compared to recent RLVR methods such as DAPO (Yu al. 2025) under the same training budget. Moreover, Re yields a superior trade-off curve between lest-time compute and performance compared to majority voting (Wang et al., 2022). We believe our work highlights the promise of integrating RL with a new paradigm of re-solving reasoning, which goes beyond the traditional single-chain approach and opens up new directions for developing more flexible and reliable reasoning in LLMs.
2
RELATED WORK
LLM for reasoning. Existing approaches enhance the reasoning capabilities of LLMs through
prompt engineering (Wei et al. 2022 Yang et al. 2023), supervised fine-tuning Yang et al. 2024a
Qin et al., 2024), and reinforcement learning (Schulman et al. 2017 Shao et al. 2024 Zheng et al.
2025 Yue al. 2025; Zhang al. 2025 Wang al. 2024). Among these, reinforcement learning
with verifible rewards (RLVR) has emerged as mainstream paradigm for bost-training optimization,
encouraging models to produce longer CoTs with planning and self-reflection, thereby pushing the frontier of reasoning performance [OpenAl, 2024| Qu et al. 2024] Gandhi et al.) 2024] ZZng et al. 2025). Nevertheless, even after extensive RLVR training, LLMs remain prone to overthinking and underthinking, leading to redundant or low-quality reasoning steps (Chen et al.] 2024 Cuadron et al. 2025; Wang et al. 22025]
caassta-ammiezzzz et al., 2025). This remains a fundamental limitation of the prevailing paradigm, in which the model generates single CoT trajectory and ultimately derives its final answer within that trajectory (Wen et al. 2025 Shojaee et al.) (025). Recent studies (Yang) al. 2025b Fu et al. 2025) have attempted to address this issue by backtracking earlier steps or lerminating low-confidence reasoning chains, but these methods are limited to supervised fine-tuning or decoding strategies and do not leverage the potential of RL. To the best of our knowledge, our work is the first to propose a reasoning paradigm that allows models to abandon unproductive reasoning paths and re-solve problems from scratch through reinforcement learning.
Recent studies have shown that LLMs can effectively improve reasoning per- formance by increasing inference-time compute (Snell et al. 2024 Welleck et al. [024]] Wu et al.)
Test-time scaling.
2025 Muennighoff et al.
exemplified by OpenAl's 01, learn to produce traces that are longer than the typical solutions via SFT or RLVR (OpenAl 2024 Qin
2025 Large reasoning models,
al. 2024 Guo et al. 2025 Zhao et al.] (024). Some works improve performance by allowing models to iteratively revise their answers through multiple rounds of self-correction (Xiong et al. 2025 Zhao al. [025| Xi et al. 2024) Paul et al.||202 2024 Yang et al. 2024b). In addition, parallel sampling methods (e.g.. majority voting (Wang et al. [ Wan et al. (024) and tree search (Hao) ct al. 2023 Zhang et al. 2024)) further improve performance by increasing the number of samples. In contrast, our approach scales test-time compute by enabling the model to abandon unpromising reasoning trajectories and re-solve the problem when necessary. thereby unlocking the potential of test-time scaling for reasoning.
DIFFICULTY OF RECOVERING FROM SUBOPTIMAL EARLY REASONING
To gain a deeper understanding of the limitations of test-time scaling in existing LLMss, we first analyze the correlation between CoT length and reasoning accuracy (Section 3.I.. and further investigate why LLMs fail even when they having sufficient capabilities (Section 3.2). In our experiments, we select Qwen2.5-7B-Inntrruee (Yang et al. 2024a) a representative instruction-tuned LLM and DeepScaleR 1.5--Previww (Luo et al. 2025) as a representative long-CoT reasoning model extensively trained with RLVR. We evaluate them AMC23 and AIME25, respectively, which aligns the difficulty of datasets with the capabilities of each model.
3.1 CORRELATION BETWEEN RESPONSE LENGTH AND PERFORMANCE
Although training models to produce longer CoTs can significantly improve reasoning performance, aim examine whether, for the same problem across multiple samples, longer CoT's actually lead
3

---

Published as a conference paper at ICLR 2026
(ween:3-7B-instruct en AMC23
Deepscaler 1.5b on AIME25
9:14
Propartion
Proportion
8:12
Accuracy
Recurarcy
0.33
Accuracy
0.00
saz
<6u
0.00
<2.0
4.
12.00-14
16.81774
Response lengths
MW
Figure 3: Correlation between CoT length and reasoning performance. The bars represent the proportion of cases within different COT length intervals, while the line indicates the average accuracy for responses in each interval. The large plots in the top row show the correlation across the entire dataset. The small plots in the second row illustrate the correlation at the level of individual problems.
to higher accuracy. To this end, we analyze the correlation between response length and accuracy by sampling 128 responses problem. As shown in Figure our results reveal clear negative correlation between CoT length and accuracy, both across the entire dataset and at the level of individual problems of varying difficulty. We further conduct case analysis of CoTs with different lengths for the same problem, which shows that longer responses are typically caused by early critical mistakes, such as following the wrong solution path overanalyzing the problem's 's assumptions, making recovery unlikely regardless the number of additional tokens generated. Additional results provided in Appendix
3.2 IMPACT OF INITIAL REASONING QUALITY ON FINAL ACCURACY
As shown in Section 3.1 shorter CoTs are typically more likely to be correct for given problem. We hypothesize that this is because suboptimal early reasoning leads the model to generate longer responses, yet recovery is still challenging. To illustrate this phenomenon, we design an experiment in which we truncate different proportions (20%, 40%, 60%, 80%) of incorrect responses and prompt LLMs to continue reasoning from these prefixes. We then measure the average accuracy of completions from each truncated prefix and compute the relative drop accuracy compared to reasoning from scratch on the original problem. As shown in Figure[4] for each prefix length we report the number of cases where the relative drop in accuracy exceeds the 25% 75% threshold ("All Drops"). The results show that as prefix length increases, the relative drop becomes larger, indicating that the longer model continues along an incorrect trajectory, the more likely is fail.
Furthermore, we investigate how early such performance degradation begins in an incorrect response. For each response, we record the shortest prefix at which the relative drop first exceeds the threshold ("First Drops"). The results reveal that for most incorrect responses, a significant drop in accuracy already occurs when only the first 20% of the response is used as the prefix. This demonstrates that once early reasoning is misguided, the model rarely recovers and struggles to return to the correct reasoning path.
3.3 TAKEAWAYS
Based on the above analysis, we conclude that: (1) For given problem, shorter responses following smoother reasoning tend to achieve higher accuracy, whereas longer responses are often associated with lower accuracy; (2) The quality of early reasoning process is crucial for the final accuracy.
RE2: REINFORCEMENT LEARNING WITH RESOLVING
The above results highlight that a coherent reasoning process and high-quality early reasoning are crucial for model performance. However, when tackling challenging problems, it is often unavoidable
4

---

Published as conference paper at ICLR 2026
Prefix Accuracy Drops of Sween25--8-IIntruuee on AMC23
Prefix Accuracy Drops of Deepscaler on AIME25
Relative drop 25%
Relative drop 75%
Relative drop 25%
Relative drop > 75%
200
All Drage
Droge
100
Brage
All Drage
Count
100
50
0.2
0.4
0.6
0.8
0.2
0.4
0.6
0.8
0.2 0.4
0.6
0.8
0.2
0.4
0.6
0.8
Preff:: Truncation Ratio
Preff: Truncation Ratio
Proftx: Truncation Ratio
Preffx Truncation Ratio
Figure 4: Number of cases where the relative drop exceeds the threshold when continuing from different proportions of incorrect response prefixes, compared to reasoning from scratch. "All Drops" counts all such cases each prefix proportion, while "First Drops" records the earliest prefix for each response where the drop exceeds the threshold.
group
COT,
Mewig
Au
COT prefix;
Policy model
R2
Group computation
COT
Nutt
R,,
Aie
computation groupe"
group 2
R:r
Query
Policy model
COT prefix,
Policy model
CDTTa
Cennel
computation
A2
Ru
Ase
Avg secumaty of groups,
*** group
Iodi
COT prefix,
Aa
Palicy model
R,
Group computetion
COT,,,
R,,
Correct final answer
Incorrect final answer
Choose to retry
A Advantage
Figure 5: The framework Re². For each query, Re² samples multiple prefixes, then generates multiple continuations for each prefix. The advantage is calculated within each group, while the out-of-group accuracy is used as the reward for the redo action.
for models to explore early reasoning paths that initially appear promising but ultimately fail to yield correct solutions. L.MSs also prone generating low-quality content during the initial stages of reasoning. Similar to how humans may reconsider their strategy when solving difficult problems, we aim equip models with the ability restart the reasoning process when the current trajectory appears unpromising or leads to confusion. To this end, we propose Rea, which leverages reinforcement learning to train models to flexibly re-solve problems during reasoning.
4.1 PREFIX GROUP GENERATION
Recent RLVR methods such GRPO (Shao al. 2024 and DAPO YYu al. 2025) improve pass@1 accuracy by sampling multiple reasoning trajectories in parallel for each query and rewarding only those that yield correct answers. contrast, Re² aims to improve the final answer quality by allowing the model to rationally abandon an ongoing reasoning path and restart from scratch when the trajectory appears confusing or unpromising. Meanwhile, our method requires estimating the success rate of re-sol re-solving from scratch, which provides the reward signal that guides the model to learn whether to redo or to commit to a final answer. To this end, we first sample n full responses for each query. Each response is randomly truncated proportion uniformly drawn from (0, 0.8). producing n diverse prefixes that serve as intermediate reasoning states. For each prefix, the model generates CoT continuations, and all continuations derived from the same prefix grouped together for subsequent advantage calculation, as illustrated in Figure5] To incentivize the resolve capability of the base model, design specialized prompting strategy, described in Appendix A
5

---

Published as a conference paper at ICLR 2026
4.2 REWARD STRATEGY WITH RE-SOLVING
To encourage the model to rationally abandon unpromising reasoning paths and restart from scratch when necessary, we design reward strategy that explicitly incorporates the option re-solve. For the j-th COT continuation of the i-th prefix Pre,, denoted as Oij. there are three possible outcomes C..: providing a correct final answer (Ct, correct), providing an incorrect final answer (Oa, = incorrect), or choosing to re-solve the problem (Ct.j = resolve).
For continuations that yield final answers, the reward assignment follows standard RLVR: the model receives a reward of for a correct answer and 0 otherwise. For continuations that choose to ro-solve, the reward given by the expected accuracy of re-solinng from scratch, estimated using out-of- group CoT completions, i.e. completions whose prefix is not Pre; Specifically, let Pas(correct (riiinoorrs and (ri ve) denote the empirical probabilities of correct, incorrect, and resolve outcomes among the (n 1) out-of-group continuations. When at most R redo rounds are allowed, the reward is given by
if C.. = correct
0,
if C.j = incorrect
T
1- (rrerssoiv))"" (c
if C., = resolve
" (resooive
This three-wat way reward strategy encourages the model continue reasoning when the current trajectory is promising, and to re-solve when the trajectory is confused or flawed, since the expected accuracy of re-solving exceeds that of continuation.
4.3 ADVANTAGE COMPUTATION AND PARAMETER UPDATE
Algorithm Rea:
Reinforcement Learning with Resolving
Input initial policy model тæ; task prompts D; maximum training steps s; number of prefixes n; number of continuations m; max resolve rounds R; clipping thresholds Eiou: Enigh' update steps per batch 1
for step= 1,... do
1:
Sample a mini-batch D, from D
For each question q E Dú, sample responses from Te (: q) and truncate them random ratios to form prefixes )
For each prefix Pre,, sample m continuations {Or,) Tesl ( g. Prefix,)
4: Compute rewards {ra.}} according to Eq.
Filter out degenerate groups all {ri,j {rag}y= are identical
6: Compute group-wise advantages Ai,; for each continuation according to Eq. for iteration = !....//! do Update the policy model me by maximizing JRea(O) (Eq. Update the old policy TBae +- Te
Output updated policy we
After computing rewards under the re-solving strategy, we compute group-wise advantages and update the policy parameters following DAPO (Yu et al. 2025). Specifically, the advantage for the j-th continuation O,, prefix Pre, is defined as the reward normalized by subtracting the group mean and dividing by the group standard deviation:
Tse mean (
A
std sd((fr.,) )=
If all continuations within group yield the same outcome (all correct, all incorrect, all choosing resolve), then A., and the gradients vanish. Such degenerate groups are filtered out during
'The detailed derivation is provided in Appendix B

---

Published as a conference paper at ICLR 2026
training. The computed advantage is then broadcast to all response tokens of the corresponding continuation.
Formally, for each query g D, first sample prefixes [Pre; Tea (- gq) For each prefix Pre,, we then sample m continuations {Oi, 10 ~ Teml qq, Pre). The optimization objective is:
TREe (0) = BigrVD, (Press 1 t1) Pl..1
1 7uTE
m
1
10
iif # EJ And clip
"
1 Elows Ehigh ) Ans )}
min
(3)
where x/dit (Oij. g, Pre;, Ocg.<t) denotes the conditional probability of the t-th token in continuation Oz, given the query & and the prefi Pre;- The models Ti and Teas correspond to the training policy and the sampling policy, respectively. The clipping thresholds Elow and Ehigh are hyperparameters used to bound the importance sampling ratio for stable optimization. The overall training algorithm summarized in Algorithm
EXPERIMENTS
5.1 EXPERIMENTAL SETUP
Training datasets. We construct our training set from the DAPO-Math-17K dataset (Yu al.. 2025). which is collected from AoPslaan official competition sources. The dataset covers wide range of mathematical domains and contains 17K problems in total. To ensure reliable rule-based reward signals and minimize parsing errors, all answers are transformed into integers.
Baselines & Models. We compare Re² against the vanilla model (before RL training) and the recent RLVR method DAPO (Yu et al., 2025), which follow for advantage computation and parameter updates. To ensure a fair comparison, both methods are trained with the same amount of generated tokens during RL optimization. To evaluate the effectiveness of Re² across model types and scales, we conduct experiments on both base and instruction-tuned LLMs, including Qwen-7BB -a-n-B Qwen-14B 14B-1a4aa BBsee Llama-3.2. 1.mma-3.2-3B-B1 3.2-retInttr and Qwen2.5-7B-Insse 2024a), We further evaluate on DeepSeek-RR PrepSeek-R1-iistinee --Dstitll-Llama (Guo et al. 2025) a reasoning model specifically finetuned to generate long chains of thought.
(Dubey al. 2024 Yang et al.,
Benchmarks. To comprehensively evaluate the reasoning ability of our model, We adopt five widely used benchmarks covering diverse difficulty levels and domains: AIME 2024 (MAA Committees) contains 30 challenging problems from the 2024 American Invitational Mathematics Examination. The exam is designed to test advanced rroliig- skills across algebra. geometry, combinatorics, number theory, and probability, and is often used as challenging benchmark for evaluating reasoning ability in language models. AIME 2025 (MAA Committees) follows the same format AIME 2024, with 30 comparably difficult problems. As the most recent edition, reduces the risk of contamination from pretraining post-training data. AMC 2023 (AI-MO) 2024) consists of 40 problems covering algebra, geometry, number theory, and combinatorics. Compared to the AIME benchmarks, its difficulty level is relatively lower. GSM8K (Cobbe et al. 2021) is curated dataset of 1,319 elementary- math word problems. Each problem typically requires two to eight reasoning steps, primarily involving multi-step arithmetic. making it a standard benchmark for assessing fundamental mathematical reasoning. GPQA (Rein et al.) 202T) is a challenging dataset of graduate- level questions in physics, biology. and chemistry, where even PhD-level domain experts achieve only around 69, 7% accuracy. In our experiments, we use the highest-qualin subset, GPQA-Diamont which consists of 198 carefully selected questions designed provide a rigorous test of advanced scientific reasoning.
Training and evaluation details. For Re², we use a learning rate of 1 x 10-6, Each training step processes batch of 32 queries, with = 8 prefixes sampled per query and 8 continuations generated for each prefix. The maximum sequence length is set to 8192 tokens. The clipping
7

---

Published as a conference paper at ICLR 2026
Table Experimental results on five reasoning benchmarks. Re² consistently improves the overall reasoning performance of each model over DAPO (p-value < 0.05). Red numbers in parentheses indicate performance gains relative to DAPO.
Models
Methods
AIME24
AIME25
AMC23
GSM8K
GPQA
Avg
Base Model
DAPO
11.9
10.3
64.7
91.8
29.7
41.7
Qwen2.5-7B Base
+ Re
17.1
19.0
70.8
93.6
36.8
47.5 (+5.8)
+ DAPO
18.2
15.7
64.0
94.3
44.8
47.4
Qwen2.5-14B Base
+ Re²
28.5
23,4
68.5
94.6
49.6
52.9 (+5.5)
Instruct Model
None
6.2
0.4
23.0
67.2
2.7
19.9
Llama3.2 --ssruuttt
+ DAPO
15.0
0.5
32.3
80.4
20.7
29.8
+ Re²
17.7
2.8
38.4
83.2
20.2
32.5 (+2.7)
None
11.4
7.5
51.4
85.3
33.4
37.8
Qwen2.5-7B-Instmul
+ DAPO
16.0
8.6
62.3
92.6
35.4
43.0
+ Re²
18.6
21.2
64.7
94.1
38.4
47.4 (+4.4)
Reasoning Model
None
39.3
27.3
84.3
88.6
36.9
55.2
DeepSeck-R1 Distill-Llama-Be
+ DAPO 38.4
26.5
86.9
89.6
38.4
55.9
+ Re²
47.2
29.6
88.7
92.2
44.8
60.5 (+4.44)
parameters are fixed Elow 0.2 and Ehghh 0.28, and the maximum number of redo rounds is R = 5. For DAPO, adopt the same learning rate of 1 10-6 To ensure comparable token budgets with Re", each batch contains 128 queries with n = samples per query. All other hyperparameters are kept identical to those used in Re². During evaluation, the maximum sequence length is increased to 16384 tokens, with sampling performed using a temperature of 0.6 and top-p of 0.95. For models trained with Re², whenever sampled completion produces redo action, sampling is restarted until a final answer is generated, and the first valid final answer is taken as the model's output.
5.2 MAIN RESULTS
As shown in Table 1 Re² improves reasoning performance across all five benchmark datasets and five model types, including base, instruction-tuned. 14B parameters. On pretrained models such as Qwen2.5-7B and Qwen2.5-14B, our method achieves larger gains compared to DAPO. These consistent gains on in-domain mathematical benchmarks of varying difficulty (AIME24, AIME25, AMC, GSM8K) well the out-of-domain scientific reasoning benchmark since AIME25 released after all the evaluated models were trained, it is free from potential data contamination, and Re² achieves superior performance on this benchmark, further validating its effectiveness. Notably, Re achieves substantial improvements on AIME24 and AIME25, highlighting its effectiveness in tackling more challenging reasoning problems.
ceasoning-optimizeed models ranging from 3B to
demonstrate the robustness of our approach. Moreover,
5.3 PERFORMANCE UNDER TEST-TIME SCALING
When tackling challenging problems (e.g., the AIME series), models trained with Re² may perform multiple redo attempts and generate several candidate solutions before producing a final answer, thereby consuming more tokens during inference. To fairly assess the effect of this additional token usage, we compare DAPO and Re² under the same number of sampled outputs, regardless of whether sample corresponds to direct final answer redo attempt. Accuracy is then measured using majority voting over these samples.
As shown in Figure 6 Re² fully exploits the benefits of test-time scaling: once the number of samples exceeds 64, they consistently surpass RLVR-trained models, whose performance has already saturated, and continue to improve test-time compute increases. However, when the number
8

---

Published as a conference paper at ICLR 2026
(a) Qwen2.5-144-Bars: on
(b) Qwen2.5-14B-BBl on
DAPO (Majority voting)
0.30
DAPO (Majority voting)
Accuracy
0.30
Accuracy
0.25
0.20
0.15
21
2'
2
2
2
2'
27
2
21
2"
2
2'
2
2'
21
Number Samples
Number Samples
(c) Qwen2.5-78-Instruct on AIME24
(d) Qwwn2.5-7B-instreee on AIME25
(Majority voting?
(Majerity voting)
Accuracy
Re
Accuracy
0.25
Re²
0.20
0.20
0.35
0.15
2
Number Samples
Number Samples
Figure 6: Test-time scaling of DAPO and Rea under the same training budget as number of samples increases.
of samples is small, Re² tends to trigger more redo actions on hard problems, which reduces the proportion of valid final answers within the sampled outputs and can lead to lower accuracy than RLVR under these settings.
5.4 TRAINING DYNAMICS OF RE
Figure shows the training dynamics of Re², including the average reward, the reward under the resolve action, and the behavioral distribution over correct answers, incorrect answers, and re-solve actions. Both the average reward and the reward for resolving steadily increase as training progresses. In terms of behavior, the probability of choosing to re-solve rises sharply during the first 20 steps and then gradually decreases with further training. Meanwhile, the probability of producing incorrect answers drops substantially, while the probability of generating correct final answers increases slowly. These results suggest that Re² rapidly activates resolving hehavior and then refines it, enabling the model abandon unproductive reasoning paths while maintaining exploration of correct but more challenging trajectories.
Average Reward
Resolve Reward
Behavior Probabilities
Wrong
Right Rado
0.5
M
Reward
Reward
0.5
0.4
0.4
0.4
0.3
0.3
0.3
0.2
Mn
0.2
0.1
50
75
100
125
50
75
125
25
75
125
Training Step
Training Step
Training Step
Figure 7: The training progress of Rea on Qwen2.5-14B-Ban
5.5
ANALYZING THE MECHANISMS BEHIND R S EFFECTIVENESS
To further investigate the underlying mechanisms behind the effectiveness of our method, we manually inspected sampled cases and analyzed accuracy improvements across different difficulty levels. Through manual inspection, we found that Re² produces reasoning chains of consistently higher quality than RLVR algorithms such as DAPO. This is largely because Re² allows the model to restart reasoning when the current trajectory is unlikely to lead to the correct answer, thereby substantially reducing the tendency to force final answer from flawed reasoning nonexistent conditions. We believe that the reward modeling in Re² is fundamentally more rational than the standard 0/1 end-reward paradigm in RLVR. Under pure end-reward objective, the model is

---

Published as a conference paper at ICLR 2026
encouraged always output final answer even when it is uncertain- oftertain-ooten producing spurious steps and incoherent logic in an attempt to "guess correctly." contrast, Re² enables the model to output final answer only when it is confident, and to honestly indicate the need to restart when the reasoning becomes unpromising. This more rational and self-aware behavior allows the model to better recognize when its current chain of thought is unreliable and avoid optimizing trajectories that accidentally guess the correct answer, which is a common issue in RLVR training.
To illustrate these findings, we categorized problems by difficulty and measured the accuracy and resolving rate of Re². We mixed AMC and AIME25 questions to create a test set with a balanced distribution difficulty levels. In Fig. B(a,, we grouped questions into seven difficulty levels based on the accuracy of the Qwen2.5-7B Instrruct model obtained through multiple samples. We then evaluated DAPO and Re² on each difficulty group and recorded Re² resolving rate. We observe that for questions the base model is completely unable to solve (Group 1), reinforcement learning cannot teach the model to solve them either. In such cases, DAPO often produces incorrect answers with unclear or erroneous reasoning chains, while Re² almost always refrains from giving a final answer and attempts to resolve the problem instead. For difficult but solvable questions (Group 2), Re² achieves
than twice the accuracy of DAPO due to its ability to restart from failed prefixes. Across all difficulty levels, Re² consistently outperforms DAPO, and its resolving rate decreases as question difficulty decreases.
In Fig. bb), we group questions by the difficulty estimated by DAPO, providing a more direct comparison against standard RLVR method. We find that the largest improvement occurs questions that RLVR occasionally solves (Group 4), where accuracy increases from 51.2% to 81.7%. These are questions that RLVR models are capable solving, but their ability answer correctly highly unstable because they cannot discard unpromising prefixes. Re² overcomes this limitation: by allowing the model restart, substantially increases the probability of following successful reasoning trajectory.
1.0
"weittB-it accuracy scquracy
DAPO ace
0.8
Re' accuraby
0.8
Reselve probability
0.8
resoive prob
0.
Resolve probability
Accuracy
resolve probabiiity
0.4
0
0.4
0,4
0.0
(a) Difficulty level Qwen-7B-tt hardest)
Difficulty level by DAPO hardest)
Figure 8: Analysis of accuracy and behavior across problem difficulty levels. (a) Accuracy improve- ments DAPO and Re?, along with the resolving rate of Re², when questions are grouped by the difficulty estimated by the base model. (b) Accuracy improvements and resolving rate of Re" when questions are grouped by the difficulty estimated by DAPO.
CONCLUSION
In this paper, we investigate the limitations of test-time scaling in existing LLMS and show that once early reasoning is misguided, the model rarely recovers and struggles to return to the correct reasoning path. To this end, we propose new reasoning paradigm that allows language models to flexibly restart reasoning when the current trajectory appears unpromising. We introduce Reinforcement Learning with Re-solving (Re?), which leverages pure reinforcement learning to encourage models to adopt re-solving behaviors. Empirical evaluations demonstrate that Re consistently outperforms standard RLVR methods across benchmarks varying domains and difficulty levels, while also raising the upper bound of performance achievable under test-time scaling.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
This paper presents work aimed at advancing the field of reasoning with large language models. As with all research involving large language models, there are inherent risks, including the spread of misinformation and the propagation of societal biases.
REPRODUCIBILITY STATEMENT
Our work is based on open-source models and datasets. In Section 5 and Appendix A we provide detailed descriptions of the prompt templates, method implementation, and experimental setups.
ACKNOWLEDGMENTS
We want thank all the anonymous reviewers for their valuable comments. This work supported by the National Science Foundation of China (NSFC No. 62576232) and the Young Elite Scientists Sponsorship Program by CAST (2023QNRC001..
REFERENCES
AI-MO. Ame 2023, 2024.
URL hEtps///uggingfceoco////ae///
simo-valln ammo-validationnaae
Akhiad Bercovich, Itay Levy, Izik Golan, Mohammad Dabbah, Ran El-Yaniv, Omri Puny, Ido Galil, Zach Moshe, Tomer Ronen, Najeeb Nabwani, et al. Llama-nemotron: Efficient reasoning models. arXiv preprint arXiv:2505.009/ 2025.
Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, et al. Do not think that much for 2+ 3=? on the overthinking of ol-like llms. arXiv preprint arXiv:2412.21187. 2024.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reñichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:21 10. 14168, 2021.
Alejandro Cuadron, Dacheng Li, Wenjie Ma, Xingyao Wang, Yichuan Wang, Siyuan Zhuang, Shu Liu, Luis Gaspar Schroeder, Tian Xia, Huanzhi Mao, et al. The danger overthinking: Examining the reasoning-action dilemma in agentic tasks. arXiv preprint arXiv:2502 08235, 2025.
Jhouben
Samuel Beaussant, and Mehdi Mounsif. Large reasoning models thinking straight: on the unreliability of thinking trajectories. arXiv preprint arXtv:2507.00711, 2025.
Bowen Ding, Yuhan Chen, Futing Wang, Lingfeng Ming, and Tao Lin. Do thinking tokens help or trap? towards more efficient large reasoning model. arXiv preprint wrXiv:2506.23840 2025.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv e-prints, pp. arXiv-2407, 2024.
Yichao Fu, Xuewei Wang. Yuandong Tian, and Jiawei Zhao. Deep think with confidence. arXiv preprint arXiv:2508, 10200081526000 15260, 2025.
Kanishk Gandhi, Denise Lee, Gabriel Grand, Muxin Liu, Winson Cheng, Archit Sharma, and Noah D Goodman. Stream of search (sos): Learning to search in language. arXiv preprint arXiv:2404.03683, 2024.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseck-rl: Incentivizing reasoning capability in 1lms via reinforcement learning. arXiv preprint 2rX::22001.12948 2025.
11

---

Published as a conference paper at ICLR 2026
Shibo Hao, Yi Gu, Haodi Ma, Joshua Hong. Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. Reasoning with language model is planning with world model. In NeurlPS 2023 Workshop on Generalization in Planning, 2023.
Shih-
Shih-Yang Yang Liu, Xin Dong, Ximing Lu, Shizhe Diao, Mingjie Liu, Min-Hung Chen, Hongxu Yin, Yu-Chiang Frank Wang. Kwang-Ting Cheng, Yejin Choi, et al. Dler: Doing length penalty right-incenttvizzing more intelligence per token via reinforcement learning. arXiv preprint arXiv:2510.15110, 2025.
Michael Luo, Sijun Tan, Justin Wong, Xiaoxiang Shi, William Y. Tang, Manan Roongta, Colin Cai, Jeffrey Luo, Li Erran Li, Raluca Ada Popa, and Ion Stoica. Deepscaler: Surpassing ol -preview with a 1.5b model by scaling rl, 2025. Notion Blog.
MAA Committees. Aime problems and solutions. Solutions
com/
http://nnxxxpppppppppppe Problems aaaeemssaadd_ddeeeeeeeeae aadaaanS___oo
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candes, and Tatsunori Hashimoto. sl: Simple test-time scaling. In Workshop on Reasoning and Planning for Large Language Models, 2025.
OpenAL. Learning to reason with 1lms, 9 2024.
URL
htmppeaai.c.m/eaeeeeoo com/ index
with- 11ms/
Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beatriz Borges, Antoine Bosselut, Robert West, and Boi Faltings. Refiner: Reasoning feedback on intermediate representations. In Proceedings of the 18th Conference of the European Chapter of the Association for Compulational Linguistics (Volume 1: Long Papers), pp. 1100-112 126, 2024.
Yiwei Qin, Xuefeng Li, Haoyang Zou, Yixiu Liu, Shijie Xia, Zhen Huang, Yixin Ye, Weizhe Yuan, Hector Liu, Yuanzhi Li, et al. O1 replication journey: A strategic progress report-pan 1. arXiv preprint arXiv:2410.188 18982, 2024.
Yuxiao Qu, Tianjun Zhang, Naman Garg, and Aviral Kumar. Recursive introspection: Teaching language model agents how to self-improve. Advances Neural Information Processing Systems, 17:55249-55285 2024.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a benchmark. In First Conference Language Modeling, 2021.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: / 1707.06347 2017.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, al. Deepseekmath: Pushing the limits of mathemat- ical reasoning in open language models. arXiv preprint arXiv22402.03300, 2024.
Parshin Shojaee, Iman Mirzadeh, Keivan Alizadeh, Maxwell Horton, Samy Bengio, and Mehrdad Farajtabar. The illusion of thinking: Understanding the strengths and limitations of reasoning models via the lens of problem complexity. arXiv preprint arXiv:2506.00 06944, 2025.
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute optimally can more effective than scaling model parameters. arXiv preprint 240V:22008.03314 2024.
Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi kl. 5: Scaling reinforcement learning with 1lms. arXiv preprint MwXiv:2501.12599 2025.
Guangya Wan, Yuqi Wu, Jie Chen, and Sheng Li. Reasoning aware self-consistency: Leveraging reasoning paths for efficient I1m sampling. arXiv preprint 20X 2024.
Shuhe Wang, Shengyu Zhang, Jie Zhang, Runyi Hu, Xiaoya Li, Tianwei Zhang, Jiwei Li, Fei Wu, Guoyin Wang. and Eduard Hovy. Reinforcement learning enhanced Ilms: A survey, arXiv preprint arXiv:2412. 10400, 2024.
12

---

Published as a conference paper at ICLR 2026
Xuezhi Wang. Jason Wei, Dale Schuurmans, Quoe V Le, Ed H Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representatiins, 2022.
Yue Wang, Qiuzhi Liu, Jiahao Xu, Tian Liang, Xingyu Chen, Zhiwei He, Linfeng Song, Dian Yu, Juntao Li, Zhuosheng Zhang, et al. Thoughts are all over the place: On the underthinking of ol-like lms.. arXiv preprint arXiv:2501. 18585, 2025.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of of-thh prompting elicits reasoning in large language models. Advances in 2022.
neural information processing systems,
Sean Welleck, Amanda Bertsch, Matthew Finlayson. Hailey Schoelkopf. Alex Xie, Graham Neubig, Ilia Kulikov, and Zaid Harchaoui. From decoding to meta-generation: inference-time algorithms for large language models. Transactions on Machine Learning Research, 2024.
Hao Wen, Yifan Su, Feifei Zhang, Yunxin Liu, Yunhao Liu, Ya-Qin Zhang, and Yuanchun Li. Parathinker: Native parallel thinking as new paradigm to scale IIm test-time compute. arXiv preprint arXiv:2509.044: 04475, 2025.
Siwei Wu, Zhongyuan Peng, Xinrun Du, Tuney Zheng, Minghao Liu, Jialong Wu, Jiachen Ma, Yizhi Li, Jian Yang. Wangchunshu Zhou, et al. A comparative study reasoning patterns of openai's 's ol model. arXiv preprint arXiv:2410. 0224410.33639 13639, 2024.
Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Inference scaling laws: An empirical analysis of compute-optimal inference for Ilm problem-solving. In The Thirteenth International Conference Learning Representations, 2025.
Zhiheng Xi, Dingwen Yang. Jixuan Huang. Jiafu Tang, Guanyu Li, Yiwen Ding, Wei He, Boyang Hong, Shihan Do. Wenyu Zhan, al. Enhancing Ilm reasoning via critique models with test-time and training-times supervision. arXiv preprint arXiv:2411.16599 2024.
Wei Xiong, Hanning Zhang, Chenlu Ye, Lichang Chen, Nan Jiang, and Tong Zhang. Self-rewarding correction for mathematical reasoning. arXiv preprint XrXiv:2502.19613, 2025.
An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jian- hong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2. 5-math technical report: Toward mathematical expert model via self reff-improvement arXiv preprint 22211222499.2122.. 2024a.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint ar 009:8550 2025a.
Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoe V Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers. In The Twelfth International Conference on Learning Representations, 2023.
Ling Yang, Zhaochen Yu, Tianjun Zhang, Minkai Xu, Joseph E Gonzalez, Bin Cui, and Shuicheng Yan. Supercorrect: Advancing small Ilm reasoning with thought template distillation and self- correction. arXiv preprini AXXi:24100009000 2024b.
Xiao-War Wen Yang, Xuan- Yi Zhu, Wen-Da Da Wei, Ding-Chu Zhang, Jie-Jing Shao, Zhi Zhou, Lan-Zhe Guo, and Yu-Feng Li. Step back to leap forward: self-backtracking for boosting reasoning of language models. arXiv preprint arXiv:2502.044 044004 2025b.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, et al. Dapo: An open-source Ilm reinforcement learning system at scale. arXiv preprint arXiv:2503. 14476, 2025.
Yu Yue, Yufeng Yuan, Qiying Yu, Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, TianTian Fan, Zhengyin Du et al. Vapo: Efficient and reliable reinforcement learning for advanced reasoning tasks. arXiv preprint arX1v:2504.05/18 2025.
13

---

Published as a conference paper at ICLR 2026
Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl- Z0o: Investigating and taming zero reinforcement learning for open base models in the wild, 2025. URL JRRthss://aaxxi org/ abs 2503.188881
Dan Zhang, Sining Zhoubian, Ziniu Hu, Yisong Yue, Yuxiao Dong, and Jie Tang. Rest-mcts": Llm self-training process reward guided tree search. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Kaiyan Zhang, Yuxin Zuo, Bingxiang He, Youbang Sun, Runze Liu, Che Jiang, Yuchen Fan, Kai Tian, Guoli Jia, Pengfei Li, et al. A survey of reinforcement learning for large reasoning models. arXiv preprint arXiv:2509.088: 00827,, 2025.
Xutong Zhao, Tengyu Xu, Xuewei Wang. Zhengxing Chen, Di Jin, Liang Tan, Zishun Yu, Zhuokai Zhao, Yun He, Sinong Wang, et al. Boosting IIm reasoning via spontaneous self-correction. arXiv preprint arXiv:2506.00 06923, 2025.
Yu Zhao, Huifeng Yin, Bo Zeng, Hao Wang, Tianqi Shi, Chenyang Lyu, Longyue Wang, Weihua Luo, and Kaifu Zhang. Marco-ol: Towards open reasoning models for open-ended solutions. arXiv preprint 1rXXi:24111144000 2024.
Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, et al. Group sequence policy optimization. arXiv preprint arXiv:2507.18071, 2025.
A IMPLEMENTATION DETAILS
During training. Re² employs specialized template, as illustrated in Figure 9 to elicit redo behavior from the vanilla model. The template is designed to encourage the model explicitly indicate when it decides to abandon the current reasoning trajectory and restart the problem.
When performing reward assignment, if the model outputs phrase such as "It's better redo the question. inn."" before producing a boxed final answer, we truncate the response at that point and append a special <eos> token mark the end of the sequence. This ensures that redo actions are clearly distinguished from final-answer completions and allows the policy to learn consistent behavior during reinforcement learning.
Re² template
Solve the following math problem step by step. you obtain final answer, enclose in boxed().
User:
(QUESTION)
Note: As you work through the problem, continuously assess your solution path real time. If you feel your current approach has become unclear or that rethinking the problem from scratch would be more productive, **simply say "It's better to redo the question." "** Once you say this, the answer should be immediately terminated (so do not use this phrase lightly-wher when reflecting to yourself, use "Do need redo the question?" instead). When faced with unclear or tangled reasoning, don't 't hesitate to make use of the option to start over. Use this choice wisely for the best results.
Assistant:
Figure 9: The template of Re?
B DERIVATION OF THE REWARD OF RE²
We derive the expected reward for choosing resolve with at most R allowed rounds, For out-of- group completions (i.e., prefixes Pre;), let P=ocoorecct) (iiioooorectt (re ve)
14

---

Published as a conference paper at ICLR 2026
denote the empirical probabilities of the three outcomes, estimated from the (n- x out- group CoTs. Each redo round draws one outcome from this distribution; a correct yields reward an incorrect yields reward 0, and resolve consumes round and restarts from the same distribution. Hence, the expected reward of choosing resol ve equals the probability that the first non-resol ve outcome within the first R rounds is correct:
Eir resolve, R
Σ ( Pzi(correct)
first t-1 are resolve the t-th correct
Ppccccretttt Σ Pr(
Evaluating the finite geometric series gives
1 1 P=
Eir resolve, R Pzi(correct)
1- (resoovv)
Therefore, for Ogg that chooses resolve, we set
1 (Tesollv))
Ti.j = P
1- Pri
15

---

Published as a conference paper at ICLR 2026
Table 2: Accuracy with 95% confidence intervals on five reasoning benchmarks, confidence intervals are given in parentheses.
Models
Methods
AIME24
AIME25
AMC23
GSM8K
GPQA
Base Model
DAPO
11.9 (=1.0)
(=1.0)
64.7 (=1.3)
91.8 (=0.1)
29.7 (+0.5)
Qwen2.5-7B Basc
Re²
17.1 (=1.4)
19.0 (=1.2)
70.8 (#0.3)
93.6 (=0.0)
36.8 (20.3)
DAPO
18.2 (=1.2)
15.7 (#1.2)
64.0 (+1.3)
94.3 (=0.1)
44.8 (+0.6)
Qwen2.5-14B Base
Re²
28.5 (=1.1)
23.4 (+1.3)
68.5 (#0.3)
94.6 (=0.0)
49.6 (+0.3)
Instruct Model
DAPO
15.0 (#0.9)
0.5 (+0.3)
32.3 (+1.3)
80.4 (+0.2)
20.7 (+0.5)
Maimaa.2-3B-Insaee
Rc²
17.7 (=1.1)
2.8 (+0.5)
38.4 (=0.8)
83.2 (+0.1)
20.2 (+0.3)
DAPO
16.0 (=1.1)
8.6 (#0.9)
62.3 (+1.3)
92.6 (#0.1)
35.4 (+0.6)
Re²
18.6 (=1.6)
21.2 (+1.1)
64.7 (#0.4)
94.1 (+0.0)
38.4 (#0.4)
Reasoning Model
DeepSeek-RI-
DAPO
38.4 (=1.5)
26.5 (+1.4)
86.9 (+0.9)
89.6 (#0.1)
38.4 (+0.6)
Re²
47.2 (=0.7)
29.6 (=0.8)
88.7 (=0.2)
92.2 (+0.0)
44.8 (+0.3)
C EXPERIMENTS
C.I MAIN RESULTS
We additionally provide a comparison between DAPO and Re² with confidence intervals, as shown
in Table 2
C.2 PERFORMANCE UNDER TEST-TIME SCALING
We further compare the test-time scaling perfor- mance of Re² with broader set of baselines.
Test-Time Scaling AIME25
These include: GRPO (Shao et al. 2024). the classical RLVR algorithm and the
ORPO
technique
ccurar
used in training DeepSeek-RI (Guo et al. 20255): )
0.20
DLER Liu et al. 2025). the recent sfate-of-the- art efficient reasoning method that reduces to- ken consumption while maintaining performance through truncated-length penalties and training- stabilization strategies; and DeepConf (Fu et al. 2025), which leverages internal confidence sig- nals during decoding to dynamically terminate low-quality reasoning traces.
Total Computation (tokens)
Figure 10: Test-time scaling of Re² compared with additional baselines.
We use the number of consumed tokens as the measure of computational cost and evaluate them on the challenging AIME25 benchmark, which has no risk of data leakage. As shown in Figure I0 method achieves better test-time scaling than all competing approaches.
C.3 ANALYSIS THE RESOLVE REWARD ESTIMATOR
To better understand the performance of the resolve reward estimator in Re². we evaluate how accurately it estimates a model's resolving accuracy. For each training question, we draw 1024 inde- pendent samples and treat the proportion of correct responses the ground-truth resolving accuracy. We then compare the bias and variance of the estimator under different sampling configurations.
Specifically, fix the number of suffixes at and conversely fix n = while varying m E {2, 4, 8, 16). As a bascline, we include an exponential moving average (EMA) estimator with decay rate of 0.9.
and vary the number of prefixes {2, 4, 8, 16).
16

---

Published as a conference paper at ICLR 2026
As shown in Fig. the Re reward estimator becomes increasingly accurate either m increases, exhibiting consistently lower bias and variance. also outperforms the EMA baseline across all settings, demonstrating the effectiveness of leveraging the naturally generated n X m suffix samples for estimating resolving accuracy.
Bias vi (m B)
0.030
Veriance ve n (m
goldd
0.009
Bias vs m (n B)
Veriance vs (n B)
0.08
actinator
2.025
LPA autivattrr
0.055
LHA astivater
EPA variance
0.070
0.000
0.05
0.013
0.050
0.007
0.020
0.043
0.03
0.005
10 [number predices)
30
30
(number preficee)
(number suffizus)
[number sufficess)
Figure 11: Bias and variance of the estimated resolving accuracy under different values of and
C.4 ANALYSIS OF THE DEGENERATION GROUP RATE
We report the degeneration group rate of DAPO and Rea, defined as the rate of groups in which all samples receive the same reward and therefore have zero advantage. As shown in Fig. 12 in Re², degeneration groups that are "all-wrong" during the later training stages gradually turn into "all-redo."" indicating that the model shifts from forcing an answer on unsolvable questions to choosing to redo them. The overall degeneration rate of Re is about 10% higher than that of DAPO. However, as noted in DAPO Yu et al. 2025). "the filter strategy does not necessarily impede training efficiency, because the generation time is typically dominated by the generation of cong-tail samples if the RI, system is synchronized and the generation stage is not pipelined." Consistent with this observation, Re² does not incur additional training cost due to this effect.
Degeneration group rate of DAPO
Degeneration group rate of Re²
Total
Total
0.7
All correct All wrong
correct
Mapadyn M
wrong redo
Proportion
0.5
Mya /ey Man
MM Am
myw
0.3
V Mm hm
0.1
20
80
120
20
80
Training steps
Training steps
Figure 12: Degeneration group rate during training for DAPO and Rea.
C.5 ANALYSIS THE REASONING LENGTH
We analyze the evolution of reasoning lengths for DAPO and Re² throughout training. As shown in Figure [3] combined with our inspection of a number of cases, DAPO exhibits steady growth in CoT length during training due to increasing amounts of self-reflection, eтror correction, and switching of reasoning strategies, similar classical RLVR. Wrong CoTs are noticeably longer than correct ones because many failed trajectories attempt to extend or patch flawed prefixes. This represents the strategy optimization that occurs under the one-shot chain-of-thougee Re² gradually stabilizes its CoT lengths, and the lengths of correct, wrong, and redo CoTs do not show substantial differences. Both the statistical results and our qualitative observations indicate that Re² behaves more rationally under the multi-chain, resolving reasoning paradigm: the model does not force itself into producing a strained and ultimately incorrect reasoning chain, nor does over-commit to unpromising trajectories.
paradigm. In contrast,
In addition, Cedo-CoTy in Re² tend to be longer at the beginning of training, and their lengths gradually approach those of final correct incorrect CoT' as training progresses. Consistent with our manual
17

---

Published as a conference paper at ICLR 2026
inspection of cases, early in training, the patterns that trigger redo relatively shallow- for example, CoTs becoming excessively long and close to the context window limit, frequent switching between reasoning threads, resorting brute-force enumeration problems that actually require summarizing underlying patterns. In later stages, however, redo decisions become more closely tied the intrinsic quality of the CoT, such the effectiveness of the chosen approach the soundness of the assumptions.
DAPO CoT length
Re² CoT length
1800
1600
CoT length
1400
1200
1000
600
20
40
60
80
100
120
20
40
60
80
100
120
Training steps
Training steps
correct CoT
wrong CoT
redo
Figure 13: Reasoning lengths during training for DAPO and Re?.
D TRAINING COST ANALYSIS OF RE²
Training Re² involves a two-stage generation pro- the model first produces prefixes and then generates n x m suffixes. This sequential genera- tion procedure
Scaling on
be more time-consuming than directly generating responses in parallel. To quan- tify this overhead, measure the rollout time of DAPO and Re² on an 8xA 100--000--400E server.
0.175
0.150
0.073
To produce the same number of rollout samples, DAPO uses global batch size of 128, generat- ing 16 samples per question. Re² uses a global batch size of 32, generating 8 prefixes followed by suffixes for each prefix. average, DAPO requires 388 seconds per rollout step. In comparison, Re² takes 89 seconds for prefix generation and 342 seconds for suffix generation, resulting a total of 431 seconds, which corresponds to an 11 1% increase in rollout time.
Training (hours)
Figure 14: Accuracy improvements of DAPO and Re under the same training time.
Considering the performance improvements Re² provides during training, this additional cost is acceptable. As shown in Fig. 14 Re² outperforms DAPO under the same training time budget.
E CASE ANALYSIS OF COTS WITH DIFFERENT LENGTHS
Here, we present a case study comparing short and long CoT responses obtained from multiple
samples on the same problem Figures 15 I6 17 and [8] When the model follows correct reasoning path with high-quality initial steps, can often reach the correct answer within a relatively short COT. In contrast, when the initial reasoning direction is misguided, the model tends to produce longer CoTs with many additional tokens, yet still fails to arrive at the correct solution. This illustrates the critical importance of carly reasoning quality and highlights the inefficiency of over-extended reasoning when the initial trajectory is flawed.
18

---

Published as a conference paper at ICLR 2026
longer and shorter response in AIME25
Problem:Let k be real numbers such that the system |25+ = 5 and |z-4-k| -z-3i-ki has exactly one complex solution 2. The sum of all possible values of can be written as m where m and n are relatively prime positive integers. Find m + n. Here i = V-I. Shorter Response:Okay,
need to solve this problem where have two complex equations:
1. 1. +-+200 20i 2| = 5
2- = -i
And the system has exactly one complex solution 2. We need to find the sum of all possible values of k. express it m/n where m and are coprime, and then find m + n.Arigh,, let's break this down step by step.First, let's understand what these equations represent geometrically. Since
yi. gi.Then, 25 20i (x + vi) (25 a:) + (20 y)i.The modulus of this is
a complex number, we can write
25 + 20 /(25 x)² x2 20 y)² 5.So, squaring both sides: (25 xr²² + (20 y)² 25. That's the equation of a circle in the complex plane with center at (25, 20)and radius 5. So, 2 lies on this circle.Now, moving to the second equation: 22- 4 k| |2 3i Ll.Let me write as x + yi
again. Then Left side= (x yi)
kl (x 4 k) yil v(1 4 k)² Rightt
side= |(x + yi) - 3i k| = {x - k) + (y - 3)il = v(x (ir-k) k)² + (y 3)2 -3)..Sttting
them equal: (x 4 k)² y²
Squaring both + ( ---3 -3 Let's expand both sides:Left side:
v(z k)² + (v-3))
sides:(z - 4 k)² + y²
(r
2)²
(x (4 + k))² + y =
k)2
+ y² = (x² 2(4 k)z + (4+ k)3) + v².
Right side: (I (y-
(I² 2k + k2) + +2² 6y + 9). Set them
equal:>² (44=))==(443)+
2kr 222 y" -6y 9. Simplify both sides by
subtracting x²
= -2kr k22 6y + 9. Bring all terms to the 0. Simplify term by term:First, combine the left:-2(44 --== 2 -8r. Next, constants: + k)² k2 9. 2)
12 9 -- 16 + 8k + k22 122 9 = 7 Sk. So now, equation becomes: +r 8k by= 0. So that's the equation of the second condition.So now, we have two conditions:1. = lies on the circle with center (25, 20) and radius 5; 2. z lies the line defined by -&r 6y + 7 + 8k O.Since the system has exactly one solution, this implies that the line is tangent to the circle. So, the distance from the center of the circle to the line must be equal to the radius.So, the center of the circle is (25, 20), and the line is -8x + 6y + 7 + 8k = 0.The distance from the center (25, 20)to the line is equal to the radius 5.The formula for the distance from point (No, yo) to the line az + by + = 0 is |azo + byo + cl/va² 82.In In our case, the line is Sa + 6y + (7 + 8k) = O. 62) - 200 + 120 + 7 /kl/ 36 = (-00 + 120 + 7) 8k) / 100 = ( 73) 8k//10.
Let's (1 +16
so = -8, 6, c = 7 8k. The distance is - 8 * 25 + 6 * 20 + 7 8k / ///(/((88/))
Set this equal to the radius 5:| 73 Sk| 10 5=> 18k 73 50. Therefore, 8k - 73 50
or 8k 73 = -50.Solling:Case 1: 8k - 73 = 50 => 8k = 123 => k = 123/8. Case
-0 => 88: 23 => 23/8.Therefore, possible h are 123/8 and 23/8. Thus, the sum is 123/8 + 23/8 = (123 + 23)/8 = 146/8 73/4.So m = 73, = 4, which are
2:8k
coprime. Thus, + n 73 77. Final *nswer**
77 (Correct Answer)
Figure 15: Case Analysis of CoT' with Different Lengths in AIME25
19

---

Published as a conference paper at ICLR 2026
longer and shorter response in AIME25
Okay, have this problem here where I need to find all real numbers & such that the system of equations:.. 1. |25 + 20i - =| = 5 :2 | - 4- k|| = 3i - ki| has exactly one complex solution 2. Then. need to find the sum of all possible values of is and express it as m2 where m and n are coprime, and then find m + n..Alright, let's break this down step by step.First, let me understand what each equation represents geometrically.Stariin with the first equation: [25 + 20 - 2| 5. Since is complex number, let's denote 2 = x + yi where x and y are real numbers.So So. substituting 2 = x + yi into the first equation: 25 + 20i - (x + yi)l 5. 5.Simplify the expression inside the absolute value: (25 x) + (20- y)i| =5 .hh modulus of complex number
Longer Response:
5.Squaring both s2des:(25 - x)² (20 y)² 25.This is the equation of circle in the complex plane with center at (25, 20) and radius 5.So, the first equation represents a circle centered at (25, 20) with radius 5.Now, moving
bi is Va² + B2, s0: x)² + (20 y)²
the second equation: 1z 4 - k| = |z 3i k|.Again, substituting = = x + yi:Left side: - 4 - k) + (y Right side: [(x) + (y 3)i k). Wait, perhaps better substitute more carefully. Wait, actually, let's note that is x yi. so:Left side: [x yi) (4 + 1) = [xx 4 k) + yil Right side: [ + y2) (3i + k)] (x- 3) Therefore, the equation becomes: (2 1u----6 14
(x- 3yi} Which
is equivalent to: / (xx - =)²² + y² = v(x - 1)22 + (y - -) Squaring both sides to remove the square roots: r - 4 -))² + y² (r k)2² + (y 3)2 Let's expand both sides:Left side: (x- (4- 2y² = (x (x-((4) (1-(4+))) 2=))))+++++++ +2
Which expands to: x² 2x (1(4+))+(44) +4+
side: (x - k)2 + (y - 3)2 Which expands to: x²² -22r + 12 + y² Gy + 9.Now, set left side equal to right side:z² 2x(4 k) + (4 + k)² + y² x² 2k + + + y² 6y + 9.Simplify both sides:First, notice that x² and y² appear on both sides, so they cancel Let side: -2r(4 + k) (4 k)2 Right side: -2kx + k2 6y + 9.So, bring all terms to side:-2r(4 k) (4+
(-2kx) ) - 122 + 6y 9 = 0.Simplify term by term:First, expand -2r(4+ 2k2: Whichhich +22:4hh)) is 8z 10--24 2kr + 2kr
-r. Wait, perhaps let's ss do it step by step:After cancelling x² and y². we have:Left: - 2x(44 +444(444 (4+
2kx - Gy + 9.So, subtract the right side from both sides: fides:-2x((4 2xx 4+ k) (4+ (-2kr) k2 + Gy -9 9 = 0.Simplify each
term:First term: 22
2k:. term:
term: 6y e0y-9.Commpn 9.Compute each
k:2 16 + 8k.3.6y (Error Putting it all together: 8r 16 8k + 6y 9 = 0.Simplify constants:16
part:1. 2x ())++ +)+ (-8z 2kx) 2krr -x,2
7, -r+ 7 6y = L.Lettss 6y (8k 78)+7) = 0,We can divide the entire equation by-1 11 to make it look cleaner: -6y -
0.Alternatively, = 0.This is the equation representing the set of points = = x- --yisuch that 4 4- kl = = 3i kl.This equation is actually the perpendicular bisector of the segment joining the points 4 4 kand 3i + kin the complex plane. Wait, let's think in terms of iordiinntes.(Errro In the complex plane, is point x. y). The equation [z (4 4 k)] | (3i + k:) caan be rewritten as (x + yi) (4 + k)) = (xr + yi) (0 + (3 + k)j) ,SS,, this is the set of points equidistant from (4, and (nd + k). Therefore, the perpendicular
keep it as: - +8 6
bisector of the segment connecting (4, k) and (0, 1.,++K +.)..eerrroorr the second equation represents the perpendicular bisector of the segment between (4, k) and (0,3+A)) + &o, now we have two conditions: 1. 2 lies on the circle centered at (25, with radius 5,2.z lies on the perpendicular bisector of the segment between (4, k) and (0, + We are told that the system has exactly one solution. That means that the line (perpendicular bisector) intersects the circle at exactly one point. Therefore, the line must be tangent to the circle. Therefore, the distance from the center of the circle to the line must be equal to the radius of the circle.So, perhaps the steps 1 Find the equation of the perpendicular bisector of the segment between (4, k) and (0, + k).2. Find the distance from the center of the circle (25, 20) to this line.3. Set this distance equal to the radius 5, and solve for k.4. Sum all possible values of k.So, let's proceed step by sirp.First, let's find the equation of the perpendicular bisector. Given two points A(4, k) and B(0,3 3 + The midpoint M of segment AB is:M, = 1+0 2, My k++3+k) 3+2k So, midpoint is (2, (3 + /2),Nxt,,,,, the slope of AB is: (3+
2
2
2
3
-3 The.ehfee
0-4 -4
the slope of the perpendieular bisector is the negative reciprocal, which 4/3. 3/3.hhereorre, the equation of the perpendicular bisector is:y- - M, = (4/3)(r - M in M, 2and
My = (3 + 2k) - (3 + 22 = (4/3)(z 2).
Figure 16: Case Analysis of CoT' with Different Lengths in AIME25
20

---

Published as a conference paper at ICLR 2026
longer and shorter response in AIME25
Let's write this in standard form oz + by + e = 0.Multiply both sides by 3 to eliminate the
fraction:3yy - (3 + 2))/2) 4(r 2) Simplify:33
(9 +6k)/2 4r S.Multiply all
9 6k = 82 16.Bring all terms to left side: -8x + by - 9 - 6k + 16 0.Simplify constants: -9 16 = 7.Thus: 8x + Gy + 6k
terms by 2 to eliminate denominator:6y
Or, 8z + 6y + (7 6k) 0 .W can write this as: 8z 6y + (Gλ: 7) = 0 WWiit, let me check.From above:After multiplying by 2:6y - 9 - 6k = 8z 16. moving all terms to left:-8r + 6y + (-9- 6k + 16) 0. Which is: 8r + by + (7 61) O.So, if we want to write it as or + by + c = 0. (Error 1) then: 8z + 6y + (7 6k) = 0.Alternatively, multiply both sides by -1:8x 6y - 7 6k O,Whh is:8 Gy 6k 7 0.Thus, the equation is 8x-6y 6y (6k - 7) = 0.Alternatively, let's veriff,.Attrrnatiely, perhaps it's better to keep the equation as: above: -8x - Which can be written as:8r Gy 7-6k -kt perhaps let's consider the standard form as:8. - 6y + (-7 6k) 0.Either way. perhaps it's better to find the distance from the center (25, 20)
this Thne.The distance from point (ro, Uo) to the line ar + by + c = 0 is (azo + byoo + cl/ cl/V(a" + Soo first, let's write the equation in standard form.From above, the equation is: -r + 6y + 7 6k = We can write it as:-8r+- + 6y + (7 6k) = 0. O.Thus, coefficients are a = 8, b- 6, c
= 7 Therefoee,,, distance from (25, 20) is: 25 + 6* 20 + (7 - 6/)l ((1//(((())))))) 8)2 + +
numerator:Fing compute
each term: 25- -200, 6 20 120.So, total so far: 2200 120 Add (7- 6k):
-80 + 6k -33 6A.So numerator
73 -em..ennorn is /64 + 36 = /100
10.Since the distance must be equal to radius 5, we have: - 73 - 6kl = 5.Therefore. - -73
6Y = 50 or -73 6k = 50Case -73 6Λ = 50, -6k 50 + 73- 123,k = -123/6 / = 41//2.Case 2: - -73 6k = 50. - 6k -50 + 73 23, = 23//6.Therefore, Therefore, the possible values of l are 421/2 and 23/66. Thherefoee,, the sum of all possible values 42 + -2366 convert to common denominator:-411 total is - 123/6 - 23/6 = (-146) /6 73/3.(Error 3) But wait, the question says "the sum of all possible values of k. perhaps need to check if both solutions perhaps let's see if these k's result in exactly one volution.Since each i gives a different line, each line can intersect the circle at two points, one point, or none. Since we have exactly one solution, it's when the line is tangent to the ThereeTheer,oor, perhaps both k's are valid, but perhaps we need check if both lead to lines that are tangent to the perhaps only one of them is valid because for some k, the line might not intersect the circle or something. But perhaps both are valid. Therefore perhaps sum (-41/2) + (-23/6)= as above, which is (-123/6 sum of all possible values of k", which could be negative or positive, perhaps we need to check if both are valid. A Alternatively, perhaps the sum is (-41/2) (-23/6) (-123 -23)/= (-146)/6= -73/3.Bur But since the problem says "sum of all possible values of k. perhaps we can write it as a fraction.But perhaps let's verify. First, let's check for k = -41/2. -ompu.e the distance from center to the line: - 73 -
6k| Which implies:
223/6.
-123/66, so
valid. Wait,
23/6) = (-146)/6 = 13.)/3.Buu since the problem says "the
73 - 6 (-41/2) = 5, which is the radius. -73 + 123 = Thus |50|/10
-73 + (6* 41 = -73 + (246)/2 So this is correct. Similarliimllally.. for k -23/ 23/6:Compute |- -73 - 6 + (-23/6) = 73 + 23| = - -50 50. Thus, same result, distance is 5. 5.Therefore. both k's are Henie, the sum is (-41/2) + (-23/6) = (-123/6 23/6) = (-146)/6 - (-73)/3.Thul
m = =733 = 3. Wait, but m and must be positive integers. So perhaps take absolute values? Wait, perhaps the problem expects the sum as positive.But perhaps not, since k can be negative. The problem says "sum of all possible values of kn' perhaps we just add them as they are. are.Thus, (-41/2) + (-23/6) = (-41* x 3- 23)/6 = (-123- 23)/6 = (-146)/6 = (3666=((33)) (-73)/3.So the sum is 73/3, which can be written as -3/33. Since m and n are coprime, m = 73, n = 3, but since the sum is negative, perhaps the question expects the absolute value? But no, because the problem says "the sum of all possible values of k can be written as m " Therefore, the sum is -73/3, so 73, 3, which are coprime. Thus, m + + 73 + 3 = 76. Final * 76 (Wrong Answer)
(Error: 1. Repeated reformulation of the equation form. 2.Coordinate extraction error, all following calculations invalid. 3. Ineffeccivee repeated verification of the answer)
Figure 17: Case Analysis of CoTs with Different Lengths in AIME25
21

---

Published as a conference paper at ICLR 2026
longer and shorter response in AMC
Problem:How many positive perfect squares less than 2023 are divisible by 5?
Shorter Response: determine how many positive perfect squares less than 2023 are divisible by 5, we start by noting that perfect square is divisible by 5 if and only if the integer itself is divisible by This because if n² is divisible by 5, then n must be divisible by 5 (since 5 is prime number).
5k where k is an integer. Then, n²² (5k)² 25k We need k2 < 80.92. The
25 < 2023. Solving for k, we get:25k² largest integer & that satisfies this inequality is 8, because 82 64 and g2 = 81 which is greater than 80.92. Therefore, k can take any integer value from I to 8. This gives us 8 possible values for k. Thus, there are positive perfect squares less than 2023 that are divisible by 5. The answer is: 8 (Correct Answer)
2023 = k2 < 25
Longer Response: determine how many positive perfect squares less than 2023 are divisible by 5, we start by noting that perfect square divisible by 5 must itself be multiple of 25 (since the square of any integer that is not multiple of cannot be divisible by 5, and the square of an integer that is multiple of 5 must be multiple of 5 = 25.We need to find the largest integer n such that 25n² < 2023. This is equivalent to finding the largest integer n such that n² 2023 80.92. Since n must an integer, we take the greatest integer less than /80.92, 25
which is 8 (since /80.92 s 8.996). Thus, n can take any integer value from to 8. (Error) We can verify this by calculating the squares of these integers and checking if they are less
than 2023:12 = 1,22 4.3 9,42 16.52
25.6 = 36, 72 49,8*
64,9*
81, 10a 100, 112 121, 122 = 144, 132 = =169, 142 196, 152 -225, 162 = 256, 17
289, 18 324, 19 =361, 20
400,21 = =441,22" = 484, 23 = -529, 24 576, 25
625, 26 = 676, 272 722,,88 784,292 = -84,.02 = -900, 312 = 9911,22 = 1024, =
1089, 34 1156, 352 1125,, 362 =1266,37 372 1369, 382 1444, 392 1521, 40
1600, 412 1681, 422 = 17764 432 =1849, 442 =1936, 452 2025.Since 452 = -2025 is not
44. Therefore, the positive perfect squares less than 2023 that are divisible by are the squares of the integers from to 44, which are 44 in Thus, the
less than 2023, the largest integer
number of positive perfect squares less than 2023 that are divisible by 5 is: 44 (Wrong Answer) (Error:Overlookiig the "divisible by constraint during verification.)
Figure 18: Case Analysis of CoTs with Different Lengths in AMC
22

---

Published as a conference paper at ICLR 2026
F RE EXAMPLES
We present examples of Re in Figures 19 20
21
and 22
Re² Examples
Problem:How
many ordered pairs of positive real equation + 22) + (2a + b) 32ab?
numbers
(a, satisfy the
Reansoning Trace Re solve the equation (1 + 22 + 2b + b) = 32ab for ordered pairs (a, b) of positive real numbers, we start by simplifying and analyzing the equa- Firs.First, let's rewrite the equation for clarity: + (2 (2+ + (2a + b) 32ab. We can factor out the common terms in the first two factors:(1
32ab.This simplifies to:2(1 + 21 + (2a + b) = 32ad.Diviiing both sides by 2, we get:(1 + 21 + (2a b) 16ab.Next, to see if we can find any solutions. Let's start with = 6. Substituting a = into the equation, we get:(1 + (1 (2a + a) 16a This simplifies to: 21) + +3(( = 16aa Expanddng the left side, we get:3a(1 1 + 21 + a) = 3a(1 + 3a + 2a²) )- 3a + -9 + -a .So the equation becomes:3a + 9a 6a 16a Rearranging terms, we gettaa 9a² +3a 16a O.which
2a)2(1 b)(2a b)
will test some simple values for and b
simplifies to:6a² -7a2 33 = 0.Factoring out we getaa 2 - +a+ = 0.Since is positive real number, a f O. Therefore, we need to solve the quadratic equation:6a" 7a + 0. discriminant of this quadratic equation (-:(-
4 6 3 49 72 = 233.Since the
discriminant is negative, the quadratic equation has no real solutions. Therefore, there Restarting from scratch:
positive real solutions for when b. It's better redo the question.
Next, let's try another approach by assuming a 1 and solving for b1
32- 1 b, b,which simplifies to: [1 + (2 (1 (1) 16b 2b) b) 16b. Simplifying further, we get:2(2 + 26 26 - 26²)
16b or 2(2 + 46 26 166.which simplifies to:4 + 8b 4h2 = 16b or 4b2 8b + = O.Dividing by 4, we get:b 2b + 0,which factors to:(b - 1)2 0.So, b 1. Therefore, (a. b) = (2. is solution. To confirm that there are no other solutions, we can use the symmetry and the nature of the equation. Given the complexity and the fact that we have found a solution, we can conclude that there are no other positive real solutions. Thus, the number ordered pairs (a, b) that satisfy the equation is: (Correct Answer)
Figure 19: Re² Examples
G LIMITATIONS AND FUTURE WORK
In this section, outline several limitations of our study that highlight areas for future research and improvement: (1) Although our method improves performance on challenging problems through better test-time scaling, models may still require multiple re-solving rounds to reach the correct answer. At present, there no mechanism to explicitly control the probability of invoking the re-solve action during inference, which may affect efficiency. (2) Our experiments primarily focus large language models, leaving the potential of applying Re² to other generative modalities (e.g., vision or multi-modal reasoning) unexplored. (3) We mainly study reasoning benchmarks, while extending Rea to more complex tasks, such as tool use, or search-intensive problems could also provide benefits.
H RE² APPLY IN PANGU
IB is fast-thinking language model trained from scratch on Ascend NPUs and designed for efficient execution on edge devices. We performed post-training on eight A100 GPUs using our methods, with training details consistent with 5.1 As shown in Table3] the accuracy of our method on the four benchmarks is significantly better than the original model. The accuracy improvement on GSM8K even reached 16.6%. This demonstrates that our method remains effective on models designed for efficient execution edge devices trained with NPUs.
23

---

Published as a conference paper at ICLR 2026
Re² Examples
Problem:Ralph has record store where people can trade their own records for new ones. People can trade old records for I new one. 5 people come in with old records and leave with 7 new records between them. How many old records did the people bring in?
determine how many old records the 5 people brought to Ralph's 's record store, need understand the trade ratio and the total number of new records they received. The trade ratio is 2 old records for I new record. So, if each person trades 2 old records for new record, and people are trading, we need find out how many old records were traded in total get new records.Let's denote the number of new records each person receives as n. Since each new record is obtained by trading 2 old records, each of the 5 people receives
Reansoning Trace of Re
records, the total number of new records is 5n. We know that the total number of new records they leave with 7, so we have: 5n SSolving for n n 5 This means each person receives new records.
However, since each person can only receive whole number of new records, we need consider the total number of new records and the trade ratio in another way. It's better to redo the question. Restarting from scratch:
If people together leave with 7 new records, and each new record is obtained by trading 2 old records, the total number of old records traded new records 2 old records per new record 14 old records Therefore, the 5 people brought in total of 14 old records. So, the number of old records the people brought in is: 14 (Correct Answer)
Figure 20: Re² Examples 2
Table The accuracy of OpenPPangu-Fmhbedded-- 1B and Re² on four benchmarks.
Methods
AIME25
AMC23
MATH500
GSM8K
OpenPangu-Embeedder Embedded-
4.7
22.2
49.3
64.4
Re²
8.6
35.3
65.8
81.0
I USE OF LLMS
LLMs were used solely for language polishing of the manuscript.
24

---

Published as a conference paper at ICLR 2026
Re² Examples 3
Problem:Let v. and 2 be positive real numbers that satisfy the following system of equations:logg) y= 2 logz( 2z ( 3 loga (yy) - Then the value of lloge y is where m and are relatively prime positive integers. Find m + n.
Reansoning Trace of Lee:Let's start by expressing the given logarithmic equations in terms of
2, V, and 2.The given equations are:log2 y= 2.00g2 (+2 gl00g2
4. We
can rewrite these equations as:log2(x) - logz(v=)
(l0gg(1) logy
1 1=)gg(=) log)(ry) = }. the properties of logarithms, we can further simplify these equations:logy(7) (log, (v) + log)(=)) = (1) (logg(r) + logg(=)) 3 [lgg(())
(log2 (x) + (v)) = Let's denote:log,(1) = a, (1) = b, logg(z) = e.So the equations become:a (b+c)= = 2.b b. (a+ 3.C (a+ = 4 Simplifying each equation, we (Equation 2) - b (Bquation 1).-b +
get:a- - b-
(Equation 1),b have system of linear equations:a
(Equation 3). We
1 (Equation 2),-c a +
(Equation Let's add all three equations c) (-b + c) + (-a + c) = 1/2 +
+ 7 4.Simplifying the left side:a - c c b a a --- b = 2--2------22 2c = 1/2 ++3 H.
tagether:(a
right side can be simplified as: + 1 +
12
6
+
12
12 = 12
13 13.S0 we
have:2a 3b 2c 13 However, let's instead solve for b, and c. by subtracting pairs of equations to find individual expressions. Equation from Equation 1:(a-b-c)- (-b
_2
3 - = This simplifies to:0 6. This is contradiction, which indicates need to recheck method of solving. Instead, let's solve for a, b, and c by another method.It's better to redo the question. Restarting from scratch:
-
Let's add Equation 1, Equation 2, and Equation 3 and then solve for a, b, and e individu- ally.From the system:a b c
(Equation 2),c a 6 I (Equation 3). We can add these equations:(a - b c) + (b-a-c))
(Equation 1),b a
= + 4: g.a-b-cbb- a-caaaaa- ----cc-oo a b = 12 a c-b = 13 (a++))))) = + -13 This
13
12-
-12-
is incorrect, so let's re-evaluate by solving the system of linear equations. Instead, let's solve for a, b, and c directly. It's better to redo the question. Restarting from scratch: We can use a different approach by expressing b, and in terms of common variable. Let's add and subtract the equations in a different way. From:a- -bc 3. b-b c J. c. 1 Let's solve these equations by eliminating variables step-by-step. We can start by expressing b
in terms of and from the first equation:e
2 J.Sbstitute e into the second
equation:b - a - (a - b - ))) = 1.bb - a a a b 1 4.26 2a + 1 = }.2((b - a) =
2
2
I = 2 3 = -.b a) = G. -
a = 12, = a - 12 NNoow substitute
3 2
b = a
1o
i and
e
= - b- into the third equation:e = a - (a
12
1
a 12
1
2 E
1
I
I2 I.
G -5 Substiituee b
and c 12
5 into the third
equation: 12
5
(a
[2]
1
-a-at 12
1 -5
+ 1 2a
2a
4: - 2a 1 2a
+
4
2a 1.
Substitute
1
12
12.
24
24
back to find and cb
a 12 24
12
21
24
2
24
33
12 Now
we have:a express x =²²²²² in terms of a, b, and c:log, (r 22 = 4a
21'
7 b = g:
c =
12
We need to find (logg
( y²²²2 )) First, we
+
3h
+ 2c.Suubsig a, b, and
c: 4a + 36 + 2c 4
24 + +3
T2
28
12
24
28
24
27
25 24
=
28+27+20
24 = 25
75
8 Thus, log, (a 322
-
25
25.o (
(1
5 The
24
8
fraction 25 is in simplest form, so m 25 and = 8. Therefore, +n++ 25 + +8 33.Thh
final answer is: 33 CCorrect Answer)
Figure 21: Re² Examples 3
25

---

Published as a conference paper at ICLR 2026
Re² Examples 4
Problem:Let f be the unique function defined on the positive integers such that Ed f.f (5)
for all positive integers n.. What is ((203))
Reansoning Trace of Re determine the function f defined on the positive integers such that Ed n d f (3) 1.for all positive integers n we start by analyzing the functional equation. We will use the properties of the function and the structure of the equation to find (n) Step-by-Step Analysis
1. Base Case: ni = E. d (2) = 11.'Th The only divisor of is itself, so the sum reduces to:
1. (-((1)= - f(1)= 1.
2. General Case: n = where is a prime number. d. (() = 1.The divisors of p are
and p, so the sum becomes: 1 f(p) + p- f(1) = f(p) + p- p-
f(p) + p 3. General Case: n = pi122 ... Pg (where pt are distinct primes. FaimesEdd d. d
- f(p) = 1 p.
For
"1
pp we have: > dppp d
1. The divisors of p are (1, (1, pi ... p(t)) So,
the sum becomes: 11 - f(pi" ((p11))
L.Substituting [DTT" = - p" for m 2 and
f(1) 1, get:p! f(p((()))))))))))))))) + pl [(p + f(r 3) +... pi f(1) =1
(1- + (1 p1 -1--1) p (1-pi +... 1"
=1 - - (p
Pi ) Pi
" =1= - (
I,1++1 +
The sum + P1 + Pi
+...
pi is geometric series: o1 - 1)/(p1 1).So we
get(pfit1 - 1))(p1 -1
ps ee 1. Simplifying this, we need to check if this holds for general n It's better to redo the question.
Restarting from scratch:
Let's hypothesize that f(n) = 1. We substitute f(n) 1 into the original equation:
n Ean d. f ( - Ean d 1 E d- = d n n Edjn For this equal 1, we Edin d This is not generally true for all so need to re-evaluate our hypothesis. It's better redo the question. Restarting from scratch:
need: n Edi =1 1
Instead, let's directly use the functional equation and the values we have derived.
Determining Firsst, factorize 2023:2023 = x 172 Using the functional equation and the derived values, we need find f(0233) We can use the multiplicative property of f if we assume
f is multiplicative. That means if n pi p2 ...pt then f(n) = f(ff(((()))))) ( ) ) W.We already have:f(p)
- P for prime p.So: ((7 - f.7
16..Sinee
2023 7 x 17 we have:
= f(7) f(17)). need to determine
f(172) : Edira d 172
= 1.The divisors of 172 are 17, So the sum is:1 f(172) + 17 f177 + 172. f(1) = 1.Substituting f(17g = 1 172 f(17) 1
and f(1) = : + 17 (-16) + 1.111,f111 1 - 289 (17)
= f(7) f(17²) = -1- -96.Thus the value of 1, 1(7() =1 -16-oo
S(2023) is: 96
(Correct Answer)
Figure 22: Re² Examples 4
26


Reviews:
[{"review_id": "pPoAIq2BNE", "reviewer": "Reviewer_Gh1P", "summary": "This paper proposes to allow the model to restart the reasoning path from scratch, preventing the  previous low-quality reasoning steps from hindering the model's reasoning.", "questions": "- The finding in Section 3 about the correlation between the response length and accuracy seems to be contradictory to the finding in Deepseek-R1's paper. Does the author have any insight on this?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The method is well-motivated. I appreciate that the author talks about the problem in section 3 before introducing the proposed method.\n- The idea of allowing the model to restart the reasoning makes a lot of sense to me.\n- The paper is well-written and easy to follow.", "weaknesses": "- The performance gain is limited to me in Table 1. I understand that it may be subjective on whether +5.8 is significant or just noise. I suggest including confidence interval when the results are this close.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "onlrMJbrOe", "reviewer": "Reviewer_WrfW", "summary": "This paper introduces a reinforcement-learning–based training paradigm, Reinforcement Learning with Re-solving (Re²), that teaches language models to abandon unproductive partial chains of thought and restart problem solving when the current trajectory appears misguided. The method frames each query as a distribution over truncated prefixes sampled from full responses; for every prefix, the policy generates multiple continuations that can either commit to a final answer, produce an incorrect answer, or explicitly choose a resolve action that restarts from scratch. Rewards are assigned in three ways: 1 for correct final answers, 0 for incorrect ones, and for resolve actions the expected accuracy of solving from scratch, estimated from out-of-group completions and capped by at most R redo rounds via a closed-form geometric term. Advantages are computed group-wise within each prefix’s continuations and optimized with clipped policy updates, without any preliminary supervised fine-tuning. Across five benchmarks (AIME 2024/2025, AMC 2023, GSM8K, GPQA-Diamond) and five models from 3B to 14B parameters, Re² improves average accuracy over a strong RLVR baseline (DAPO) under matched training token budgets, and it shows stronger test-time scaling as the number of samples increases. The authors also provide analyses indicating that errors in early reasoning correlate with longer, lower-accuracy chains, and they report that Re² amplifies the otherwise rare redo behavior from roughly 0.5% to over 30% during training.", "questions": "How sensitive are the gains to the resolve reward estimator? Please report variance and bias when computing out-of-group accuracies with different n and m, and compare against a baseline that uses a moving average or a separate value head so we can see whether batch-coupled estimates are essential or merely convenient. \n\nBeyond DAPO, could you include GRPO-style objectives, recent efficient RLVR methods, and a decoding-only backtracking strategy under matched compute to clarify whether the benefit stems from the re-solve action space or other training details?\n\n For benchmarks with non-integer answers or programmatic verification (e.g., algebraic expressions or code), do you observe similar trends, and what verifier failure rates do you tolerate before resolve reward becomes noisy? \n\naLso, please provide wall-clock training cost, instability metrics (e.g., degenerate-group rate over time), and an ablation on the template that triggers redo, since the current detection mechanism might entangle prompt engineering with policy learning.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The core idea is original in the RLVR context: instead of privileging a single forward chain, the policy is explicitly trained to reconsider and restart when early evidence suggests confusion. This operationalizes a common human strategy and formalizes it with a clear three-way action space and a principled reward for resolve based on out-of-group success rates with a finite-round geometric expectation, yielding an intuitively aligned decision rule between “continue” and “restart.” \n\nMethodologically, the paper is careful to control training token budgets against DAPO, computes advantages within prefix groups to stabilize learning, and filters degenerate groups to avoid vanishing gradients. The empirical study spans multiple datasets and model types, including an RL-trained reasoning model, and consistently shows nontrivial gains in average accuracy, especially on the most challenging math tasks; furthermore, the test-time scaling plots indicate that Re² keeps improving with more samples where standard RLVR saturates, suggesting a better compute–performance frontier at inference. \n\nThe presentation is clear about the training pipeline, reward derivation, and hyperparameters, and it includes qualitative traces that concretely illustrate how restarting avoids early misdirections that standard methods tend to entrench.", "weaknesses": "Althouth the truncate-then-continuate method is shown to be effecitve and better than DAPO, it is much slower in rollout procedure. I would be interested to see how much computaional cost overhead is added for this continuation step.\n\nWeaknesses\n\nComparisons emphasize DAPO, but the space of modern RLVR baselines is broader (e.g., GRPO-style group objectives, critique-assisted RL, efficient RL variants), and decoding-time strategies that terminate low-confidence chains or backtrack could provide competitive non-RL baselines under similar token budgets; the absence of these ablations weakens the claim of superiority of the paradigm rather than the specific implementation. \n\nOn evaluation, final answers are integers to ease verification, which simplifies reward parsing but narrows scope; it remains unclear whether the approach retains its advantage with symbolic or multi-step numeric answers that require more complex verifiers. \n\nThe test-time comparison that counts redo attempts against the same sample budget is reasonable, yet when the sample count is small the method underperforms because many draws are resolve rather than final answers; guidelines or controllers for resolve frequency at inference are not yet provided, which limits practical deployment where token budgets are tight.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "e5N4qVRuCu", "reviewer": "Reviewer_hpUj", "summary": "The paper introduces a new reinforcement learning framework that allows large language models to restart reasoning when current solution paths become unproductive. Unlike standard RL with verifiable rewards (RLVR), this methods adds a re-solve action that lets the model abandon low-quality chains of thought and reinitiate problem solving. Using a novel three-way reward scheme (correct, incorrect, re-solve), Experiments across five reasoning benchmarks show consistent 4–6% accuracy gains and superior test-time scaling efficiency compared to prior RLVR methods like DAPO.", "questions": "1. I'm curious whether reflection is actually essential for reasoning, or if the ability to restart and resample is more critical. Does $Re^2$’s improvement suggest that reflection (continue generating from a given prefix is not a kind of reflect in my mind) within a single chain is less valuable than exploration through re-solving?\n\n2. How sensitive is $Re^2$ to the number of rollouts during training or inference? Would smaller rollout budgets still yield similar benefits?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper proposes a novel re-solving paradigm that lets LLMs restart reasoning when current paths fail, breaking from the one-shot chain-of-thought assumption in prior RLVR work.\n2. The approach is rigorously implemented, with a clear algorithm, fair baselines, and strong empirical validation across multiple benchmarks.", "weaknesses": "While the paper presents an interesting framework, the underlying mechanisms behind the method's effectiveness are not sufficiently analyzed. It remains unclear why the re-solving behavior leads to higher reasoning accuracy or a higher performance ceiling. For example, does restarting primarily improve exploration diversity, mitigate early trajectory bias, or simply increase the effective number of rollouts? A deeper behavioral or theoretical analysis would make the contribution more convincing.\n\nMoreover, the paper lacks ablation studies to identify which components truly contribute to the gains. The new “redo” prompt, three-way reward, and prefix grouping may each affect learning in different ways. Notably, $Re^2$ and DAPO are trained with different prompts—the former includes explicit self-evaluation and restart instructions—so part of the improvement may arise from prompt design rather than algorithmic changes. Controlled ablations such as removing the “redo” instruction, disabling the re-solve reward, or turning off prefix grouping would better clarify which parts of the method are essential to its success.\n\n\nIf the authors could clarify these questions, it would really help me understand the mechanism better, and I will be happy to raise my score.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XWTQmsXiWt", "reviewer": "Reviewer_q7Jy", "summary": "The authors argue that suboptimal initial reasoning often leads LLMs to inefficient overthinking. To address this, they propose Re^2, a RL framework that enables the model to restart the reasoning from scratch in the middle. Trained purely via RL without supervised fine-tuning, Re^2 significantly improves performance across diverse reasoning benchmarks compared to DAPO, under the same training compute.", "questions": "- In the experiment described in Section 3.2 ('Start generating from x% of incorrect responses'), can we really assume that all the prefixes are incorrect? Even if the final response is incorrect, the first x% of the response may contain some correct reasoning or statements.\n- Some recent papers (2025) use DPO to help LLMs check and correct their own answers without the need for an external model or supervision. Citing these papers would clarify the contribution of Re^2.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper presents a novel reinforcement learning framework that enables large language models to restart the reasoning process when the current trajectory is likely to lead to an incorrect solution.\n- Re^2 consistently outperforms strong baselines across five diverse reasoning benchmarks (AIME24/25, AMC23, GSM8K, GPQA), covering both in-domain and out-of-domain settings.\n- The reward design for the re-solving action is mathematically grounded, relying on estimated future success probabilities, and is formally derived in the appendix.\n- Models trained with Re^2 exhibit favorable scaling with test-time compute, in contrast to DAPO-trained models whose performance tends to plateau or even degrade under increased sampling.", "weaknesses": "- The paper only compares Re^2 with DAPO, omitting other competitive baselines. Additionally, the paper only reports test-time scaling results using majority voting.\n- Re^2 introduces an additional cost at inference time due to repeated re-solving, and there is no mechanism to regulate its frequency. This may be problematic in latency-sensitive settings.\n- Under majority voting with a small number of samples, Re^2 underperforms DAPO. This is counterintuitive, given that Re^2 outperforms DAPO when the number of samples is 1 (see Table 1).\n- The approach assumes access to verifiable reward signals, which limits its applicability to tasks with clear correctness criteria and makes generalisation to open-ended domains uncertain.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)