Once-More: Continuous Self-Correction for Large Language Models via Perplexity-Guided Intervention
Authors: Jiaxun Gao
Him Wai Ng
Z. Jane Wang
Venue: ICLR 2026 Poster (2026)
Keywords: Large Language Model
Natural Language Processing
Self-Correction
Agent
Guided Generation
Post-Hoc Refinement
Primary area: foundation or frontier models, including LLMs
TL;DR: Once-More is a training-free framework that prevents LLM errors from compounding by using token-level perplexity and verifier feedback to enable inference-time self-correction via logit redistribution.

Abstract: Large Language Models (LLMs) often experience compounding errors during long text generation. Early mistakes can propagate and lead to drift, faulty reasoning, or repetition. While scaling up models improves capabilities, it requires substantial computational resources, and the resulting self-correction behaviour remains unpredictable at inference time. Self-correction is a promising technique for addressing this issue. However, existing approaches have limitations. Supervised training methods can build self-correcting behaviours into models, but require training data collection and lack cross-domain generalizability. Current post-hoc iterative refinement methods operate only at inference time, but must wait for substantial portions of the draft to be generated before providing feedback. This feedback does not guarantee effective guidance, and the same mistake patterns can still reappear. In this paper, we introduce Once-More, a model-agnostic post-hoc self-correction framework that intervenes during generation. Once-More leverages token-level perplexity and feedback from verifiers to provide continuous guided steering of the generation path through a logit redistribution mechanism. This approach essentially helps accumulate "more correct" steps throughout the generation process. Evaluation on multiple benchmarks demonstrates that Once-More achieves state-of-the-art results compared to other self-correction methods. To our knowledge, Once-More is the first post-hoc method to leverage token perplexity and external feedback to perform continuous guided self-correction.

Published as a conference paper at ICLR 2026
ONCE-MORE: CONTINUOUS SELF-CORRECTION FOR LARGE LANGUAGE MODELS VIA PERPLEXITY-GUDDED INTERVENTION
Jiaxun Gao"
Him Wai Ng Jane Wang
Department of Electrical and Computer Engineering University of British Columbia
j jiaxun. gao@ece.oo..goooooooooooooo..... ubc. ca michael .ng.ue....@ubcc... zjanewBece. ubc. ca
ABSTRACT
Large Language Models (LLMs) often experience compounding enrors during long text generation. Early errors could propagate and lead to drift, faulty rea- soning, repetition. Self-correction is a promising technique for addressing this issuc. However, the existing main approaches have limitations. Supervised training methods can build self-correcting behaviours into models, but require training data collection and lack cross-doman generalizability. Current post-hoc iterative refinement methods operate only at inference time, but have wait for substantial portions of the draft to be generated before providing feedback. Such feedback can not guarantee effective guidance, and the same error patterns can reappear. In this paper, we propose Once-More, a model-agnostic post-hoc self-correction framework that intervenes during generation. Once-Mort leverages token-level perplexity and verifier feedback to provide continuous, guided steering of the gener- ation path through a logit suppression mechanism, and therefore helps accumulate "more correct" steps throughout the generation process. Evaluation multiple benchmarks demonstrates that the proposed Once-More achieves state-of-the-art re- sults when compared to representative self-correction methods. To our knowledge, Once-More is the first post-hoc method to leverage token perplexity and external feedback for continuous, guided self-correction.
INTRODUCTION
Large Language Models (LLMs) have demonstrated remarkable capabilities in text generation and complex reasoning tasks (Sun et al. 2025 Ferrag et al. 2025). However, their autoregressive nature poses fundamental challenge: during generation, LLMs can produce errors or inaccuracies that further propagate through subsequent tokens, making errors to compound and eventually drive the generation away from the target (Arbuzov al., [0255] Wang et al.) 2023). This eroo-comppunding issue raises serious concern when deploying LLMs in critical decision-making processes. Many LLM-based agentic workflows suffer from reliability issues stemming from this concern, which limits their viability for real-world deployment. (Pan et al. 2025 Gabison & Xian 2025).
To address such concerns, researchers have developed various self-correction methods which enable LLMs to modify or revise their outputs at inference (Kamoi et al.) 2024| Pan et al., 2024). Such methods identify errors during generation and guide models toward better responses through targeted feedback mechanisms (Paul al. 2023 Tyen et al., 2023 Shinn et al. 2023). The feedback
the final output is provided via external knowledge bases (Gou et al.
ZZengg et al.) 2024) or
by more capable models (e.g., GPT-4) (Koutcheme al. (022). However final-output feedback is too coarse to provide effective steering, and thus these methods often fail to prevent recurring
error patterns and lead con-convergent refinement loops (Kamoi al.) 2024 Xu al. 2024).
Some methods use supervised fine-tuning as an alternative to build self-correction behaviour directly
into models (an al. 2025 Liu al. 2025 Wang et al.) 2024a); however, they are inherently
"Co-first authors. Listed alphabetically by surname.

---

Published as a conference paper at ICLR 2026
Input
checked by: Perplenity Verifir(s)
Propase
Producer LLM
Prooet to Regenerate Unit
Is Unit Y Enough?
Cowelt Output
Cowline Feedlaack Perplesity bazed
No! (Heeds Meri)
#0
Generation Done?
Final Output
Figure I: Overview of the proposed Once-More framework (Sec. 3.1 The Producer generates adaptive units, which checked by Perplexity and Verifier(s). Rejected units trigger regeneration via the combined feedback and perplexity driven guidance, creating a continuous self-correction process that intervenes before errors propagate.
limited by the distribution of their training data. They are susceptible to performance degradation on out-of-distribution tasks, where uncorrected errors can cascade through the generation.
Rather than building domain-specific models with built-in self-correction, we believe that itera- tive refinement remains the most general and practical approach. However, it requires fundamental improvements in guidance granularity. Effective self-correction should function continuously through- out generation, progressively steering models along better trajectories. and eventually allow "more correct" incremental decisions to accumulate into better final outputs. Motivated by this vision, in this paper, we propose an inference-time framework, Once-More, which performs continuous self-correction any LLM generation process. It transforms the LLM generation process into multi-agent interaction process: a Producer generates content while Verifier(s) evaluate and provide continuous guidance. Once-More operates at the level of units of generation (c.g.. clauses, sentences, paragraphs, or code blocks). A complete generation is a sequential composition of unit generations. When the Producer generates unit, the framework computes its perplexity to assess potential errors. High perplexity triggers verification for goal alignment, correctness, and consistency. Rejected units regenerated using evaluation feedback combined with perplexity-driven logit redistribution until the Verifier(s) accept the unit. Figure shows the high-level overview of the proposed Once-More. The framework also supports LLM. non-LLM, or cool-augmented Verifiers.
The proposed Once-More are evaluated on various benchmarks, including Olympiad mathematics (AIME 2024 (Art of Problem Solving 2024a b) and 2025 (Art of Problem Solving 2025a b)). graduate-leved science questions (GPQA) (Rein et al. (2024). LiveBench (reasoning) (White et al.)
2024), SVAMP (Patel al., (02I). and GSMBK (Cobbe et al. 202I). It achieves sfate-of-the-art performance when compared to other self-correction methods. Our contributions are as follows: (1) We propose the Once-More framework to perform continuous, fine-grained self-correction, which yields State-of-the-Ar Art performance while being more token-efficient than previous iterative refinement methods; (2) We present a method to uniquely fuse perplexity-ban based uncertainty signals prevent repeated errors while
with external verifier feedback. It is employed in Once-More preserving model beliefs during generation. To our knowledge, the proposed Once-More is the first post-hoc method to leverage token perplexity and external feedback for continuous, guided self-correction; (3) Our proposed continuous guidance mechanism in Once-More enables LLMs to perform reasoning-like corrections dynamically during generation, which helps detect early errors and move to better generation trajectories.
RELATED WORK
Error accumulation in LLMs. The problem of eπror propagation in autoregressive models has been extensively studied. Classical analyses of exposure bias demonstrate that models trained on teacher-forced prefixes often struggle test time, leading error compounding in long generations
2

---

Published as a conference paper at ICLR 2026
2022; Schmidt 2019). LLMs also suffer from the error accumulation issue, although LLM's error accumulation patterns are more nuanced than simple exponential decay (Arbuzov et al. While scaling up LLMs enables the emergence of self-correction behaviours (Liu al. 2024 Wang et al. 2024b), they remain unpredictable and uncontrollable at inference. To address this concern, we propose making self-correction explicit and controllable through serplexity-guiders intervention. Our proposed approach is model-agnostic and can be used with smaller-sized LLMs, making it more practical for local or edge use cases.
(Arora et al.
2025).
Self-correction via iterative refinement. Several approaches can enable LLMs to revise their outputs through iterative processes. For instance, CRITIC uses external tools verify model outputs and provides corrective feedback (Gou et al., 2024). Self-Refine employs models as both generators and evaluators (Madaan et al. 2023), and Verify-and- Edit incorporates knowledge bases for factual correction (Zhao et al., (023). While these approaches can improve response quality, they operate on completed drafts or coarse-graine
steps, and therefore could allow errors to compound before intervention occurs. The prompt-only feedback also occasionally fails to prevent recurring error patterns. Our proposed Once-More addresses the above concerns by intervening at a more granular level with continuous monitoring, enabling more direct and localized feedback that has more effective impact on the model's generation process.
Self-correction via supervised fine-tuning. Another line of work builds self-correction behaviours directly into the models through training. For instance, S3c-MATH trains spontaneous self-correction for mathematical reasoning (Yan et al., (2025), LLaVA-SCo extends this vision-language models approach finetunes models to internalize correction
(Lu et al., 2025), and the Bearnnngffrom-ti behaviours (Wang al. 2024a). However, their gains remain modest, and these methods require extensive training data and are susceptible to performance degradation on out-of-distribution tasks. To tackle such issues, our proposed framework remains model-aggostee and training free, while providing continuous guidance for self-correction behaviours.
Multi-agent and role-playing approaches. Formulating generation as a multi-agent interaction has been shown to be effective for complex tasks. E.g., Reflexion equips agents with verbal feedback and episodic memory to improve over trials (Shinn et al.) 2023), MAgICoRe combines solver, reviewer, and refiner roles with reward models for targeted step-wise error correction (Chen et al. 20024), and ReAct interleaves reasoning with external actions to reduce hallucinations (Yao et al. (023).. These methods leverage role specialization to provide feedback in different perspectives, and inspired by their success, our proposed Once-More adopts this idea in its Producer- Verifier architecture. However, their coarse-grained feedback and prompt-only interactions cannot guarantee effective steering. We therefore plan to enhance the multi-agent system with continuous monitoring and direct probability intervention.
Decoding time steering. The approach that manipulates sampling at inference time offers another avenue for controlling generation behaviour. E.g., GeDi guides generation toward desired attributes
with discriminative models (Krause et al., 2020). Contrastive Decoding reduces hallucination by
subtracting weaker model logits (Li et al. [2022] O Brien & Lewis 2023), and DoLa sharpens
factuality through layer-wise contrast (Chuang et al. 2023). Recent work on Cautious Next Token Prediction demonstrates that perplexity-basee mechanisms can effectively guide generation decisions (Wang et al.) (0255), providing a theoretical basis for our approach. However, existing methods only operate globally or target high-level attributes, lacking fine-grained control. Inspired by these methods, our proposed method manipulates sampling at inference time, but also employs localized logit manipulation to perform a more effective generation steering.
3 ONCE-MORE FRAMEWORK
We now present Once-More, a model-agnostic framework that performs continuous self-correction during LLM generation. The key idea is simple: rather than waiting for a complete output before providing feedback, we monitor generation quality at intermediate steps and intervene immediately when potential errors are detected. This section describes the framework's 's architecture (Sec. 3.I). how it monitors generation quality (Sec. 3.2). and how it guides corrections (Sec. 3.3)
3

---

Published as a conference paper at ICLR 2026
Input (AIM)22 problem (3)
Language model answer
(1001 + 100 + (100d 10ee 999
Accasted uni
Step 2: The verical
X
Rejected unit
Feed back
(The veriiall is ans mustt invile multi-d tiiidgg rumbens, not single dights Re-eoprees the vertical mi)
of nationa that
Regenerated Rejected unit
The "the of thrce numbers number 10d.
"hht
would
Inager
imsgee
(Cliniy isp-to-botion numbbers forred (e sing engleddiggt namber or mu (i-dig(()
(3) ad
Inogee
winger
cf
99
Acceptted unit Acsepted
impossible the masimum
numbers formed
invllid.
The number box muiti- digit number
Let's try different interpettaionn num bers are formed as twa-dfiii numbber
Arzapted unit
The sur of the three columi-irse twp-digt numbers rige..
Apcepteed unit
(10e 1 d) (100 c) (10c f)
...
Figure 2: Illustration example: The proposed Once-More corrects mathematical interpretation error on AIME 2024 Problem 83. rejects an incorrect single-digit interpretation (unit 3), provides feedback, and guides the model to reconsider. The model then explicitly reasons why the initial is impossible and pivots to the correct two-digit interpretation, demonstrating intermediate reasoning.
3.1 FRAMEWORK ARCHITECTURE
As shown in Figure L Once-More transforms standard LLM generation into monitored multi-agent process with three core components:
Producer. A frozen, pre-trained LLM that generates content incrementally in units (adaptive chunks such as sentences, paragraphs, or code blocks). The Producer operates at inference time without any training or fine-tuning. The only requirement access token-level probability scores during decoding, which many providers already expose through their APIs. This requirement matters only for our regeneration mechanism and doesn't constrain the choice of model architecture.
requrregueeee
Verifier(s). One more agents that evaluate each provisional unit against the task Goal and constraints, returning binary decision (accept or reject) along with optional feedback F in natural language or structured format. Verifiers may be LLMs, non non-LLM programs, or tool-augmented modules as in tool-using agents (Yao et al. 2023 Shen et al. 2023
Shen 2024).
Generation Units. Rather than working with fixed token windows, Once-More adapts its interven-
tion granularity the task context. For mathematical reasoning, a unit might be single equation derivation step; for code generation, function or block; for prose, a sentence or paragraph. Units are identified using syntactic markers (punctuation, indentation) learned boundary predictors. This adaptivity enables intervention at the most appropriate granularity for each task.
The Generation-VVriffcattoon Loop. The proposed Once-More framework operates through contin- generation, monitoring, and correction loops. Figure drauusstttmsstteeeccc demonstrates this mathematical problem, showing how the framework catches and corrects an incorrect interpretation mid-solution. Let Goal denote the task and Context the generation history that has been verified and accepted. The framework operates through the following loop:
1. Generate: The Producer generates provisional unit Y "u.....,be conditioned on (Goal, Context), where y; one output token at time
Monitor:
Compute the unit's perplexity PPIumi(Y) an uncertainty signal (Sec. 3.2
3. Decide:
If PPL (Y) < Ps. (low uncertainty), trust the unit and append to Context.
If PP (Y) Pa. (high uncertainty), invoke Verifier for explicit checking.
4. Verify & Correct:
4

---

Published as a conference paper at ICLR 2026
Perplesity Correct and Inconeet Answers A0M43004
Distibution of Mean 10p-10% Perplexity for Cornect mncoree Answers GRGMD
Distriuttonn Mean Top 10% Perplexity for Correct and Incornect Amswers in Livellench
Corect moar
Carrct
90
10-
30
2.6
3.0
3.4
2.5 Moon Mp--2R Magrindy
Moan Top 10% Perp loity
Mhon Mg--10 Marg
(a) AIME24
(b) GPQA-Diamond
(c) LiveBench(Reasoning
Figure Distributions of the mean top-10% 10% perplexity for correct vs. incorrect answers across benchmarks. Incorrect units consistently exhibit higher perplexity.
accepted:: Append to Context and create a checkpoint.
If rejected: Trigger guided regeneration (Sec. 3.3 using verifier feedback and probability adjustments.
If regeneration also fails: Roll back to the previous checkpoint and regenerate from there, the error may have originated earlier.
3.2 MONITORING GENERATION QUALITY VIA PERPLEXITY
To detect potential errors during generation, need real-time quality signal that doesn't 't require external verification for every token. We leverage perplexity, which is standard language modelling metric that measures prediction uncertainty (Jelinek et al. 1977 Christopher Manning [021] Hugging Face 2025). Recent work shows that entropy-based metrics provide reliable signals for response
quality (Fu al. 2025
Yang al. 2025b Wang et al. 2025a), which motivates our approach.
candidate), perplexity low. When probability mass spreads across multiple candidates, perplexity increases, signalling uncertainty. High uncertainty often correlates with potential errors, indicating the model is likely unsure what to generate next, and it's probably on the wrong path.
Intuition. When model confidently predicts the next token (high probability
Token-Level Perplexity, At each position t, the Producer outputs a probability distribution qi (v) the vocabulary V. Rather than computing perplexity over the entire vocabulary (expensive). approximate using the top-K most likely tokens S: = [vr,1..... v (Holtzman et al. 2018 Fan
al. 2018)
K
PPI. (K)
1
Σ
- log de (v..))
exp
K 1=1
This efficiently captures local uncertainty: PPI. (K) mass spreads across alternatives.
when one token dominates, and increases as
Unit-Level Aggregation & Verification Trigger. For a unit Y [y....., .,,,, we average token- level perplexities:
PPL (Y)
PPL (K)
Verification is triggered when PPLunit (Y) > Pin. where Pt is calibrated on small held-out set to target a desired verification rate (e.g., check the top 25% most uncertain units). Details of the calibration procedure in Appendix A.22 Figure [egureeBempr empirically validates this approach: benchmarks, incorrect units consistently show substantially higher perplexity than correct ones, demonstrating clear separation between reliable and problematic generations.
5

---

Published as a conference paper at ICLR 2026
3.3 GUIDED REGENERATION VIA PROBABILITY ADJUSTMENT
When a unit is rejected, simply asking the model to "try again" often produces the same mistake, since the model's learned biases lead it down the same path (Xu al. 2024). The model may acknowledge the feedback but still sample the same tokens due to strong learned priors. To break this cycle, Once-More performs guided regeneration: it (1) incorporates verifier feedback into the prompt. and (2) directly adjusts the token probability distribution to suppress previously chosen tokens and explore alternatives.
Overview of the Adjustment Mechanism. The adjustment process operates at the token level during regeneration:
1. Identify high-uncertainty tokens: Convert the rejected unit's token-level perplexities into position-wise suppression strengths; therefore, tokens with higher uncertainty receive stronger suppression.
2. Align attempts: Match tokens between the rejected attempt and the current regeneration attempt to determine which tokens suppress each position.
3. Redistribute probability mass: At each position, decrease the probability of the previously chosen token and proportionally increase alternatives, ensuring a valid probability distribution.
We now detail each step.
Step 1: Converting Perplexity to Suppression Strength. Let [Z1...... Ln denote the rejected unit with token-level perplexities (PPL,)_>) We normalize these into suppression weights:
PPL: mPP(P
E [0.1]
max(PPL) min (PPL) +E
where is small constant for numerical stability. This maps perplexity a [0,1] scale: tokens with high perplexity (high uncertainty) get suppression weights near 1, while confident tokens get weights near 0.
Step 2: Aligning Regeneration with Previous Attempt. When regenerating, the new token sequence (y1..... ynl may differ in length and content from [æ1..... , Fy_. We establish a monotone alignment to transfer suppression weights: for each position j in the new attempt, find the first matching token x; from the previous attempt (if any), creating an alignment matrix A E {0, 1)" xn.
This identifies which token should be suppressed at each regeneration position:
Iz. if match exists at position i bz otherwise (use position- based fallback)
target, 11,)
To avoid over-suppressing tokens that matched only by coincidence (e.g., common words appearing at distant positions), we apply distance decay:
4j"t Aij exp (xp((((()1) a(
(5)
where T controls the decay rate and E 2} the decay shape. This reduces suppression strength for matches far from their original position.
We further apply Gaussian smoothing to prevent over-localization, spreading suppression signals across nearby positions (details in Appendix A.3). The final effective suppression at position j is:
aj uj. 1)
where uz blends the direct and smoothed suppression signals, and a is global scaling factor.
Step 3: Probability Redistribution. Let (2) (v) denote the Producer's 's original next-token distribu- tion position during regeneration. We adjust this distribution suppress the target token while boosting alternatives:
1 - agrr(1),
target,
(suppress)
(v) =
(7)
[1+Kg e target, (boost)

---

Published as a conference paper at ICLR 2026
where ry(1) -,) (u)/ max of (v)) measures how much the model's belief about token has shifted between attempts. The coefficient Kj is computed to preserve probability mass:
Og Og (ttarget,, 4j (target,)
Kj
(2) Oj (u)
uiffurgee
ensuring that the total probability removed from the target equals the total added to alternatives (derivation in Appendix A.4). The parameter 3 2 controls how redistributed mass is allocated and set to for all our experiments.
The final adjusted distribution is:
(v) (5))
(b) Σver 9j (2) (u) sy(u)
This is valid probability distribution (non-negative, to 1) that explores alternatives while respecting the model's learned beliefs.
Regeneration Process. With the adjusted distribution (-), the Producer samples a new unit. The verifier feedback is also appended to the prompt context, providing semantic guidance alongside probability adjustment. If this regenerated unit accepted, becomes the new checkpoint. rejected again, the framework rolls back to the previous checkpoint (before the problematic unit) and regenerates from there, recognizing that the error may have originated earlier in the generation. By combining perplexity-drice uncertainty detection with probability intervention, Once-More achieves continuous, fine-grate oneingrined self-correction that prevents error accumulation during generation.
4
EXPERIMENTS
4.1 SETUP AND BASELINES
Due to the rapid development of the LLM field, prior self-correction studies employ diverse experi- mental setups across different models and benchmarks. To establish reproducible comparisons, we selected the popular open-sourced Qwen3 family of models (4B, 8B, and 14B parameters) as our base models (non-thinking) (Yang et al. 2025a). These model sizes were chosen for three key reasons: (1) they span small medium scales, revealing how self-correction scales with capacity; (2) they represent practical compute- condtriied scenarios; (3) Qwen3 offers state-of-the-atr performance these sizes, ensuring our results are relevant to current practice.
We evaluated Once-More on six benchmarks: AIME 2024 (Art of Problem Solving 2024a by and 2025 (Art of Problem Solving 2025a b) for complex mathematical reasoning. LiveBench (reasoning subset) [Rein et al. (024)) for general reasoning. GPQA Diamond (Rein et al.) (2024) for graduate-level
science questions, and SVMMP (Patel et al.) 2021) and GSM8K (Cobbe et al., 2021) for arithmetic word problems. This benchmark selection covers diverse reasoning types and difficulty levels.
We compare the proposed Once-More with four representative baselines:
Raw: Direct generation without self-correction.
Self-Refine (Madaan et al.
2023 J The most widely-cited iterative refinement method,
representing basedt-based self- self-correction approaches.
CRITIC (Gou et al. 2024) Incorporates external tools for verification, representing tool-
augmented corfecoore methods and most recent post-hoc method.
c-MATH Yan et al. 2025): supervised fine-tuning approach, representing training- based self-correction methods.
Here, implemented the Self-Refine and CRITIC baselines according their references. For Self-Refine, the number of refinement iterations was set to = 3. For CRITIC, since no suitable resources were specified in the original work, we adapted the method leaving the external evidence
7

---

Published as a conference paper at ICLR 2026
Table Accuracy performance (mean accuracy std. dev. over runs) comparisons between the proposed Once-More and baseline self-correction methods across multiple benchmarks. Results are for Qwen3 models of varying sizes (4B, 8B, 14B parameters). Once-More consistently outperforms iterative refinement approaches (Self-Refine, CRITIC) and raw generation.
ATME24
AIME25
LiveBenc((Reason)
GPQA Diamond
Raw
13.3 #3.3
23.3
20.3 =1.2
43.9 +1.3
SelfRefine
13.3
18.9 +3.8
21.0 +0.9
43.9 +
Qwen3 4B
CRITIC
14.4+119
20.0++3.3
20.7+1..
48.9+
ours
16.7+33.4
23.3+33.4
33.0++1.0
47.5+0.99
Raw
24.4=11.9
16.743.4
30.5+0..9
45.4++0.9
SelfRefine
25.6++3.9
15,5+3.9
31.2+
17.0+1.7
Qwen3 8B
CRITIC
24.4=119
18.8+3.9
31.5+11.
48.9+1.5
33.3.13.33
24.4.19
39.5
49.5
Raw 26.7=33.3
18.8+3.8
44,0+1.3
1.,0=1.5
SelfRefine
28.8=3.88
23.3+333
46.5
49.2+
Qwen3 14B
CRITIC
28.8-11.9
22.2.13.9
45.5
10.
ours 36.7
26.7433.3
52.5
55.6 1.6
Table Accuracy performance comparisons with supervised fine-tuning methods mathematical benchmarks. Once-More achieves competitive or superior performance without any training.
Llama 3 8B
Qwen2 Math 7B
MetaMath SFT
$ w/o R&I
S MathQA
MetaMath SFT
S
S MathQA
Benchmark
Ours
Ours
R&I
SVAMP
81.4
80.5 81.7
81.8
82.0
85.6
86.7
87.4
89.0
GSM8K
81.1
82.9
79.0
84.1
84.4
84.8
85.2
field empty in math experiments. On the GPQA Diamond benchmark, CRITIC the first 1000 words from web search results as external evidence. S3c-MATH results are taken from their published papers on comparable models. All reported results represent means over three independent runs. For Once-More's default configuration, we set K = for top- K perplexity calculation, with suppression factor a = 1.0, redistribution sharpness B 1.0, distance decay 1, diffusion bandwidth a and perplexity threshold 25% fixed. The verifier feedback is generated from agent using the same LLM as the producer agent across all experiments.
4.2
MAIN RESULTS
Table|[presentt performance comparisons across the AIME24/25, LiveBench (reasoning), and GPQA Diamond benchmarks. Our proposed Once-More consistently outperforms all baselines across model sizes and task types. On mathematical reasoning tasks (AIME 2024/2025), Once-More achieves 3.4 to 10 point gains over raw generation, with the 14B model reaching 66.7% on AIME 2024 and 26.7% on AIME 2025. contrast, Self-Refine and CRITIC show minimal improvements. gaining at most 2 to 4.5 points. For general reasoning (LiveBench). Once-More delivers 9.0 to 12.7 point improvements, while baselines achieve less than point gain. On GPQA Diamond, all model sizes show consistent 3.6 to 7.6 point improvements with Once-More. Table | compares Once-More against supervised fine-tuning approaches, including MetaMath-SFT and M'c-MATH. Once-More achieves the best performance on SVAMP across both model families (82.0% for Llama 8B, 89.0% for Qwen2 Math 7B) without any training. While 53c-MATH outperforms Once-More on GSM8K for Llama (82.9% 79.0%), Once-More leads Qwen2 Math (85.2% 84.8%). The results demonstrate that Once- Moce-More can match or exceed specialized training methods while maintaining model-agnostic deployment.
8

---

Published as a conference paper at ICLR 2026
Average Wall-cook Time Arsaver Taxt and Mattood Owend 345 model
Average Thisen Usage by Tiik and Wethod on Gween 140 model
5,000
Sertccre
+,003
Acorogh
(a) Average wall-clock time for answer.
(b) Average token usage for
Figure 4: Run time comparisons across tasks on Qwen3-14B.
4.3 ANALYSIS AND ABLATION STUDY
Scaling effects self-correction. On AIME 2024, Once-More's improvements scale progressively: 25.6% relative gain for 4B, 36.4% for 8B, and 37.4% for 14B models. This scaling pattern suggests that self-correction benefits from richer internal representations in larger models. They can generate more informative feedback and possess more nuanced token distributions for effective logit redistribution. The same pattern holds across AIME 2025, where gains increase from 16.5% (4B) to 42% (14B). This indicates that Once-More can scale with model capacity, unlike post-hoc methods that show diminishing returns.
Small models and the feedback quality problem. Existing self- ceff-correction methods do not work on smaller models. Self-Refine shows no improvement on the 4B model AIME 2024 and actually degrades performance on AIME 2025 for the 8B model 166.%%+15.5%% CRITIC fares slightly better but still achieves minimal gains. This is likely due to smaller models' limited capabilities to produce meaningful feedback on long outputs. Once-More mitigates this by combining verifier feedback with logit redistribution, ensuring corrections happen even when feedback quality low. The finer unit-level granularity also makes feedback inherently easier for verifiers provide.
Mathematical reasoning improvement. On AIME24 and AIME25, Once-More substantially outperforms CRITIC and Self-Refine. This is because mathematical errors are unforgiving, a single misstep can invalidate an entire solution. This makes existing post-hoc methods less reliable as they must locate the faulty step in complete solution and regenerate a full answer. In contrast, Once-More's continuous monitoring catches mistakes they occur, enabling correction at the point of failure rather than after completion.
Efficiency analysis. Figure compares wall-clock time and token consumption across methods on Qwen3 14B. Despite regeneration cycles, Once-More demonstrates competitive wall-clock time, matching outperforming Gelf-Refine and CRITIC across all tasks while achieving superior accuracy. For token usage, Once-More consistently achieves the lowest consumption among self colf-correction methods, using 17-21% % fewer tokens than Self-Refine AIME tasks. This efficiency advantage persists across other benchmarks. Notably, GPQA Diamond exhibits shorter generations across all methods, providing fewer opportunities for continuous intervention and partially explaining the smaller performance gains on this task. Overall, Once-More achieves superior accuracy without incurring additional computational costs compared to existing self-correction approaches.
Effects of verifier feedback and logit redistribution Table evaluates the contributions of feedback and logit redistribution on Qwen3-14B. Removing redistribution but keeping feedback yields 33.3% on AIME24 (+6.6) and 51.5% on GPQA (+3.5). Removing feedback but keeping redistribution gives 30.3% (+3.6) and 48.4% (+0.4). The full system reaches 36.7% and 55.6%, which are gains of +10.0 and +7.6 over raw. The effects are roughly additive on AIME24 and clearly more than additive GPQA, suggesting that feedback and redistribution resolve different failure modes: feedback supplies semantic guidance about the error, and redistribution enforces exploration away from the previously chosen tokens that the guidance actually changes the trajectory.

---

Published as a conference paper at ICLR 2026
Table Ablation study on Once-More components using Qwen3-14B. Both verifier feedback and logit redistribution contribute substantially, with the best accuracy performance from combining both.
Method
AIME24
GPQA
Raw
26.7
48.0
w/o Redistribution
33.3
51.5
Feedback
30.3
48.4
Full Once-more
36.7
55.6
Table Unit length
ablation
study
using
Qwen3-14B, with mean accuracy (%)
+ std. dev. over 3 runs.
Unit length (sentences)
Benchmark
32
128
AIME24
36.6 + 3.3
34.3 + 1.9
35.5
1.9
30.0 + 2.7
26.7 2.7
26.7 + 2.7
LiveBench
52.3 + 1.2
52.1 = 1.0
52.3 - 1.2
49.8 + 1.7
50.3 - 1.3
45.3 0.6
GPQA-D
55.6 1: 1.8
55.5 +
1.3
56.1 +
1.5
49.7
1
1.0
49.3
1 0.3
50.0
1.7
Effects of answer unit granularity eealuates the contributions of answer unit granularity on Qwen3-14B model under default htyer-paramette setting. The results do not indicate significant performance differences for small unit lengths (S 4 sentences). This shows that the proposed Once- More framework is robust under fine-grained segmentation. However, when the unit length increases significantly (2 32 sentences), performance drops across all three benchmarks and converges to the performance of the raw model (without Once-More) unit length of 128.
Table 5: Accuracy (%) results under different Producer-Verifier configurations and raw output.
Producer:
Qwen-1 14B
Qwen-4B
Verifier:
14B
8B
4B
Raw
14B
8B
4B
Raw
AIME24
36.6
36.6
30.0
26.6
33.3
23.3
16.7
13.3
LiveBench
52.5
49.5
46.5
44.0
41.0
38.5
33.0
20.3
GPQA-D
55.6
51.5
50.0
48.0
54.0
52.5
47.5
43.9
Asymmetric settings of Producer/Verifier the durong-produce / weak-verifier and the weak-producer strong-verifier. Experiments conducted under default hyperparameters. Results show that, with a strong Producer (Qwen-14B) performance degrades only mildly with weaker verifiers (4B/8B), remaining substantially better than raw baseline. With weak Producer (Qwen-4B), increasing verifier strength (using 14B) yields massive improvements across all benchmarks.
Table[]preeents the results using both configurations of
CONCLUSION
We presented Once-More, model-agnostic,, training-free framework for continuous self-correction during generation, which monitors uncertainty at the unit level, invokes verifier feedback when needed, and enforces exploration via perplexity-guieee logit redistribution. This fine-grained, intervene-as. you-go design efficiently reduces eтror propagation and turns incremental improvements at each unit into stronger end-to-end generations. When tested on diverse benchmarks and model sizes, the proposed Once-More consistently outperforms post-hoc refinement baselines while remaining token- efficient, demonstrating that controllable, inference- steering can yield reliable gains without additional training. Despite these gains, we acknowledge specific limitations: the framework may struggle to trace errors rooted deep in the generation history, potentially leading to regeneration loops; confident errors (false negatives) may occasionally bypass the perplexity trigger; and the system's performance ceiling remains influenced by verifier quality. Future work will focus on addressing these challenges through dynamic rollhack mechanisms and adaptive thresholding.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
To ensure the reproducibility of our work, we provide comprehensive implementation details through- the paper and supplementary materials. The Once-More framework's complete algorithmic description appears in Appendix A.I with mathematical formulations for perplexity computation (Section (3.2) and logit redistribution mechanism (Section B.3). Experimental setup details, including all hyperparameters (K 5, 1.0, 3 = 1.0). model specifications (Qwen3 4B/8B/14B), and benchmark descriptions are provided in Section [4.1] We also detail our implementation procedures for Self- and CRITIC, including specific adaptations for mathematical tasks. All experiments use publicly available models and benchmarks (AIME 2024/2025, LiveBench, GPQA Diamond, SVAMP, GSM8K) with standard evaluation metrics. We report results averaged over three inddpen- dent runs to account for stochastic variation. A GitHub repository will be used to host the complete implementation of Once-More after this work is published. It will include the Producer- Verifier architecture, perplexity monitoring system, logit redistribution algorithm, and scripts to reproduce all experimental results.
REFERENCES
Mikhail Arbuzov, Alexey Shvets, and Sisong Beir. Beyond exponential decay: Rethinking error accumulation in large language models. arXiv preprint arXiv:2505.2418% 2025.
Kushal Arora, Layla EI Asri, Hareesh Bahuleyan, and Jackie Cheung. Why exposure bias matters: An imitation learning perspective of error accumulation in language generation. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Findings of the Association for Computational Lingnistics: ACL 2022, pp. 700-710, Dublin, Ireland, May 2022. Association for Computational Linguistics. doi: 10. 186111/11111/000 1/2022. 12222//.in- andingg--all.8.. URL 2022. fe2..fiidingg-aaae 58/]
Art of Problem Solving. 2024 aime htttesr/arroffrroleeeel/vvveenrr
index.php/2000 AIME I 2024a. Accessed: 2025.
Art of Problem Solving. 2024 aime ii |etpes//aatoorrrooommo/////g.o/.
Lndex.ppp/200 20::0024_AMMMM_ AIME_II 2024b. Accessed: 2025.
Art of Problem Solving. 2025 aime i. httpes//arroofrroooeeeeee///...o fndex.ppp//023 AIME 2025a. Accessed: 2025.
Art of Problem Solving. 2024 aime ii httpps//aatccfcccceeeeeevvvoooo.
pndex.php/20255 A1005__AAM_____ ATME_TI 2025b. Accessed: 2025.
Justin Chih- Yao Chen, Archiki Prasad, Swarnadeep Saha, Elias Stengel-Eskin, and Mohit Bansal. Magicore: Multi-agent, iterative, coarse-to-fine refinement for reasoning. arXiv preprint
arXiv:2409.12147 2024.
Christopher Manning, Natural language processing with deep learning cs224n/ling284, lecture
6. https: web.
1214/
s 2lides/cs2224n-22221 - hatpraa-leeeeeee--aaaa-----aaaae Laancy-rnn.par 2021. Accessed: 2025.
Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv preprint (0t::2309.03883.. 2023.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:21 10. /4168, 2021.
Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. arXiv preprint arXiv:1805.04833 2018
Mohamed Amine Ferrag, Norbert Tihanyi, and Merouane Debbah. From Ilm reasoning to autonomous ai agents: A comprehensive review. arXiv preprint arXiv:2504. /9678, 2025.
11

---

Published as a conference paper at ICLR 2026
Yichao Fu, Xuewei Wang, Yuandong Tian, and Jiawei Zhao. Deep think with confidence, 2025. URL aattps://arxi.. http://.twwvvo.....//.oo org/ abs///ss/////
Garry A Gabison and R Patrick Xian. Inherent and emergent liability issues in Ilm-based agentic systems: principa--agen perspective. arXiv preprint arXiv:2504. 03255,, 2025.
Zhibin Gou, Zhihong Shao, Yeyun Gong, yelong shen, Yujuu Yang, Nan Duan, and Weizhu Chen. CRITIC: Large language models can self-correct with tool-interactive critiquing. In The Twelfth International Conference on Learning Representations, 2024. URL hattps:/
jek
Ari Holtzman, Jan Buys, Maxwell Forbes, Antoine Bosselut, David Golub, and Yejin Choi. Learning to write with cooperative discriminators. arXiv preprint 1rXiv:1805.00088,222 2018,
Hugging Face. Perplexity of fixed-length models. httpasforrnersseeeeerrleseseee
atpss//haggingface.co/ooo
2025. Accessed: 2025.
Fred Jelinek, Robert L Mercer, Lalit R Bahl, and James K Baker. Perplexity-4 measure of the difficulty of speech recognition tasks. The Journal of the Acoustical Society of America, 62(S1): S63-S63, 1977.
Ryo Kamoi, Yusen Zhang, Nan Zhang, Jiawei Han, and Rui Zhang. When can Ilms actually correct their own mistakes? a critical survey of self-correction of Ilms. Transactions of the Association for Computational Linguistics, 12:1417-1440, 2024.
Charles Koutcheme, Nicola Dainese, Sami Sarsa, Arto Hellas, Juho Leinonen, and Paul Denny. Open source language models can provide feedback: Evaluating Ilms' ability to help students using gpt-4-as-a-judge. In Proceedings of the 2024 on Innovation and Technology in Computer Science Education V. 1, pp. 52-58. Association for Computing Machinery, 2024.
Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, and Nazneen Fatema Rajani. Gedi: Generative discriminator guided sequence generation. arXiv preprint 20Xx:2000.06366, 2020.
Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. Contrastive decoding: Open-ended text generation as optimization. arXiv preprint arXiv:22 /500,, 2022.
Dancheng Liu, Amir Nassereldine, Ziming Yang, Chenhui Xu, Yuting Hu, Jiajie Li, Utkarsh Kumar, Changjae Lee, Ruiyang Qin, Yiyu Shi, et al. Large language models have intrinsic self-correction ability. arXiv preprint 1:XXv:2400615675 2024.
Zixuan Liu, Guangkai Jiang, and Siavash Khajavi. Llava-sco: Teach vision language models to self- correct. Proceedings of the IEEE/CVE Conference on Computer Vision and Puttern Recognition June 2025.
(CVPR) Workshops. pp. 3415-3424
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with felfbece Advances in Neural Information Processing Systems, 36:46534-46594 2023.
Sean O' Oriii and Mike Lewis. Contrastive decoding improves reasoning in large language models. arXiv preprint axXiv:2309.09117 2023.
Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang. Automatically correcting large language models: Surveying the landscape of diverse automated correction strategies. Transactions of the Association for Computational Linguistics, 12:484-506.
2024.
Melissa Z Pan, Mert Cemri, Lakshya A Agrawal, Shuyi Yang, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Kannan Ramchandran, Dan Klein, Joseph E. Gonzalez, Matei Zaharia, and lon Stoica. Why do multiagent systems fail? In ICLR 2025 Workshop Building Trust in Language Models and Applications, 2025. URL https: oppnreview net/forunt d
12

---

Published as a conference paper at ICLR 2026
Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are NLP models really able to solve sim- ple math word problems? In Kristina Toutanova, Anna Rumshisky. Luke Zettlemoyer, Dilek Hakkani-Tuu Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2080-2094, Online, June 2021. Association for Computational Linguistics. doi: 10.18653/v1/2022 waw.:o....aaa--........ -nacc--main 168. URL https://aclanthon
org.oog/////// 2021. senacl-mainn 168/
Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beatriz Borges, Antoine Bosselut, Robert West, and Boi Faltings. Refiner: Reasoning feedback on intermediate representations. arXiv preprint arXiv:2304. 01 0/904, 2023.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-prodt q&a benchmark. In First Conference on Language Modeling, 2024.
Florian Schmidt. Generalization in generation: closer look at exposure bias. arXiv preprint 1xXv::9900.0029: 2019.
Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang, Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face. Advances in Neural Information Processing Systems, 36:38154- 04:38154-3818 2023.
Zhuocheng Shen. Llm with tools: A survey. arXiv preprint arXiv:2409.18800 2024
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 31:863648863333 8652, 2023.
Jiankai Sun, Chuanyang Zheng, Enze Xie, Zhengying Liu, Ruihang Chu, Jianing Qiu, Jiaqi Xu, Mingyu Ding, Hongyang Li, Mengzhe Geng, et al. A survey of reasoning with foundation models: Concepts, methodologies, and outlook. ACM Computing Surveys, 57(11):1-43 2025.
Gladys Tyen, Hassan Mansoor, Victor Cärbune, Peter Chen, and Tony Mak. Lims cannot find reasoning errors, but can correct them given the error location. arXiv preprint arXiv:2311.085/6. 2023.
Peiyi Wang, Lei Li, Zhihong Shao, RX Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wa, and Zhifang Sui. Math-shepherd: Verify and reinforce llms step-by-step without human annotations. arXiv preprint arXiv:23/2.08933 2023.
Renxi Wang, Haonan Li, Xudong Han, Yixuan Zhang, and Timothy Baldwin. Learning from failure: Integrating negative examples when fine-tuning large language models as agents. arXiv preprint arXiv:2402. 11651, 2024a.
Shenzhi Wang, Le Yu, Chang Gao, Chujic Zheng, Shixuan Liu, Rui Lu, Kai Dang, Xiong- Hui Chen, Jianxin Yang, Zhenru Zhang, Yuqiong Liu, An Yang, Andrew Zhao, Yang Yue, Shiji Song, Bowen Yu, Gao Huang, and Junyang Lin. Beyond the 80/20 rule: High-entropy minority tokens drive effective reinforcement learning for LLM reasoning. In The Thirty-ninth Anmual Conference on Neural Information Processing Systems, 2025a. URL https: http////pperviiieee/ / wwtprrewwwwwwweeeereeeeeeee
id=yfcpdy4
Yifei Wang, Yuyang Wu, Zeming Wei, Stefanie Jegelka, and Yisen Wang. A theoretical understanding of self-correction through in-context alignment. Advances Neural Information Processing Systems, 37:89869-889 89912, 2024b.
Yizhou Wang, Lingzhi Zhang, Yue Bai, Mang Tik Chiu, Zhengmian Hu, Mingyuan Zhang, Qihua Dong, Yu Yin, Sohrab Amirghodsi, and Yun Fu. Cautious next token prediction. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Findings of the Association for Computational Linguistics: ACL 2025, pp. 25685-25697, Vienna, Austria, July 2025b. Association for Computational Linguistics. ISBN 979-8-89176 256-5. doi: 10.18653/ v1/2025. 10002...inings-aaa acl. 1318. houLhtts:// http://aaaaannntt.aooooooooo hww.::hhhwlo.....o.//.o.o 2025. ftre2....iiii findings- acl
131 87
13

---

Published as a conference paper at ICLR 2026
Colin White, Samuel Dooley, Manley Roberts, Arka Pal, Ben Feuer, Siddhartha Jain, Ravid Shwartz- Ziv, Neel Jain, Khalid Saifullah, Sreemanti Dey, et al. Livebench: A challenging, contamination- limited IIm benchmark. arXiv preprint arXiv:2406.199
2024.
Wenda Xu, Guanglei Zhu, Xuandong Zhao, Liangging Pan, Lei Li, and William Wang. Pride and prejudice: Llm amplifies self-bias in self-refinement. In ACL (7), pp. 15474-15492, 2024. URL Pittps:/ddon org. 10. 1ttp:///..1113//2/22222222
aww.::2....loooooo.. 826
Yuchen Yan, Jin Jiang, Yang Liu, Yixin Cao, Xin Xu, Mengdi Zhang, Xunliang Cai, and Jian Shao. S3c-math: spontaneous step-level self-correction makes large language models better mathematical reasoners. Proceedings of the Thirly-Ninth AAAI Conference on Artificial Intelligence and Thirty- Seventh Conference Innovative Applications of Artificial Intelligence and Fifteenth Symposium on Educational Advances in Artificial Intelligence, ALAA225IAAAT2//AAAA22 AAAI Press,
2025. ISBN 978-1-97775---8877 897-88 doi: 111600/aai..339/24...... 10.1609/aaa 1t116609/aaa..331122...33.7..
URL http::://o// org
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505,00 009388 2025a.
Zhen Yang, Mingyang Zhang, Feng Chen, Ganggui Ding, Liang Hou, Xin Tao, Pengfei Wan, and Ying-Cong Chen. Less is more: Improving Ilm reasoning with minimal test-time intervention, 2025b. URL http:: htttps://axxiv.rrgg org/
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
Ruochen Zhao, Xingxuan Li, Shafiq Joty, Chengwei Qin, and Lidong Bing. Verify-and-editt knowledge-enhanced chain-of-thought framework. arXiv preprint arXiv:2305.03266 2023.
Tianyu Zheng, Ge Zhang, Tianhao Shen, Xueling Liu, Bill Yuchen Lin, Jie Fu, Wenhu Chen, and Xiang Yue. Opencodeinterppeeer:: Integrating code generation with execution and refinement. arXiv preprint arXiv:2402 14658, 2024.
14

---

Published as a conference paper at ICLR 2026
A
APPENDIX
A.1 ONCE-MORE ALGORITHM
Algorithm 1 Once-More Framework
Require: Producer LLM, Tokenizer, Verifier set R, Goal x, role prompt, tokens max
1: Init:
Build chat prompt from (role prompt, .)); Encode to token id: step_ids.
Set past 0, Output 0, tokens t consecutive fails < 0 Initialize ckpt_stack + [00, step_ids)), supp-stack +- [None]. suppression t- None.
2: for 1 to T max do
3: Propose: Generate sentence and update (past, step ids, suppression).
4: Monitor:
Compute reliability - (ok, A). Update supp_stack with (A, t1. f1.z). Update tokens <- tokens + L.
5:
if not ok and i then
6:
Verify: Verifier judge (accept, F) Judge(R, x. Oulpul, 8). not accept and consecutive fails < 4 then
7:
8:
Rollback & Guide:
Restore last checkpoint (k*, ids");
set suppression from supp stack; trim past to k*;
reset step_ids <- ids"; pop stacks; append feedback to step_ids;
consecutive fails += 1 continue.
9:
end if
10:
end if
11:
Commit:
Append to Output;
push new checkpoints ckptstack and supp_stase stack; reset suppression +- None
reset consecutive
12: if last token of step ids is eos or tokens >> T Tmax then break 14: end if
15: end for
16: return Output
15

---

Published as a conference paper at ICLR 2026
A.2 QUANTILE CALIBRATION FOR THE RELIABILITY THRESHOLD
We set the verification trigger by calibrating a unil perplexity threshold from a short, held-out run.
Inputs and unit count. For each calibration prompt = 1,.... D, run the Producer once without verification and compute unit-level perplexity Ponit for every emitted unit (as in Section (.2). If the i-th output contains U, units, it contributes U, scalar values. The total number of calibration units is
M
Σ U
Collect all unit perplexities (P(m) )
m=1-
Empirical quantile. Form the empirical distribution by sorting the values themselves in ascending order (this sorting step ignores the original time order), yielding the order statisties
P() S P22) S S P(M)
Fix a target verification rate 7f E (0, 1) so that roughly the largest r fraction of units will be verified run time. Set q 7) and compute
h (M 1)g.
k= [h],
7 = h k
then define the calibrated threshold
P())
h S 1,
Psh =
h 2 M, (1-7) P(R) 7 P(k+1). otherwise.
P(M)s
At inference time, verify a provisional unit whenever (Y) > Pth-
A.3
MATHEMATICAL PROPERTIES OF THE FRAMEWORK
We now establish formal properties of the Once-More framework's 's perplexity computation and probability adjustment mechanism.
A.3.1 PROPERTIES OF PERPLEXITY-BASED MONITORING
Property (Perplexity Lower Bound). For any position j and K 2 1, the token-level perplexity satisfies PPT, (K) 2 1.
Proof. By definition, PPL (K) = exp(ANLL where
K
ANLI
(K)
- log T3 (1153)
(10)
K
Since (3(0).)) (0, 1] for valid probabilities, have log 4j (Ug,s) The lower bound of 1 is achieved when = 1 and z places all mass on a single token (i.e., (5(U5,)) = 1), giving (1) 0 and (x)
Property 2 (Sensitivity Control via K). Larger values of K increase the sensitivity of perplexity to distribution spread, while smaller values focus on top candidates.
Justification. When K is small, (PL(K) (K) reflects only the most probable tokens. If the top token has high probability, perplexity remains low even tail probabilities are spread. As K increases, more of the distribution is captured, increasing sensitivity to uncertainty in lower-ranked alternatives. This creates a trade-off: larger K detects subtle uncertainties but increases variance; smaller K provides conservative, stable estimates.
16

---

Published as a conference paper at ICLR 2026
A.3.2 PROPERTIES OF GUIDED REGENERATION
Property 3 (Mass Balance). Given suppression factors satisfying S agTy (trrgett and redistribution coefficient as defined in Section 3.3, the probability mass removed from the target token exactly equals the mass added to alternatives before normalization.
Proof. See Appendix A.4 for the complete derivation of Ky. which explicitly enforces this constraint.
Property 4 (Valid Probability Distribution). The adjusted distribution (z(+) is a valid probability distribution nnon-negative, sums to I).
Proof. By construction:
1. Non-negativity: For the target token, 8y starget),)) ) 1 - agry (target,) > 0 by the constraint 0yT) (target)) For alternatives, 8,(2) + (o) 0 since Rig > (as long as
alternatives exist) and ry(v) O. Since oj (2) (v) 2 for all v, we have qj (v)s((v) 2
Normalization:
The denominator Enev oj (u)s}(u) by non-negativity and the fact that (2) is a valid distribution. The normalization explicitly ensures Evey (v) = oj
Property 5 (Monotone Suppression). For fixed a, and importance ratios ry, the adjusted proba- bility qj (target,) strictly decreases with aj within the admissible range.
Proof. From the definition:
(targer,) - ayry(target,))) Σ. qj 2)
q (target,)
(11)
(u)s(
Taking the derivative with respect to oj (applying the quotient rule and using the fact that Ky depends on az), one can show that:
(target)),)
<0
(12)
daz
The key insight is that increasing 0j directly reduces the target token's weight while increasing alternatives" weights, creating double effect.
Property (Controlled Exploration via 3). The tempering exponent 8 controls the distribution of redistributed mass:
3 0: Mass spreads uniformly across all alternatives
3 = 1: Mass allocates proportionally to importance ratios (w)
3 > 1: Mass concentrates on tokens with the largest importance ratios
Justification. The boost factor for alternatives is 1 + (V)) When 3 = this becomes + Ki (constant across all alternatives), distributing mass uniformly. As increases, (v) amplifies differences: tokens with (w) > 1 (model now favours more) receive exponentially more boost, while tokens with (v) receive less. This creates exploration-exxploitaion trade-off controlled by B.
? remains close to () and s the KL Property 7 (Stability of Adjustment). When q; divergence between the adjusted and original distributions scales linearly with maximum suppression:
KL g5 (2)) O (amaa)
(13)
Proof sketch. The KL divergence is:
KL KL (ā, (19 ) Σ (v) log (v)
(14)
(2)
vv)
17

---

Published as a conference paper at ICLR 2026
When oj is small, ()) for all v, making (5()) ~ 9j (v)/ where Z R A Taylor expansion around a shows that the leading term is linear in a, The condition 3 S 1 ensures that the redistribution doesn't t create sharp peaks that would increase divergence. For detailed calculation, note that:
(e(c) log sy(() log = =s((1) (2 1)
log
(15)
4j (w)
where both (y(r) 1 and Z - are O(az).
Property 8 (Alignment and Diffusion Regularization). The combination of distance-decayel alignment and Gaussian diffusion prevents both over-localization and excessive spread of suppression signals.
Justification. The distance decay exp(-(li j)/r)²) reduces suppression for tokens matched far from their original position, preventing spurious long-range alignments from causing inappropriate suppression. The Gaussian diffusion with bandwidth a then smooths the resulting signal:
1 Zj
(j
i)2 20
új
exp
(16)
As a + O, this reduces to position-wise suppression ūj - 2j (localized). As a increases, suppression spreads to neighboring positions (regularized). The final blend nž 0.5Ø,* + 0.5ū, balances both effects, preventing over-fitting to specific positions while maintaining locality.
A.4 DERIVATION OF THE REDISTRIBUTION CONSTANT
We now derive the formula for ny that ensures probability mass conservation during the adjustment process.
Setup. At position adjust the original distribution qj
(2) (t) by applying scaling factors sy (v):
8,(0) {1 (1)5)((00) + target,
1 - a)) target,
(17)
where:
aj (0,1) is the suppression strength
(v) 9 (2) (v) max(e, of"h (w)) is the importance ratio
B 0 controls redistribution sharpness
Kj is the redistribution coefficient be determined
Mass Conservation Constraint. For valid probability distribution, the total probability mass before and after adjustment must be equal. Define:
Mass removed from target:
A 9 (target,) ) 93 (target,) 88 (target,))
(18)
(target,) ) 99 (target,) ) a.yyy (target)) (19)
ayr, (target,) 9 (target,)
(2)
(20)
18

---

Published as a conference paper at ICLR 2026
Mass added to alternatives:
-=
9j (u) sy) -..
(2)
(21)
Σ
4j
(u) (1 -
(22)
uttugg
E
(u)
(u))(u)
(23)
wyltarget
Kij
Σ
uu) (u)
(24)
uystarget
Solving for Kj. Setting A = A+ for mass conservation:
ajry(target, oj (target,) Ki uystarget,
Σ
q (u)))
(25)
Solving for Ky:
Kg agr, (target))) () (target,) Euztrgett 9j (w) ry (u))
(26)
Normalization.
After applying the scaling factors, we normalize to obtain the final distribution:
9i (2) (z())
(v) Evev (2) (w) su(
(27)
By construction (mass conservation), the numerator of the normalization constant equals:
Σ u
9j 0 (u) sy(u) = of (target,) sy (target,) + Σ
(2)
qj (u) sy(u)
(28)
1 u.sttagget
= 0 (2) (target,) - ---- --1+
Σ
9 (v) + A+
(29)
uystarget
Σ (2) (w) (A+ (A+-A-)
(30)
=1+0=1
(31)
Thus, the distribution is properly normalized even before the explicit normalization step, confirming that our choice of Ky correctly preserves mass.
Numerical Safeguards. In practice, we implement several safeguards:
Use = 10-8 in importance ratios to avoid division by zero
Clip a,r, (target)) to ensure it remains strictly less than
If the denominator in Kj is extremely small (no viable alternatives), either reduce 3 toward 0 to spread mass broadly or cap Kj and trigger fresh generation attempt
Verify that Σn 93 ( (u) > 0 before normalization
A.5 ABLATION STUDY
A.5.1 ANALYSIS OF SUPPRESSION STRENGTH
The ablation study in (suppression strength) is carried out using the Qwen- 14B model AIME24, Livebench, and GGQA-Diamont Results are reported in Table 0 from a single run. All other parameters chosen the default value.
19

---

Published as a conference paper at ICLR 2026
Table 6: Ablation study suppression strength a using Qwen3 14B,
Results from a single run.
0.1
0.5
1.5
2
AIME24
30.0
33.3
36.6
36.6
36.6
LiveBench
49.5
52.5
54.5
54.5
54.0
GPQA-D
50.5
51.5
52.0
52.5
51.0
For all benchmarks, performance is relatively stable for suppression strengths 21. While small
suppression strength 0.1) dilutes the effect of our proposed Once-More Framework and achieves the worst performance in the ablation study.
A.5.2 ANALYSIS OF DISTANCE DECAY FACTOR
The ablation study on (distance decay factor) is conducted by the Qwen- 14B model on AIME24,
Livebench and GPQA-Diamond. The granularity of T set to 0.01 (extremely small T representing decay to 0, suppression only works the target token in this specific position), 0.5, 1, 1.5 and 100 (extremely large r representing no decay, the target token will be equally suppressed regardless of its position). Results are reported in Table 7 from single run. All other parameters are chosen as the default value.
Table 7: Ablation on distance decay factor Accuracy from single run.
Benchmark
0.01
0.5
1.5
== 100
AIME24
23.3
36.6
36.6
36.6
30.3
LiveBench
43.5
52.5
52.5
52.0
50.5
GPQA-D
45.5
54.5
55.0
55.0
52.5
The ablation results show very interesting phenomenon. Without distance decay (an extremely small r = 0.01), model performance decreases significantly, becoming even worse than the raw model baseline. Under this setting, the Once-More framework only suppresses the target token if it the exact same position as in the previously rejected sequence. The model sometimes finds trick to regenerate. Assuming this is the previously rejected unit:
This sequence contains an error and was therefore rejected
The producer first generates a few other words and then repeats exactly the same sequence as the previously rejected one, just like:
Got it! This sequence contains an error and was therefore rejected
Since the position is mismatched by these beginning words, the later regenerated sequence is not suppressed all. However, this will be subsequently rejected by the reviewer, causing the model to get stuck in a loop until the maximum token limit is reached.
Table Ablation on diffusion bandwidth a. Accuracy from a single run.
Benchmark
σ 0.1
10
AIME24
33.3
36.3
26.6
30.0
LiveBench
52.5
55.0
48.5
47.5
GPQA-D
50.5
52.5
49.0
49.5
20

---

Published as a conference paper at ICLR 2026
A.5.3 ANALYSIS OF DIFFUSION BANDWIDTH
reporss the effect of diffusion bandwidth o, which controls how suppression spreads across neighbouring tokens. The results show that a large o(2 5) spreads the suppression too broadly. dilutes the suppression effect on the target token and behaves similarly using very small suppression strength (a)) Conversely, very small restricted suppression only to the current token, making the adjustment too localized and also degrading accuracy.
A.5.4 ANALYSIS OF PERPLEXITY THRESHOLD 1)
Since the parameter (top-K for perplexity estimation) directly defines unit perplexity and conse- quently impacts the threshold ratio 7) we conducted joint ablation study of these two parameters. Table and Table show that, for small (1 or 5), changing 7) primarily impacts runtime rather than accuracy. A looser threshold (17 50%) increases verification calls and computation time, while a stricter threshold (1) 12.5%) improves efficiency. Large K values generally degrade accuracy.
Table 9: AIME24 results: accuracy and wall-clock time across perplexity threshold and top-K settings.
12.5%
25%
7)
50%
Acc.
Time (s)
Acc.
Time (s)
Acc.
Time (s)
K ==1
36.6
1693
36.6
3579
36.6
6448
K = 5
36.6
2448
33.3
4133
36.6
6103
K 10
33.3
2397
33.3
3142
36.6
5954
K 15
30.0
2929
33.3
2851
33.3
7088
Table 10: GPQA results: accuracy and wall-clock time across perplexity threshold and top-K
settings.
= 12.5%
25%
7) = 50%
Acc.
Time (s)
Acc.
Time (s)
Acc.
Time (s)
K =
51.0
1987
54.5
3640
61.6
4953
K 47.5
2448
55.5
3848
55.0
4424
K 10
49.0
2397
49.5
2098
53.5
4244
K 15 48.4
2473
49.5
2486
55.0
3412
A.6 ADDITIONAL OBSERVATIONS
A.6.1 INTERMEDIATE REASONING BEHAVIOUR
Here is one example that the proposed Once-More framework exhibits intermediate reasoning behaviour. The following text is the partial output from AIME24 problem 83:
We are given a 2x3 grid where each cell contains a digit from to 9. We need to fill the grid such that:
1. The
of the two numbers formed by reading left to zight** (i.e., the top row and bottom row) is **999*.
[Perplexity acceptable: False]
[Judge] Output accepted.
2. The **sum of the three numbers formed by reading top to bottom* (i.e., the first column, second column, and third column) is **99**.
[Perplexity acceptable; True]
21

---

Published as a conference paper at ICLR 2026
Let's denote the grid follows:
(aggin/array
Ahline
& & II
Ahline
d & \I
Ahline
(ard(y)ray)
$$
## step 1: Understand the horizontal sums are told that:
$$
abc def = 999
Here, $abes and $def$ are 3-digit numbers formed by the digits in the top row and bettom row, respectively.
[Perplexity acceptable; False]
[Judge] Output accepted.
past_cache length: 400 That is:
$abe 100a + 10b c$
$def 100d 10e $$
So:
$$
(100a 10b
(100d 10e 999 Atag(1}
$$
### Step 2: Understand the vertical sums
We told that:
$6
+ d + e 99 Aquad \text (first column) } V
b + + f 99 \quad \text (second column) VA
c c \quad \text I (third column) }
$$
[Perplexity acceptable; False] [Judge] Output rejected.
[Rolling back to the corresponding checkpoint] [Feedback message the model]: (The vertical sums must involve multi-digit numbers, not single digits. Re-express the vertical sums as $ad + de ef 995, etc., clarify the prcblem. statement.)
Clarifying: the problem. says "the sum of the three numbers formed by reading top to botter. is 99". That means: The first column forms a number: sa \tines 100 + d Atimes 10
...f, but that would be too large.
[Perplexity acceptable: False] [Judge] Output rejected. [Rolling back to the corresponding checkpoint] [Foedback message to the model]: (Clarify how the top-to-bottom numbers are formed (e.g., single-digit numbers multi-digit). The current interpretation is unclear.)
Let's assume that the top-to-bottom numbers are forned by reading
each of the three columns as a single-digit value (i.e., just the
digit in each column). Then the sum of these three digits is 99,
which is impossible since the maximum of three digits is
694949-27s. so this interpretation is invalid.
22

---

Published as a conference paper at ICLR 2026
[Perplexity acceptable: False] [Judge] Output accepted.
Let's try a different interpretation: The *ttop-to-bottom+ numbers
are formed by concatenating the two digits in each column to form two-digit number.
[Perplexity acceptable: False]
[Judge] Output accepted.
For example, if the first column has digits $a$ and sbs, then the number is $10a b$. Then we can add up the three such numbers for the three columns, that total should be 99.
So have:
The top row is a 3-digit number: $100a + 10b c$
The bottom row is a 3-digit number: $100d + 10c fs
The sum of these two numbers is **999**:
$ (100a
10b c) (100d 10e f) 999$
(Equation 1)
The sum of the three column-wise two-digit numbers is **99**:
$ (10a
(10b e) + (10c f) 99$ (Equation 2)
We are to find the number of such digit arrangements that satisty both equations.
This partial output shows how Once-More conducted one successful self-correction.
23

---

Published as a conference paper at ICLR 2026
A.7 PRACTICAL IMPLEMENTATION NOTES
Length changes and alignment. If the revised unit Y' = ..,,..., V/n] has m o m
If m < n. positions j > m do not apply; early stopping may be the correction.
If n, positions j > n are generated without suppression (they did occur previously). For j S min(m, n) we apply the update above. Optionally, one align by minimal edit distance when the new attempt diverges early; our experiments did not require this.
Chunking strategy and context management
Choosing the unit. For prose, default to sentence boundaries with a token cap, splitting very long sentences into clauses; for code, we use syntax-aware spans (function, block, or diff hunk). Smaller units give finer correction. Larger units preserve coherence.
Boundary overlap. We allow a short overlap window when proposing the next unit and let the Verifier inspect a sliding window that straddles the boundary to avoid seam artifacts.
Context growth. For long outputs, optionally with compressed summaries of older content, to stay within context limits.
keep rolling buffer of recent accepted text,
Multiple verifiers. Factuality, format, and safety checks can run in parallel with conjunctive acceptance, or stages (cheap checks first). Tool use follows standard agent patterns.
24

---

Published as a conference paper at ICLR 2026
A.8 EXAMPLE PROMPTS
Below are the prompts we used in Once-More for the Producer, Verifier A (Formal & Local), Verifier B (Global/Sanity), and optional Adjudicator.
Shared verifier SYSTEM (use for both verifiers).
You verifier. You judge exactly CURRENT_SFAN in partial solution.
Do not solve the whole task. Be precise and conservative.
Inputs:
TASK: the problem/question
ACCEPTED : the slready-accepted prior steps/state
SURRENT_SPARE the producer's proposed step (s) to add now
GPQA-STYLE GRADUATE Q&A (SCIENCE)
Producer (SYSTEM+USER)
[SYSTEM]
You are graduate-level problem solver. Solve the problem step by step, separated by period. Your answer should be chosen from options A, B, D and end with: Final answer: <A/B/C/D>
[USER]
QUESTION:
[GGPAA BESICAA_URTTTTOE QURSTTON_STRRM WWL___TT_AND_OPTTOOO
Verifier (Formal & Local) (USER only; use shared SYSTEM above)
[USER]
TASK: HGPQA QUESTION STEM AND OPTIONS ACCEPTED CONTEXT: (IPRIOR NOTES OR NONE
CURRENT_SPAN: SPAN: ((PRODUCER PARAGRAPH)!
You are very strict verifier with Rubric: R1 algebraic legality (no invalid cancellations, correct zadical/log rules),
arithmetic accuracy,
R3 aammaln/oranchhcontttaaiiii
respected and stated, R4 check carefully for the hold of ionatiinnsinequalllttt check the scientific correctness of each claim. Flag missing premises, leaps, or contradictions. R7 Countorfactual statement Checking.
Does the input unit the right track to achieve the goal, given the context? Verify the input unit with your role and task. Only reject the answer according your role.
The input may not necessarily solve the goal directly as there are more details in the upcoming text. If yes, answer Yes. If no, answer and provide repair hints beginning with "Suggestion" to improve in no more than 20 words.
Verifier B (Global/Sanity) (USER only; use shared SYSTEM above; optional)
[USER] TASK: {[GPOA_QURRS QURST ON_STEM, WINLLLPTTTON [.NLSINNNANNOOOIIOOOSS
QURSTT
25

---

Published as a conference paper at ICLR 2026
ACCEPTED CONTEXT: (IPRIOR NOTES OR NONE)
CURRENT :PRRENT_SPAN: ((PRODUCER PARAGRAPH
Perform:
quick alternative micro-derivation or spot-check, S2 sanity bounds / physical plausibility,
93 attempt 1-2 counterexamples,
flag hidden assumptions.
Allowed crror_tags: ["counterexampee", "bounds", "assumption", "consistency"
AIME-STYLE OLYMPIAD MATHEMATICS
Producer (SYSTEM+USER)
[SYSTEM]
You are AIME preblem. solver. Produce one clean step-xise derivation end with:
Final answer: \boxed <integer 0-999>}
[USER]
PROBLEM:
{ BHAMEEPROBLEMMTERE
Verifier A (Formal & Local) (USER only; use shared SYSTEM above)
[USER] TASK:
I HAIME_PRRBLEM
PROBLEM TEXT)
ACCEPTED SICEPTED_CONTEXTT
CONTEXT: [[PRIOR_AAE [IPRIOR_ACCEPEED SERRPPEED_TTESS CR____R___OOO NONEE
CURRENT_SPAR SPAN: ((PRONUCK_SSTIIS))
Rubric:
R1 algebraic Legality (no invalid cancellations, correct radical/log rules),
R2 arithmetic accuracy,
R3 comuin//ranchhconstsiin respected and stated,
local entailment from prior state, notation/variable hygiene,
R6 if final answer present: integer 0-999 and derived quantity matches.
Does the input unit in the right track to achieve the goal given the context? Verify the input unit with your role and task. Only reject the answer according to your role.
The input may not necessarily solve the geal directly as there are details in the upcoming text. If yes,
Yes. If no, No and provide repair hints beginning with 'Suggestion" to improve in more than 20 words.
Verifier B (Global/Sanity) (USER only; use shared SYSTEM above; optional)
[USER]
TASK: BOIIM__RRRBLE_TTT ACCEPTED_ : BABALAAACCCCTTTTTPSSLCLLCCC CURRENT_ SPAN: ((PRODUCER_STEFS)
Perform:
s1 quick recompute of the same claim vla different micro-tactio, sanity bounds/ned checks (c.g., parity, sign, rough nagnitude),
26

---

Published as a conference paper at ICLR 2026
try 1-2 simple counterexamples consistent with constraints,
flag hidden assumptions (e.g., x != 0).
Allowed error_tags:
["counterexample", "bounds", "assumption", "consistenay").
LIVENBENCII (REASONING)
Producer (SYSTEM+USER)
[SYSTEM]
You are an problem solver. Solve the problem step by step seperated by period. This problem. is quaranteed to have solution. Should not exit without finding the exact solution.
The final answer is wrapped as: us:iincolation>.... on
[USER]
PROBLEM:
{
Verifier A (Formal & Local) (USER only; use shared SYSTEM above)
[USER] TASK:
I [LIVEEENCH__PHH BEEEEEEENC_PPRRLEMM..C
ACCEPTED :ONTEET:: CWW.TTRRAACEPTEDDLLC__ STEES_TT__OOOSS NONE))
CURRENT_SPANT ((PRODUCER_STEPS)
Rubric (apply all):
factual correctness claims standard
definitionssLiteratuee,
R2 local logical entailment (no leaps), R3 unit/notation hygiene, current answer matches reasoning, R5 the current answer is not necessarily complete.
Does the input unit in the right track to achieve the goal given the context? Veri fy the input unit with your role and task. Only reject the answer according to your role.
The input may not necessarily solve the goal directly y as there are more details in the upcoming text.
yes, answer Yes. If no, answer and provide repair hints beginning with 'Suggestion' that can be concatenated to the context improve future answer.
The suggestion should no more than words.
SVAMP
Producer (SYSTEM+USER)
[SYSTEM]
You are an problem solver. Solve the problem step by step seperated by period. This problem. is guaranteed to have solution. Should not exit without finding the exact solution.
The final answer shculd be in the format: Final answer:
\boxed <iotee<integer 0-999>1
[USER]
PROBLEM:
(SVAMP PROBLEM TEXT
27

---

Published as a conference paper at ICLR 2026
Verifier A (Formal & Local) (USER only; shared SYSTEM above)
[USER]
TASK:
ACCEPTED < [IPRIOR_ACCKPE ACCKPINI_STTAPSOOK _T_PPP__R___ONO) )
CURRENT SPAN: ((PRODUCER STEPS))
RI algeoraic Legality (no invalid cancellations, correct zadical/log rules),
R2 arithmetic accuracy,
R3 check carefully for current formulating the real life problem
into math symbols, very sensitive to negative nunbers
R4 check carefully for formulating the real life problem into math symbols,
check carefully for any unresonable nurber during
calculation, which will not happen in real life situation
R6 Check carefully about whether the current answer is overthinking
or too complex be true in real life situation.
RY Check carefully about whether the current context model the goal
correctly
Verify the input unit with your role and task. Only reject the other reasons.
answer according to rubric. not reject
The input may not necessarily solve the goal directly as there are
more details in the upcoming text. If yes, answer Yes. If no, answer No and provide repair hints beginning with 'Suggestion' to improve in no more than 20 words.
GSM8K
Producer (SYSTEM+USER)
[SYSTEM]
You are an problem selver. Solve the problem step by step seperated by period. This problem is guaranteed to have solution. Should not exit without finding the exact solution.
Final answer: Vinxed(<integer 0-999>}
[USER]
PROBLEM:
[GSMMB PROBLEM TEXT
Verifier A (Formal & Local) (USER only; use shared SYSTEM above)
[USER]
TASK: [1GSMBK_PROBLEX,
PROBLEK TEXT)
ACCEPTED CONTEXT:
ACCEPTED_CONTEXT: ((PRIOR AACEGAATTTTTTTTTPSS..OO.
CURRENT_SPAN: ([PRODUCER_STEPS)
algebraic legality (no invalid cancellations, correct radical/log rules),
R2 arithmetic accuracy,
check carefully for current formulating the real life problem
into math symbols, be very sensitive to negative numbers R4 check carefully for formulating the real life problem into math symbols,
R5 check carcfully for any unresonable number during the calculation, which will not happen in real life situation
Check carefully about whether the current answer is overthinking or too complex to be true in real life situation.
28

---

Published as a conference paper at ICLR 2026
Check carefully about whether the current context model the goal correstly
Verify the input unit with your role and task. Only reject the
answer according te rubric. Do not reject for other reasons.
The input may not necessarily solve the goal directly there are
more details in the upcoming text. yes, answer Yes. If no, answer No and provide repair hints beginning with 'Suggestion' to improve than 20 words.
in
A.9 USE OF LARGE LANGUAGE MODEL (LLM)
We used an LLM strictly for editorial assistance at the final drafting stage. Specifically, the LLM was used only to check for grammatical errors, fix typographical mistakes, and enhance sentence transitions for better readability. All technical content, including ideas, algorithms, proofs/derivations, experiments, analyses, tables/figures, and conclusions, was authored by the listed authors. The LLM did not generate, rewrite, materially alter any scientific claims results.
All -uugesggeetedd edits were manually reviewed and accepted or rejected by the authors. We retained authorship control at all times and ensured that technical meaning was changed. The LLM is not an author and bears no responsibility for the paper's content. The authors assume full accountability for all claims and results.
29


Reviews:
[{"review_id": "LlglXttKOK", "reviewer": "Reviewer_Gzea", "summary": "This work proposes a model-agnostic framework for continuous self-correction for LLMs at inference time. Unlike contemporary post-hoc iterative refinement methods that wait for complete outputs, authors propose to intervene at a more granular level of a \"unit\" (sentences, paragraphs or code blocks) using two key mechanisms: (1) perplexity monitoring to detect potential errors (2) fusing logit redistribution with verifier feedback (LLM-based or symbolic) to guide regeneration away from failed attempts.\n\nThe work motivates the necessity to intervene early to prevent the propagation of errors in early steps to the final outputs. The continuous intervention approach is conceptually interesting, and to the best of my knowledge, the combination of perplexity signals with external feedback mechanisms is novel. The proposed method achieves strong empirical results on mathematical reasoning (AIME), the reasoning subset of LiveBench, and GPQA compared to Self-Refine and Critic.\n\nHowever, the work suffers from several practical limitations, including complex hyperparameter setup, several missing details, and evaluation being limited to reasoning tasks.", "questions": "- Verifier Quality Analysis: What happens when a stronger verifier is used in an asymmetric setting? I understand this method proposes to improve the self-correction capabilities; however, seeing that the method generalizes well with scale, it may be worth showing sample results in an asymmetric setting with a stronger verifier.\n- Hyperparameter Sensitivity: Please provide sensitivity analysis for all hyperparameters. Are the results robust to these choices, or does performance degrade significantly with the choice of other hyperparameter values? At the very least, please provide values for all hyperparameters that were used for experiments.\n- Calibration Data Requirements: How many held-out examples are needed for reliable $P_{th}$ calibration? How does performance degrade if calibration and test distributions differ?\n- Computational Overhead: What is the average wall-clock time increase compared to raw generation, and chosen baseline method?\n- Perplexity-Error Correlation: What is the actual correlation between $PPL_{unit} > P_{th}$  and the presence of errors in your datasets? What are the false positive/negative rates? How does wrong perplexity calibration affect the downstream performance? \n- Verifier Disagreement: How is verifier disagreement handled in the case of multiple verifiers? Does the framework support weighted voting or consensus mechanisms?\n- Complexity of Logit redistribution: Have the authors tried simpler logit penalties/redistribution strategies? What is the comparison with a simple token penalty based on perplexity? Are all the components, such as distance decay, diffusion to prevent overlocalization, important? An ablation of these components would be appreciated.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- Novel Combination of Signals: The proposed method uniquely integrated unit-level perplexity with external verifier feedback for continuous guidance, correcting the wrong steps in the early stages.\n- Model-Agnostic and Training Free: The framework shows strong performance despite being an inference-time technique. The setup is model agnostic and thus can be broadly applied.\n- Token Efficiency Gains: The proposed method shows gains in generated token efficiency while achieving better performance.\n- Scaling Behaviour: The Authors show that the framework benefits more from larger models, suggesting it effectively leverages richer model representations.", "weaknesses": "- The authors provide no ablation for the selection of the \"unit\". Unit granularity plays a pivotal role in the proposed method; however, it is not clear how the performance changes with the changing definition of the unit.\n- The core assumption of the paper is that each unit can be verified without looking at the future unit. A discussion on this is warranted.\n- The method introduces numerous hyperparameters (K for top-K, $\\alpha$ for suppression strength, $\\beta$ for redistribution sharpness, $\\sigma$ for diffusion bandwidth, $\\tau$ and $\\gamma$ for distance decay, and $c$ for truncation radius) with minimal justification. Section 4.1 only provides the value for K, $\\alpha$, $\\beta$ without sensitivity analysis. The authors provide no ablation experiments to study the effects of various hyperparameters or how they were selected.\n- Perplexity threshold is calibrated using empirical quantiles on a held-out set of prompts. However, no further details are provided, such as how many prompts were used for calibration or the value of verification rate $\\eta$. Similar to other hyperparameters, no sensitivity analysis is done for this value.\n- Authors provide a computational cost analysis in the form of generation token efficiency. However, they do not provide information on wall-clock time. Depending on the selection of units, the wall clock time may exceed that of contemporary methods. I believe an analysis of the number of verifier calls, wall-clock time will make the claims stronger.\n- It is not clear if the proposed method works for an open-ended generation task. Authors limit their experiments to reasoning tasks where units can be sufficiently defined. However, for open-ended tasks such as story generation, the definition of a unit and the core assumption (each unit can be verified without access to future steps) may break down.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zqFYDZFK0B", "reviewer": "Reviewer_o7eJ", "summary": "This paper introduces Once-More, a model-agnostic framework that enables continuous self-correction during large language model (LLM) generation by combining token-level perplexity monitoring with verifier feedback. Unlike post-hoc refinement methods that only intervene after a full output is produced, Once-More operates incrementally, detecting and correcting high-perplexity segments before errors propagate. Its core mechanism redistributes token logits to steer the model toward more reliable generation paths while preserving its internal beliefs. Experiments on reasoning and mathematical benchmarks show consistent performance gains over baselines such as Self-Refine, CRITIC, and S3c-MATH, with improvements up to 10 points and reduced token overhead. The framework demonstrates that perplexity-guided, fine-grained intervention can make LLM generation more accurate and stable without retraining or modifying the base model.", "questions": "How does Once-More decide when to trigger a correction during generation, and what threshold or criterion determines whether a high-perplexity segment actually warrants intervention?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- The paper introduces a simple yet effective framework for real-time self-correction that can be applied to any language model without retraining or architectural changes.\n\n- The method provides a clear mechanism for detecting and mitigating local generation errors, showing how token-level perplexity can guide correction in an interpretable way.\n\n- Experimental results demonstrate consistent performance gains across diverse reasoning and math benchmarks, confirming the approach's robustness and practicality.\n\n- The framework is computationally lightweight compared to multi-pass refinement methods, since it reuses the model's own activations and verifier feedback rather than generating full alternative outputs.\n\n- The paper contributes new insights into how continuous monitoring of model uncertainty can improve stability and factual accuracy in step-by-step generation.", "weaknesses": "- The paper lacks a clear theoretical justification for why token-level perplexity reliably correlates with factual or reasoning errors, relying mainly on intuition and anecdotal examples (Figure 2).\n\n- The approach introduces additional inference cost through iterative token re-evaluation and verifier calls, but the paper does not report latency, FLOPs, or wall-clock overhead to substantiate its claims of efficiency. The only quantitative evidence concerns token usage, which measures output length rather than actual computational or time efficiency.\n\n- The method's reliance on external verifier feedback limits reproducibility and fairness, since different verifier choices or access levels could yield inconsistent results across users and model settings.\n\n- The verifier component is insufficiently analyzed: the paper does not show how verifier quality, domain alignment, or calibration affect correction accuracy or error propagation.\n\n- The experiments focus narrowly on reasoning and math datasets, leaving it unclear whether the method generalizes to open-ended generation, dialogue, or long-context settings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "gdz9Xp7eUG", "reviewer": "Reviewer_5kHm", "summary": "This paper proposes Once-More, a perplexity-guided, continuous self-correction framework that intervenes during autoregressive generation by monitoring token/unit perplexity, invoking verifiers when uncertainty is high, and applying a perplexity-driven logit redistribution to guide regeneration. The method is model-agnostic and evaluated on several reasoning and math benchmarks (AIME24/25, LiveBench, GPQA, SVAMP, GSM8K) using Qwen3 models (4B/8B/14B) and comparisons with iterative refinement and supervised baselines. Results show consistent improvements and a token-efficiency advantage over some baselines. The idea is timely and practically relevant for improving reliability of LLM generation.", "questions": "1.Could you elaborate on how verifiers are chosen or configured for different tasks (e.g., math reasoning vs. factual QA)? Is there any criterion for when to use LLM-based versus rule-based verifiers?\n2.Since Once-More operates on adaptive “units” (clauses, sentences, paragraphs, or code blocks), how is this segmentation determined in practice? Is it based on punctuation heuristics or learned boundaries?\n3.Have you observed cases where Once-More fails to correct errors or even amplifies them? For example, when perplexity misjudges a correct but low-probability reasoning step. How often does this happen, and how do you mitigate it?\n4.During regeneration, how sensitive is the framework to the quality or consistency of verifier feedback? Would weaker verifiers (e.g., smaller LLMs) substantially degrade performance?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1.Continuous, unit-level perplexity monitoring combined with verifier feedback and logit redistribution is a sensible and practical inference-time approach to reduce error compounding without retraining.\n2.Experiments cover multiple benchmarks and model sizes, showing gains across tasks and reasonable comparisons to both post-hoc and supervised baselines.\n3.The paper includes component ablation and token usage comparison, which help motivate the claimed efficiency advantage over some iterative refinement methods.\n4.Appendices contain algorithm pseudocode and mathematical derivations; hyperparameters are stated in the main text.", "weaknesses": "1.Several grammatical mistakes and minor typos appear in the manuscript (examples: “these methods often fails” should be “these methods often fail”; “it still suffer from this issue” should be “it still suffers from this issue”; duplicated word ”under under”).\n2.Results are averaged over three runs but standard deviations or confidence intervals are not reported.\n3.Ablation covers main components (feedback and redistribution) but lacks sensitivity analysis over key hyperparameters.\n4.Token counts are reported, but there is no quantitative measurement of inference latency or computational overhead.\n5.The conclusion reiterates results but would benefit from an explicit discussion of limitations and planned directions for improvement.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XZdiVzts4r", "reviewer": "Reviewer_bA69", "summary": "The paper proposes a novel algorithm for self-correction at a more fine-grained level of thoughts, sentences, or reasoning steps. The self-correction process is driven by model uncertainty estimation to guide the model toward the correct reasoning path. The authors also develop a procedure for suppressing incorrect token generation during the refinement stage. They demonstrate improvements over several baselines.", "questions": "“Struggle with domain generalization, and still suffer from error accumulation (a single failed self-correction will just cascade through subsequent generations).” — there is no proof provided for this claim. It might be true, but as stated, it sounds like an overstatement.\n\ny_i is not defined l196", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "In general, I find this work valuable for several reasons:\n- the introduction of self-correction on a more fine-grained level is novel, up to date, and an important step;\n\n- the general framework is a great contribution, especially when driven by the uncertainty estimator, which is a powerful tool for analyzing correctness and is believed to be a bottleneck in self-correcting papers;\n\n- interesting decoding strategy with observable gains.", "weaknesses": "Despite the general novelties in the methodology of the paper, I find it very raw in terms of experiments and presentation. I find the ideas really worthy and encourage authors to continue in this direction, but for now the empirical proof of validity is very scarce and unconvincing. Let me elaborate.\n\n**Experimental Procedure**\n\n The main weakness I see lies in the very weak baselines. For example,\n- Self-Refine is known to struggle with small models, and it is one of the first techniques.\n\n- The implementation of CRITIC is absolutely awful: the RAG system is not defined properly, as there is no clear definition of the search query, search engine, or how the context is incorporated. Moreover, taking the top 1000 words from the search is not a proper RAG system; it is simply an incorrect implementation of the method.\n\n- There is also an absence of other valuable baselines. I can name several self-correction techniques that should have been considered: Self-Correction [3] and its generalization STaSC [1], SCoRE [4], and REFINER [2].\n\n- Another issue is the absence of adequate classical methods. It would be very reasonable to include methods with the same compute intensity. Once self-refinement generates multiple times, the authors should include a self-consistency approach leveraging the same amount of compute.\n\nRegarding the existing benchmarks, SC3-MATH results are not reported on AIME24, AIME25, LiveBench, or GPQA.\n\nDespite the benchmarks, there is evidence that the variation of such approaches is quite high, so reporting the standard deviation is a must here.\n\nI’m really concerned about the token usage section. How is token usage calculated? Conditioning a model on 10 tokens is one forward pass, while generating 10 tokens involves 10 forward passes — these are not equivalent, which raises a lot of questions. Moreover, i see no comparison with SC3.\n\nRegarding the experiments, they really lack more in-depth analysis. For example the ablation for PPL-K, why using PPL at all, when there are many other measures of uncertainty, such as entropy, sequence probability, etc.?\n\nMoreover, I doubt the generalization of the uncertainty estimator threshold, such as perplexity. The beginning of the paper states the aim for generalization, but a PPL-based threshold would hardly generalize to other architectures — in fact, it is known to be difficult to achieve [5].\n\n**Presentation**\n\nIn general, the paper is difficult to follow. The methods are presented unclearly, and the guided generation process, in particular, remains quite opaque. Many of the technical components could be introduced and explained in a clearer and more structured way.\n\nMoreover, the paper seems somewhat overloaded with various tricks and heuristics. It would be more effective if the authors focused on one specific aspect and demonstrated its effectiveness comprehensively, rather than introducing multiple loosely justified additions. For instance, the idea of fine-grained revision even without the decoding stage is already promising on its own, i really like it. It would be valuable to show that it works well and analyze it thoroughly. I hope that ICLR is still a place for scientific papers, not a pursuit of state-of-the-art leaderboard results. After reading the paper, I gained little new understanding beyond the notion that fine-grained corrections may help, which, however, still lacks solid empirical support.\n\nFor example, when the model rolls back to a previous step, is there any analysis of such behavior? How often does it occur? How accurate are these corrections? These are important questions that could open up valuable future research directions.\n\nFinally, several important details are missing, and the accompanying repository does not include the code. This significantly lowers the reproducibility of the work, which is unfortunate.\n\n\n\n[1]  Moskvoretskii, Viktor, Chris Biemann, and Irina Nikishina. \"Self-Taught Self-Correction for Small Language Models.\" arXiv preprint arXiv:2503.08681 (2025).\n\n[2] Paul, Debjit, et al. \"Refiner: Reasoning feedback on intermediate representations.\" arXiv preprint arXiv:2304.01904 (2023).\n\n[3] Welleck, Sean, et al. \"Generating sequences by learning to self-correct.\" arXiv preprint arXiv:2211.00053 (2022).\n\n[4] Kumar, Aviral, et al. \"Training language models to self-correct via reinforcement learning.\" arXiv preprint arXiv:2409.12917 (2024).\n\n[5] Moskvoretskii, Viktor, et al. \"Adaptive retrieval without self-knowledge? bringing uncertainty back home.\" arXiv preprint arXiv:2501.12835 (2025).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)