SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior Reasoning LLMs
Authors: Dachuan Shi
Abedelkadir Asi
Keying Li
Xiangchi Yuan
Leyan Pan
Wenke Lee
Wen Xiao
Venue: ICLR 2026 Poster (2026)
Keywords: LLM
Reasoning
Primary area: foundation or frontier models, including LLMs
TL;DR: SwiReasoning is a training-free method for Pareto-superior reasoning LLMs that dynamically switches between explicit and latent thinking, with a switch count control to suppress overthinking.

Abstract: Recent work shows that, beyond discrete reasoning through explicit chain-of-thought steps, which are limited by the boundaries of natural languages, large language models (LLMs) can also reason continuously in latent space, allowing richer information per step and thereby improving token efficiency. Despite this promise, latent reasoning still faces two challenges, especially in training-free settings: 1) purely latent reasoning broadens the search distribution by maintaining multiple implicit paths, which diffuses probability mass, introduces noise, and impedes convergence to a single high-confidence solution, thereby hurting accuracy; and 2) overthinking persists even without explicit text, wasting tokens and degrading efficiency. To address these issues, we introduce SwiReasoning, a training-free framework for LLM reasoning which features two key innovations: 1) SwiReasoning dynamically switches between explicit and latent reasoning, guided by block-wise confidence estimated from entropy trends in next-token distributions, to balance exploration and exploitation and promote timely convergence. 2) By limiting the maximum number of thinking-block switches, SwiReasoning curbs overthinking and improves token efficiency across varying problem difficulties. On widely used mathematics, STEM, coding, and general benchmarks, SwiReasoning consistently improves average accuracy by 1.8%–3.1% across reasoning LLMs of different model families and scales. Furthermore, under constrained budgets, SwiReasoning improves average token efficiency by 57%-79%, with larger gains as budgets tighten.

Published as a conference paper at ICLR 2026
SWIREASONING: SWITCH-THINKING IN LATENT AND EXPLICIT FOR PARETO-SUPERIOR REASONING LLMs
Dachuan Shi', Abedelkadir Asi?, Keying Li", Xiangehi Yuan', Leyan Pan',
Wenke Lee Wen Xiao Georgia Tech Microsoft
swireasoning.github..
Math&STEM
Math&STEM
62.95
Math&GSTEM
(+2.0%)
Matth&STEM
$4.2%
Coding
6-3.350
General
BE.45
Poscet Accurtt 18)
Switto Satthotoooo Soppoiig (Ourr)
60.2%
77.1%
(Crogg Manonsoouiin
01.78
37.5%
74
73.2%
75
192
124
14.2
34.3
14.1
78.1
00.2
74.
Figure 1: @ass@1 1 accuracy under unlimited token budgets. Across the MTMM& Coding, and General reasoning benchmarks, SWIREASONING yields average gains of +2.1%, + +3.1%, and +1.9%, respectively.
Ooven1-00
mas euiiiinne
Owerr
sun ellirioncy
Delliceury
Goweston Length
Generaton CAn
Devection
Figure 2: Token efficiency (aceuracy per token compared standard CoT), under limited token budgets. Across reasoning LLM families and sizes, SWIRFASONING brings average efficiency improvements of up to + 79%.
ABSTRACT
Recent work shows that, beyond discrete reasoning through explicit chain-of- thought steps, which are limited by the boundaries of natural languages, large language models (LLMs) can also reason continuously in latent space, allowing richer information per step and thereby improving token efficiency. Despite this promise, latent reasoning still faces two challenges, especially training-free set- tings: 1) purely latent reasoning broadens the search distribution by maintaining multiple implicit paths, which diffuses probability mass, introduces noise, and impedes convergence to single high-confidence solution, thereby hurting accu- racy; and 2) overthinking persists even without explicit text, wasting tokens and degrading efficiency. To address these issues, we introduce SWIREASONING, a training-free framework for LLM reasoning which features two key innovations: SWIREASONING dynamically switches between explicit and latent reasoning, guided by block-wise confidence estimated from entropy trends in next-token dis- tributions, to balance exploration and exploitation and promote timely conver- gence. 2) By limiting the maximum number of thinking-bloc block switches, SWIRFA- SONING curbs overthinking and improves token efficiency across varying problem difficulties. On widely used mathematics, STEM, coding, and general bench- marks, SWIREASONING consistently improves average accuracy by 1.8%-3.1% across reasoning LLMs of different model families and scales. Furthermore, der constrained budgets, SWIREASONING improves average token efficiency by 57%-79%, with larger gains as budgets tighten.
EEuual advising.

---

Published as a conference paper at ICLR 2026
1 INTRODUCTION
Reasoning is one of the central capabilities of large language models (LLMs) (Yang et al., 2025; Qwen Team, 2024; Meta, 2025a;b). It allows models to tackle complex tasks such as mathematics, science, and programming (Guo al., 2025; OpenAl, 2025b; Jaech et al., 2024; Agarwal et al., 2025; Qwen Team, 2025; Abdin et al., 2025; Abouelenin et al., 2025; Anthropie, 2025; DeepMind, 2024a;b), far beyond simple question answering.
A key limitation of the dominant reasoning approach, explicit chain-of-thought (CoT) (Wolf et al., 2020; Wei et al., 2022; Yao al., 2023a; Goyal et al., 2024; Pfau et al., 2024), lies in the reliance on discrete tokens during inference. In standard CoT decoding, the model commits to single token each step. sampled from the predicted distribution. While effective and ensures readability by verbalizing intermediate steps, this discrete process collapses the full probability distribution into single trajectory, discarding uncertainty and eliminating many potentially useful reasoning paths.
To overcome this bottleneck, recent work has explored an alternative reasoning technique, latent reasoning (Hao et al., 2024; Zhang et al., 2025; Cheng & Van Durme, 2024; Xu et al., 2025a;b; Tan et al. 2025), where the model operates directly in continuous hidden space instead of discrete text space. Latent reasoning offers two key advantages over CoT: 1) higher representational bandwidth per step, since hidden vectors encode richer information than single tokens (Zhu et al., 2025b); and 2) the ability to preserve multiple reasoning hypotheses implicitly, rather than collapsing them prematurely into one tokenized path (Li et al., 2025b; Chen et al., 2025).
Latent reasoning can be broadly categorized into training-required and training-fres approaches. Training-required ones (Hao al., 2024; Su et al., 2025; Liu et al., 2024; Shen et al., 2025; Tack et al., 2025) demand substantial retraining or fine-tuning (Yue et al., 2025; Li et al., 2025a; Wang et al, 2025a; Zhu al., 2025a), making excessively expensive to apply to large reasoning language models. In contrast, training-frree approaches like Soft- Soii-Thinning (Zhang et al., 2025), which form a probabilitywweihhuee
mixture of token embeddings as inputs, operate directly at inference time without incurring additional training costs. Our work focuses on the latter category, which is cost- effective and resource- friendly for deployment in large-scale reasoning models.
Although training-free free latent reasoning eliminates the need for costly retraining, operating purely in the latent space also presents significant challenges. First, the model is not explicitly trained to perform long-horizon reasoning with latent inputs. As a result of distributional mismatches, when inference relies solely on latent trajectories, the process is less controlled and can easily drift off course (Chen et al., 2025). Instead of collapsing into a single path, the model tends spread probability mass across many implicit reasoning paths. While this preserves multiple hypotheses, it also introduces persistent noise, slows convergence, and ultimately harms reasoning accuracy (Li et al., 2025b). Second, the absence of explicit tokens does not necessarily ensure efficiency. In latent space, models may still suffer from repetitive unnecessarily extended internal deliberations and continuation (Zhang et al., 2025), essentially overthinking. This prolongs inference and over- consumes tokens, undermining the very efficiency that latent reasoning is meant improve.
To address these issues, this paper introduces SWIREASONING (abbreviated as SwIR) as a training- free framework for LLM reasoning that alternates between explicit and latent thinking, based on block-wise confidence inferred from entropy trends of next-token distributions, and suppresses over- thinking by bounding the number of switches. More specifically, the framework first tracks a refer- ence entropy within each thinking block to reflect block- wise confidence. Rising confidence triggers an explicit switch to consolidate progress along a single path, while sustained uncertainty triggers a latent switch to re-explore in continuous space. Second, switch count controller caps the num- ber of thinking block transitions and provides carly-answer checkpoints, curbing unnecessary latent loops and improving token efficiency across difficulties.
The proposed framework also benefits from reintroducing diversity by sampling in an explicit think- ing block when compared to pure latent thinking. Even though motivated differently, SWIREASON- ING resonates with the concurrent observation of Wu et al. (2025b) that introducing stochasticity benefits latent reasoning, but we achieve this via distinct mode switch mechanism rather than injecting distributions with randomness.
Our contributions are summarized follows:
2

---

Published as a conference paper at ICLR 2026
We propose SWIREASONING, training-fret free reasoning framework that dynamically altemates be- tween explicit and latent thinking based on confidence signals, thereby exploiting the expressivity of latent thinking without sacrificing the stability of explicit thinking.
We introduce switch count control mechanism that caps the number of transitions, enabling early answering based on partial reasoning trajectories at switch boundaries. This effectively suppresses overthinking and improves token efficiency under limited budgets.
We extensively validate the effectiveness of S SWIREASONING general reasoning domains across multiple benchmarks, model families, and sizes, demonstrating consistent gains in both accuracy and token efficiency over training-free free baselines.
mathematics, STEM, coding, and
RELATED WORK
Explicit LLM Reasoning. Reasoning via explicit intermediate text has been extensively studied. Chain-of-thought (Co'T) prompting elicits stepwise rationales that improve reasoning accuracy by decomposing problems into natural-language sub-steps (Kojima al., 2022; Wei al., 2022). Sub- sequent work increases robustness by aggregating multiple CoT trajectories through self- self-consistency (Wang et al., 2022). Search- and tool-augmented variants further expand the exploration space, such as Tree-of-though that branches over partial rationales (Yao et al., 2023a) and interleaving reason- ing and actions with external tools and environments (Yao et al., 2023b). Least-to tossstt prompting progressively solves subproblems to reduce reasoning load and mitigate error accumulation (Zhou et al., 2022). These approaches operate purely in the discrete token space and therefore commit to a single token at each step. While readable, the discretizations in explicit reasoning discard alternative hypotheses carly, and restrict the information bandwidth per step (Zhu et al., 2025b).
Latent LLM Reasoning. Latent reasoning operates in the continuous representation space rather than discrete natural language space used by explicit reasoning. Prior work can be broadly grouped into two categories. 1) Training-require approaches modify pretraining (Tack et al., 2025; Zeng et al., 2025) or fine-tuning objectives (Tan et al., 2025; Wang et al., 2025a;;; Jiang et al., 2025; Wu et al., 2025a; Yue et al., 2025; Li et al., 2025a; Shen et al., 2025; Xu et al., 2025a) to su- pervise hidden-state trajectories approaches (Zhang et al., 2025; Wu et al., 2025b) intervene only at inference time by manipulating hidden representations or probability distributions without updating model weights. Our work be- longs to the training-fiee free category but differs from prior single-mode methods. Instead of remaining purely latent, SWIREASONING dynamically switches between latent and explicit reasoning based on entropy- trend confidence, and further regulates the number of switches through a count controller to suppress overthinking and improve efficiency.
to endow models with latent-planning skills. 2) Training-free
3 METHODOLOGY
3.1 SWTREASONING OVERVIEW
As shown in Fig. 3. SWIREASONING is training-free framework that dynamically alternates be- tween explicit and latent reasoning. The number of switches is regulated to suppress overthinking and improve token efficiency. Sec. 3.2 presents the preliminaries of explicit and latent reasoning, Sec. 3.3 details the design of the dynamic switch, and Sec. 3.4 discusses the switch count control mechanism. Implementation details are provided in Appendix B.1.
3.2 PRELIMINARY: EXPLICIT AND TRAINING-FREE LATENT THINKING
Explicit Thinking. Let V be vocabulary and (r(, (ise) an LLM over V with parameters 0. Given a question q, the model produces a reasoning trace FiT E VT followed by a final answer a1:U VV, We write the concatenated sequence
[g. rI:, awu).
[1([c/+++0)
3

---

Published as a conference paper at ICLR 2026
Thinking Mode Switch
Explicit
Latent
Mode I
Swittch
Entropy
Unconfident
Entropy
Switch
(b) Switch Count Control
G Stop
Unconfident
Confident
Confident
Final Answer
Position
Position
Stop
X
Mixing
Argnax
Probs
Done
Reasoning
Reasoning LLMs
Vocabulary:
X
X
Embeddings:
Input
Mixing
@:Weighted
Figure 3: SWIREASONING framework. (a) Dynamic mode switching alternates between explicit and latent thinking based block-witd confidence estimated from entropy trends. (b) A switch count control mechanism limits the maximum number of thinking-block transitions, suppressing overthinking before the final answer.
At inference, decoding proceeds by repeatedly choosing a token from the predictive distribution po(: (ee)) according to a policy (e.)...g..
Greedy: arg max.ev Po(u x<())
St me with n.
Sampling: Top-k/Top-p with temperature т.
The reasoning phase stops when termination condition is met, e.g., generating (/think), after which the answer tokens 01:U' are decoded in the same manner. While explicit reasoning improves reliabil- ity by externalizing intermediate steps, its hard policy =*(-) collapses the full distribution single discrete decision at each step, ie., discards information p- (iee) beyond the chosen token.
Training-Fres Latent Thinking. It replaces the hard policy (-)-) by continuous surrogate that preserves distributional information. Let E R V denote the token embedding matrix with rows e(*) ERd. At step t, the model yields logits le E IR and distribution pt po(- xe)) E 1 it forms a soft embedding
softmax(4:). Given the next-token
ē = Σ pe(o] el E Rd
(1)
"EV
and feeds E, back to the model as the next imput representation, rather than committing to an explicit token by (-).). Upon the thinking phase being complete, the policy reverts to Te(-) for answer generation. The convexity of Eq. ensures é, lies in the embedding hull of E, retaining all first-
order uncertainty in Pt, which reduces information discards and increases robustness local noise.
3.3 DYNAMIC SWITCH BETWEEN EXPLICIT AND LATENT THINKING
Remaining in a single mode throughout reasoning is inherently suboptimal: explicit thinking pro- vides readability but may discard useful information beyond chosen tokens, while latent thinking preserves richer signals but can drift into noise and reduce accuracy. Our key insight that reasoning should switch modes based on confidence. Latent reasoning enables exploration across multiple po- tential continuations when confidence is low, and explicit reasoning encourages convergence when confidence is high, striking a balance that supports broad exploration while maintaining accuracy.
Mode Switch Criterion. We refer to the reasoning content between two consecutive switches as thinking block and estimate its confidence by entropy H, = Σ. Pc[v] log Pe [u]. Let H denote the reference entropy of the current block, which is initialized at the first step of the block and refreshed when a mode switch happens. We use a criterion that converts local entropy trends into decisions:
Latent fotent->Exppa Explicit:::
(H, H) (confidence rises), (H, > M) (confidence drops),
(2)
Explicit Latent:
(3)
4

---

Published as a conference paper at ICLR 2026
Switch Window Size. To avoid oscillations, impose dwell windows upon the mode switch criterion. Formally, with mode variable m, E {Explicit, Latent] and dwell step counter Åt, we have
Explicit,
my = Latent A (H, < H) A (AL 2 Wi.-E), my = Explicit A (H1 > Hf A ( WE-L), otherwise.
M++1
Latent,
mh.
We reset H H4, At 0 upon any switch, i.e., m4+1 # m, Otherwise, update At+ At+1. In practice, Wi-EE = while WE-L is positive, i.e., a Latent-+Explicii switch may occur immediately when H2 dips, whereas an Explicit- switch requires staying for at least WE-L steps.
The key intuition behind the asymmetric design is that two modes play different roles in reasoning. Latent reasoning is inherently divergent, allowing for rich exploration. However, prolonging the latent phase after confidence has recovered is counterproductive. increases the risks of introducing spurious signals that may mislead the model. Therefore, once confidence rises, immediate switch back to explicit reasoning is necessary to consolidate progress onto a single coherent trajectory.
In contrast, explicit reasoning is convergent, gradually unfolding chain-of-thought where each token incrementally extends the current logical path. If the model were allowed to switch back to latent reasoning the first sign of an entropy fluctuation, spurious short-term uncertainty could trigger oscillations. The dwell window WE-L ensures that explicit reasoning is given sufficient opportunity stabilize and accumulate meaningful structure.
Thinking-Related Signal Mixing. To better align mode switches with the LLMs' learned rea- soning patterns, we blend the embeddings of thinking-related signal tokens, egg., <think> and </think>, when switch occurs. Let ((hink) and eythnnk) denote their embeddings. At the en- trance to a latent thinking block, we bias the first latent step 1" toward "begin thinking" by
E Ogr -E-- + (1 ar-) ) E(tinkk)
a+ E [0, 1],
(4)
and at the exit to explicit thinking block, we bias the first explicit step t toward "end thinking"
ēs Be
(1 Bat) (e ((hink))
Brt E [0.
which encourages the model to close the latent phase and move on answer production. In practice we schedule a₁ = ag + (1 - an) ) T and Br = So + (1 _ where Tmax a predefined maximum generation length, and apply Eq. or Eq. 5 only at the steps when switches occur.
3.4 OVERTHINKING SUPPRESSION BY SWITCH COUNT CONTROI.
Even with confidence-awee switching, reasoning LLMs may still overthink. Therefore, place bound on the total number of Latent->Explice switches. Our key insight is that each switch natu- rally marks the end of a thinking block where partial reasoning trajectories have been consolidated, which may already contain sufficient evidence for arriving at reasonable solution. Under limited budgets, generating answers these natural checkpoints can make use of partial reasoning trajecto- ries, offering chance to obtain correct predictions earlier without consuming additional tokens.
Counter and Triggers. Let C2 count completed Latent Explicit switches up to step t. Given user-specified budget Cruax. we define two triggers:
Convergence trigger (at C I C. < Gmax on Latent Latent->Bit Explicit transitions): force the next token to be //think). The convergence trigger to encourage rather than enforce the end of the thinking process and the start of converging to an answer based on partial reasoning trajectories.
Termination trigger (at C. > Cmax on subsequent Latent - Explicit transition): inject concise answer prefix Sfinal "/ttink)\ The final answer is", then allow at most B additional tokens for the final answer. The termination trigger is to enforce an immediate answer generation.
Triggers are implemented as short injection queues that overwrite future-generated tokens. Formally, let Q, be the par-la injection queue. When a convergence or termination trigger fires, we set Q IID [D((/hinn))] or [S(ssa)))) At the next step, if Q₁ # 6, deterministically set +- Qopp For the termination trigger, we also start a budget counter b, = B and decrement it each step after the termination trigger fires. Decoding will be terminated once b, 00.
5

---

Published as a conference paper at ICLR 2026
4 EXPERIMENTS
4.1 EXPERIMENTAL SETTINGS
Models. We evaluate SWIREASONING on four recent reasoning LLMs: DeepSeek-RI-Disse Llama-8B (Guo et al., 2025), Qwen3-1.7B (Yang et al., 2025), Qwen3-8B (Yang al., 2025), and Qwen3-32B (Yang et al., 2025). This selection helps us validate the effectiveness of SWIREASON- ING across different model families, model scales, and training paradigms.
Domains and Benchmarks. We evaluate SWIREASONING on 11 benchmarks spanning four do- mains: mathematical reasoning (GSM8K (Cobbe et al., 2021), Math500 (Hendrycks et al., 2021), I), AIME 2024 (HuggingFaceHH4, 2024), AIME 2025 (Yentinglin, 2025)); STEM reasoning (GPQA Diamond (Rein et al., 2024)); coding reasoning (HumanEval (Chen al., 2021), LeetCode- Contest (Guo et al., 2024), MBPP (Austin et al., 2021), LiveCodeBench (Jain et al., 2024)); and general reasoning (2 (2WikiMultihopOA (Ho al., 2020), CommonsenseQA (Talmor et al., 2019)).
Baselines. We compare SWIREASONING that dynamically switches between thinking modes against three baselines with single thinking mode, including 1) explicit thinking alone: standard CoT reasoning with sampling, standard CoT reasoning with greedy decoding, and 2) training-free latent thinking alone: Soft Thinking (Zhang al., 2025).
Metries. We use the Pass@1 metric to evaluate reasoning accuracy, and token efficiency to assess the level of reasoning efficiency. Let Accm E [0, 1J denote the accuracy of method m when using generated tokens. Its plain token efficiency is the accuracy gained per token.
(E = Acco (4)
To express efficiency units relative to the standard CoT, we normalize it by the CoT's plain token efficiency when the highest accuracy is achieved. Specifically, if CoT achieves its highest accuracy Acceor using foor tokens, denote P PECor Acetur, The token efficiency of defined as
(E) Accm(e)/e PETT
Em.(e)
AccCor/ECoT
And the average efficiency gain of method Co'T is
[[Em(e) Ecor(!)) de
E A[
S (C) dl
4.2 REASONING ACCURACY UNDER UNLIMITED TOKEN BUDGETS
We first evaluate SWIREASONING in the setting where token budgets are set large enough to en- sure that each method is allowed to conduct sufficient thinking (refer to Appendix B.2 for de- tailed settings). Fig. 1 and Tab. report the highest attainable accuracies across mathematics (GSM8K, MATH500, ALME24, AIME25) and STEM (GPQA Diamond) benchmarks under this setting. Across different model families of varying sizes, SWIREASONING consistently achieves higher @ I accuracy than CoT with sampling, CoT with greedy decoding, and Soft Thinking.
Our observation that improvements are most pronounced on the more challenging benchmarks. For instance, AIME24/AIME25, which require deep deductive reasoning and are widely re- garded as more difficult, our method yields absolute gains of 3.34%% on Qwen3-8B, and 5.00% 100%%5500% 5..0%% on Qwen3-1.7B. These margins substantially exceed those observed GSM8K or MATH500 with lower difficulty, suggesting that the proposed switching mechanism is particu- larly beneficial when problems involve long reasoning chains higher uncertainty from the LLM's perspective. Overall, the accuracy results under unlimited token budgets highlight the strength of SWIREASONING in better addressing reasoning tasks compared to single-mode approaches.

---

Published as a conference paper at ICLR 2026
Table 1: Comparison of SWIREASONING and CoT with sampling. CoT with greedy decoding, and Soft Think- ing on mathematics and STEM benchmarks. SWIREASONING improves accuracy by + +2.17% on average.
MATH 500
GPQA
AIME 2024
AIME 2025
GSM8K
Average
Method
Diamond
Qwen3-8B (Yang et al., 2025)
CoT
95.60
96.00
59.60
75.83
67.50
78.91
+0.00
CoT (Greedy) 95.68
96.40
56.57
70.00
60.00
75.73
-3.18
Soft Thinking 95.38
96.00
59.60
67.92
68.33
77.45
-1.46
SwiR (Ours) 96.06
+0.46
98.40
-2.40 61.11
+1.51
79.17
+3.34
70.00
+2.50
80.94
+2.03
Qwen3-1.77 (Yang et al., 2025)
CoT
90.44
92.00
39.39
45.83
33.33
60.20
+0.00
COT (Greedy) 89.61
91.00
31.82
40.00
33.33
57.15
-3.05
Soft Thinking 90.30
90.60
34.34
38.75
36.67
58.13
-2.07
SwiR (Ours)
90.83
+0.39
93.00
+1.00
41.41
+2.02
50.83
+5.00
38.33
+5.00
62.88
+2.68
DeepSeek RI RI-Distilll Llama-88 8B (Guo et al., 2025)
CoT
89.46
91.40
46.46
43.75
26.25
59.46
+0000
CoT (Greedy) 85.82
84.80
31.81
30.00
30.00
52.49
-6.97
Soft Thinking 85.90
83.80
33.33
34.17
20.42
51.52
-7.94
SwiR (Ours) 90.07 +0.61
92.00
+0.60
47.98
+1.52
45.00
+1.25
31.25
+5.00
61.26
+1.80
4.3 TOKEN EFFICIENCY UNDER LIMITED TOKEN BUDGETS
Across models and benchmarks, SWIREASONING consistently attains improved Pareto frontiers. As shown in Fig. 2, the peak efficiency gains range between 4.6x and 6.8x over CoT depending on the model size. These improvements are not confined to single budget: the area-under-curve (AUC) advantage persists across a broad range of small to moderate budgets.
One observation from Fig. that the relatively large average efficiency gains occur on GSM8K, MATH500, and GPQA Diamond across three models (up to 213% AUC improvements in the per- benchmark panels). These tasks contain many instances with lower difficulty, which benefit most from overthinking suppression design to obtain the correct answer with partial reasoning trajec- tories. In contrast, on AIME24/25, the efficiency gaps are smaller, while the accuracy gains with unlimited budgets larger. This asymmetry is expected: the harder the problem is, the more diffi cult it is to predict a correct answer with unfinished reasoning trajectories. Overall, token efficiency results under limited budgets substantiate the advantage of SWIREASONING in gaining accuracy more efficiently as budgets tighten compared to baseline methods.
4.4 EVALUATION WITHI PASS @K K ACCURACY
In addition to Pass @ accuracy, we also measure Pass@k accuracy, where h: E [1, 64] on Qwen3- 8B. Fig. 5 shows that SWIREASONING reaches its maximal accuracy with significantly smaller than baselines. Define k* as the smallest & achieving the method's peak. On AIME24, we observe k* 13 for SWIREASONING versus 46 for CoT (about 72% fewer samples), and on AIME25, k = 16 versus 22 (about 27% fewer samples). In addition to the faster growth of the curve than COT, SWIREASONING also exhibits 1) steeper initial slope at small (higher "per-sample yield"), and 2) higher eventual ceiling than Soft Thinking and greedy COT, indicating better correctness and diversity simultaneously. Overall, Pass @k accuracy results indicate that SWIREASONING is particularly attractive for budgeted evaluation settings where & cannot be large.
4.5 ABLATION STUDIES
Switch Window Size. SWIREASONING uses dwell windows (Sec. 3.3) to enforce the model stays in thinking block for at least W steps before switching to the other thinking mode. We conduct ablation studies on Qwen3-1.7B with representative setting consisting of WE-L (64, 128, 256, 512, 1024) and report Pass @ accuracy on five benchmarks. Results in Tab. 3 demon- strate that intermediate window size of 512 consistently produces the best results.
7

---

Published as a conference paper at ICLR 2026
OverO-DB
CSPBR
Tulf Ohunn
thndis
8-130TT
1.
270
1750
1000
12h3
1060
312
13:0
1300
MAT1500
MATT5520
23
130
1000
(AQ6-1 amond
50
B[
34
20
1000
18:00
6:00
683
340
800
X08
380
4.00
300
804
AMP25
AB477
7[buthiimayy) +-301!
Coneration (inght
Game 2t Length 14]
Figure 4: Token efficiency comparisons. SWIREASONING achieves highest token efficiency throughout all
token budgets in 13 out 15 evaluations, with an efficiency improvement of + 84% over CoT on average.
Maximum Accuracy
Masimum Accuracy
Pessgl aicuracy (3)
Por (%)
K-13
72% Earlier
K-d6
2024
27%
AIME
CCT (Crendy) Thaslnng
Syrt Tharkrng
CoT (Sowedy)
(Cum)
Figure 5: Pass@k accuracy (k E [1, (4) evaluation with Qwen3-8B on AIME 2024 and 2025 benchmarks.
SWIRFASONING achieves maximum reasoning accuracies +550% earlier compared to CoT on average.
When window sizes are small, LLMs may jump back latent mode prematurely, before explicit reasoning has consolidated coherent trajectory. This increases exposure to noisy signals and harms final accuracy, especially on difficult tasks such as AIME24 and AIME25. When window sizes are too large, I.LMs become sluggish to reenter latent exploration confidence declines. A promising improvement direction make adaptive the model's real-time density effective reasoning.
Thinking- Relatedg-Relatd Signal Mixing. SWIREASONING Cu. Bu [0. the initial ratios for mixing thinking-reliieee related signals at switching instants (Sec. 3.3). We sweep ag and Bn independently and report Pass @ accuracies in Tab. 2.
8

---

Published as a conference paper at ICLR 2026
Table 2: Ablations on cxo and & for signal mixing. Greener indicates better performance within each column.
MATH 500
GPQA Diamood
AIME 2024
AIME 2025
MATH 500
GPQA Diamond
AIME 2024
ALME 2025
GSM8K
Average
30
GSMHK
Average
0.0
89.23%
89.80%
35.86%
46.67%
35.00%
59.31%
0.0
81.50%
67.20%
28.79%
8.33%
9.17%
39,0033
0.1
84.%
91.00%
36.36%
46.25%
36.25%
59. 39.94%
0.1
81. 88%
70.20%
31 8.%
11.67%
8.75%
40 86.8
91.60%
37.50%
82.11%
28.28%
9.17%
40.87%.
0.3
90.45%
91. .60%
38.38%
47.08%
38.33% %
61.17%
0.3
90.67%
92.00%
37.37%
45. 45.44%%
37.92%
60.68%
0.4
89.61
80%
40.91%
48. 33%
32.50%
60.83%
90.98 %
91. .40%
88.8
47.92%
36.67%
60.97%
0.5
90.45%
93.00%
34.34%
50.83%
36.25%
60.97%
0.5
90.37%
91.20%
42.42%
47.92%
35.83%
61.55%
0.6
90.83%
92.000%
39,391%
44.58%
37.92%
60.94%
0.6
90.59'
90.40%
42.42%
42.50%
36.67%
60.52%
0.7
90.06%
91 1.60%
37.37%
45.00%
37.08%
60.22%
0.7
90.83%
93.00
41.41%
50.83%
38.33%
62.88%
0.8
90.00%
92. 009%
37. .7%
48%33%
35.42%
60. 70.74%%
0.8
89. 99%
92, 20%
39.39%
49. 17%
35 833
61.32%
39.
50.42%
92.20%
48.75%
60.52%
1.0 /
90.14%
90.60%
41.41 %
49. 17%
37.92%
61.85%
1.0
90.44%
91 1.000%
33.33%
46.67%
38.75%
60.04%
For the exit bias 9o. very small Bo implies excessive interference with when to conclude the think- ing process and severely degrades accuracy (e.g., AIME24 drops to 8.33% at 30=0.0). Performance rises sharply and peaks near =070.7, which achieves the best average 62. 62.88% and is either the best or the second-bes best on most datasets. A promising improvement direction is to make Bu difficulty- aware, so that it will be automatically adjusted based on problem difficulty.
The situation for the entrance bias On is different. We observe a broad performance plateau for oo E [0.4, 0.9), with the highest average 01=1.0 (61.85%) however, only marginally higher than other values like 00=0.9 (61.36 136.).)). Task-wise, problems with different levels difficulty tend to have various preferences over ag. We expose ag users for adjustment based on task difficulty. The more detailed hyperparameters we adopted for the experiments are provided in Appendix B.3.
Maximum Switch Count. We suppress over- thinking by bounding the number of mode switches with budget Cmax (Sec. 3.4), and re- ducing Cmax leads to earlier convergence. In Fig. 2 and Fig. 4, moving rightward on the a-axis corresponds
Table 3: Ablation on switch window size. Greener in- dicates better performance within each column.
smaller token budgets, i.., smaller Cuaxx- We collect data points in these figures by incrementing the value of Ciax
Size
Window
GSMBK
500
GPQA Diamond
AIME 2024
AIME 2025
Average
64
89.69%
92.60%
40.91%
47.92%
34.17%
61.06%
128
90.45%
91.00%
89%
48.33%
36.259
60 98%
256
89,76%
90.80%
39.90%
49.58%
36.25%
61 61.26%
from I until further increases in Cmax no longer alter generation results in most cases, i.e., max- imum accuracy is reached at saturation. De- tailed data is provided in Appendix C.8.
512 /
90.83%
93.00%
41.41%
50.83%
38.33%
62.88%
91.20%
As analyzed in Sec. 4.3, decreasing Caux yields significant improvement in token efficiency, which confirms the intended behavior of the switch count control design: it curbs prolonged latent exploration and commits to answer path early, thereby mitigating overthinking. With switch count control, a small number of confidence-aware blocks usually suffices for masdeteeeoeerate problems, while difficult instances benefit more from allowing few more switches before the final answer.
More ablation studies are provided in Appendix C.5-C.7.
4.6 EXPERIMENTAL RESULTS ON LARGER MODELS
Table 4: Comparison of SWIRFASONING and CoT with sampling, CoT with greedy decoding, and Soft Think- ing on Qwen3-32B. SWIREASONING improves accuracy by +1.92% on average.
MATH
GPQA Diamond
AIME 2024
AIME 2025
Method
GSM8K
Average
500
CoT
95,83
97.40
66.16
80.42
72.08
82.38
+0.00
CoT (Greedy) 95.91
97.20
69.70
80.00
73.33
83.23
+-0.85
Soft Thinking 95.75
97.40
67.17
74.58
66.25
80.23
-2.5
SwiR (Ours) 96.21
+00.88
98.40
+1.00
70.20
+1.04
82.92
+2.50
73.75
+1.67
84.30
+1.92

---

Published as a conference paper at ICLR 2026
In addition to the 1.7B and 8B settings, evaluate SWIREASONING the 32B scale using Qwen3- 32B. We report pass@1 accuracy on GSM8K, MATH500,, GPQA Diamond, AIME 2024, and AIME 2025. Baseline hyperparameters follow the recommendations from their original papers, and all methods use the same configuration as in the main paper. Table shows that under no token budget constraint, SWIRFASONING improves accuracy by +1.92% average gains are most notable on more difficult benchmarks such as GPQA Diamond (+4.04%). These results indicate that SWIREASONING scales larger models and achieves consistent accuracy gains.
standard COT. The
4.7 EXPERIMENTAL RESULTS ON BROADER DOMAINS
Table Comparison of SWIREASONING and baselines coding. multi hopp QA, and commonsense reasoning tasks. SWIREASONING improves accuracy by + +2.00% on average.
CeetCode-Conteer
Method
HumanEval
MBPP
Easy-level
Medium-level
Hard-level
CoT
92.68
57.78
68.13
43.18
94.16
CoT (Greedy)
93.90
64.44
58.24
47.73
91.44
Soft Thinking
92.07
55.56
61.54
38.64
94.16
SwiR (Ours)
95.73
+3.05
64.44
+6.66
69.23
1+1.10
61.36
+18. 18
95.33
|+1.17
LiveCode- Bench
2WikiMul- tihopQA
Common- senseQA
Method
Average
CoT
62.01
79.00
83.95
78.54
+0.00
CoT (Greedy)
50.18
79.50
83.95
76.03
Soft Thinking
56.99
79.00
83.70
76.73
-1.81
SwiR (Ours)
63.44
I ++1.43
81.50
+2.50
85.34
++1.39
81.24
+22.7
In addition math and STEM domains, further evaluate SWIREASONING on coding, multi- hop QA, and commonsense reasoning. We use Qwen3-8B and report pass@1 accuracy on Hu- manEval (Chen et al., 2021), LeetCode-Contest (Guo et al., 2024). MBPP (Austin al., 2021), and LiveCodeBench (Jain et al., 2024) for coding, 2WikiMulihopQAA (Ho et al., 2020) set from LongBench (Bai al., 2024) for multi-hop QA, and CommonsenseQA (Talmor al., 2019) for commonsense reasoning. As shown in Tab. 5, under no token budget constraint, SWIREASONING improves accuracy by +2.70% on average over standard CoT (the reported average the simple mean of accuracy over the full LeetCode-Contes and accuracies over five other benchmarks).
coding tasks, the largest gains 1811.18%) are observed on the hard-level subset, indicating that SWIREASONING is most helpful for problems that require stronger reasoning capabilities. On multi- hop QA tasks, which require retrieving and connecting multiple disparate facts, SWIREASONING outperforms CoT by +2.50%. This suggests that the exploration capability of latent reasoning is effective in navigating complex reasoning paths. On commonsense reasoning tasks, SWIREASON- ING surpasses CoT by +1.3%%. This demonstrates the robustness of SWIREASONING in general knowledge scenarios. Overall, these results confirm that SWIREASONING generalizes to broader domains with consistent accuracy gains.
CONCLUSION
This paper presents SWIREASONING, a training-free inference framework that integrates explicit shain-of-thoughh thinking with latent thinking through an entropy trends-based controller. The framework is conceptually straightforward but empirically effective: when block-wise uncertainty decreases, we collapse
single explicit path consolidate progress. When uncertainty rises and has persisted for a minimal dwell window, we expand into latent space to explore more alternatives. Complementing this mode switch, a switch count controller caps the number transitions, thereby curbing overthinking while preserving prediction quality. Together, these two mechanisms yield consistently improved Pareto frontiers for reasoning LLMs, effectively enhancing both maximum accuracy under unlimited budgets and token efficiency under limited budgets. Looking ahead, in- tegrating SWIREASONING with reinforcement learning- based training may unlock even stronger reasoning capabilities.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
The authors would like thank Microsoft and Georgia Tech for their support. This material is based upon work supported by the National Science Foundation under grant no. 2229876 and is supported in part by funds provided by the National Science Foundation, by the Department of Homeland Security, and by IBM. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation or its federal agency and industry partners.
ETHICS STATEMENT
This work focuses enhancing the reasoning accuracy and token efficiency of LLMs, which does not raise safety concerns. This work involves no collection of sensitive data. All evaluations are conducted using publicly available models and benchmarks under their original licenses.
REPRODUCIBILIIY STATEMENT
We provide implementation details in Appendix B.1, details of the benchmark settings in Ap- pendix B.2, and details of the hyperparameters in Appendix B.3 to facilitate reproducibility.
REFERENCES
Marah Abdin, Sahaj Agarwal, Ahmed Awadallah, Vidhisha Balachandran, Harkirat Behl, Lingjiao Chen, Gustavo de Rosa, Suriya Gunasekar, Mojan Javaheripi, Neel Joshi, et al. Phi-4-reasoning technical report. arXiv preprint 133 2025.
Abdelrahman Abouclenin, Atabak Ashfaq. Adam Atkinson, Hany Awadalla, Nguyen Bach, Jianmin Bao, Alon Benhaim, Martin Cai, Vishrav Chaudhary, Congcong Chen, et al. Phi-4-mini technical report: Compact yet powerful multimodal language models via mixture-of-lore arXiv preprint arXiv:2503.01743. 2025.
Sandhini Agarwal, Lama Ahmad, Jason Ai, Sam Altman, Andy Applehaum, Edwin Arbus, Rahul Arora, Yu Bai, Bowen Baker, Haiming Bao, et al. gpt-oss-120b & gpt-oss-20b model card. arXiv preprint arXiv:2508. 109255, 2025.
Anthropic. System card: Claude opus 4 & claude sonnet 4. 2025. URL https://www anthropic. c. com wttpc/eaaeeeeeeeeeeeee 4- 4- emmeemmmarrrr
Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry Quoc Le, al. Program synthesis with large language models. arXiv preprint arXiv:2108.0771 07733, 2021.
Yushi Bai, Xin Lv, Jiajie Zhang. Hongchang Lyu. Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, et al. Longbench: A bilingual, multitask benchmark for long context understanding. In Proceedings the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3119-3137, 2024.
Zefan Cai, Yichi Zhang, Bofci Gao, Yuliang Liu, Yucheng Li, Tianyu Liu, Keming Lu, Wayne Xiong, Yue Dong, Junjie Hu, et al. Pyramidky: Dynamic kv cache compression based on pyra- midal information funneling. arXiv preprint arXiv:2406.0200 2024.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cummings, Matthias Plappert, Fo- tios Chantzis. Elizabeth Barnes, Ariel Herbert- Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec
11

---

Published as a conference paper at ICLR 2026
Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob Me- Grew, Dario Amodei, Sam McCandlish, llya Sutskever, and Wojciech Zaremba. Evaluating large language models trained code, 2021.
Xinghao Chen, Anhao Zhao, Heming Xia, Xuan Lu, Hanlin Wang, Yanjun Chen, Wei Zhang, Jian Wang, Wenjie Li, and Xiaoyu Shen. Reasoning beyond language: A comprehensive survey latent chain-of- thought reasoning. arXiv preprint arXiv:2505.. 16782, 2025.
Zhuoming Chen, Avner May, Ruslan Svirschevski, Yuhsun Huang, Max Ryabinin, Zhihao Jia, and Beidi Chen. Sequoia: Scalable, robust, and hardware-awwer speculative decoding, arXiv preprint arXiv:2402. / /2374, 2024.
Jeffrey Cheng and Benjamin Van Durme. Compressed chain of thought: Efficient reasoning through dense representations. arXiv preprint arXiv:2412.13171. 2024.
Karl Cobbe, Vincet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:21 10, 14168, 2021.
Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning. arXiv preprint (tt.vv:2377.086911 2023.
Tri Dao. Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory- efficient exact attention with io-awareness. Advances in Neural Information Processing Systems, 55:16344-16359 2022.
DeepMind. Alphazero: Shedding new light on chess, shogi, and go. 2024a.
URL
-hweeeedddiiiee-----ii--oooooooo -hww----iggg-----e---seeeeeeeeee wwwhcchhssssss--seeeeeeee shogisand--ggg--ee and-go//
DeepMind. Ai solves imo problems at silver medal level.
URL
solves-ino----eeeeeee-eo tettoo-rroootttsiieeeeeee silver-medat-levell 2024b.
Yonggan Fu, Zhongzhi Yu, Junwei Li, Jiayi Qian, Yongan Zhang, Xiangchi Yuan, Dachuan Shi, Roman Yakunin, and Yingyan Celine Lin. Amoeballm: Constructing any-shape large language models for efficient and instant deployment. Advances in Neural Information Processing Systems, 37:78299-7888 78319, 2024.
Sachin Goyal, Ziwei Ji, Ankit Singh Rawat, Aditya Krishna Menon, Sanjiv Kumar, and Vaishnavh Nagarajan. Think before you speak: Training language models with pause tokens. In The Twelfth International Conference Learning Representations, 2024.
Daya Guo, Qihao Zhu, Dejian Yang. Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Y. Wu, Y. K. Li, Fuli Luo, Yingfei Xiong, and Wenfeng Liang. Deepseek-coder: When the large language model meets programming the rise of code intelligence. arXiv preprint arXiv: 2401. 2024.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501. 12948, 2025.
Chi Han, Qifan Wang, Hao Peng, Wenhan Xiong, Yu Chen, Heng Ji, and Sinong Wang. Lm- infinite: Zero-shot extreme length generalization for large language models. arXiv preprint arXiv:2308. 16137, 2023.
Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in continuous latent space. arXiv preprint arXiv:2412.06769 2024.
Dan Hendrycks, Collin Burns. Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2 103.03874 2021.
12

---

Published as a conference paper at ICLR 2026
Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing multi- hop qa dataset for comprehensive evaluation of reasoning steps. arXiv preprint 1rXiv:2011.01060 2020.
HuggingFaceH4. Aime 2024 (american invitational mathematics examination 2024). Hugging Face dataset, 2024. URL htwph:ttp///t. httpss////hugginggfac.c co/ heoo/ataseeee hatps///aggggggggaee/aga/ 2024.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai ol system card. arXiv preprint arXiv:2412.167200 2024.
Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and lon Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974, 2024.
Huiqiang Jiang, Qianhui Wu, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. LImlingua: Compressing prompts for accelerated inference of large language models. arXiv preprint arXiv:3310005736, 2023.
Nan Jiang, Ziming Wu, De-Chuan Zhan, Fuming Lai, and Shaobing Lian. Dart: Distilling autore- gressive reasoning silent thought, 2025.
Schoon Kim, Karttikeya Mangalam, Suhong Moon, Jitendra Malik, Michael W Mahoney, Amir Gholami, and Kurt Keutzer. Speculative decoding with big little decoder. Advances in Neural Information Processing Systems, 36:39236-39233 2023.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199 222113. 2022.
Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, pp. 19274-19286. PMLR, 2023.
Hengli Li, Chenxi Li, Tong Wu, Xuekai Zhu, Yuxuan Wang. Zhaoxin Yu, Eric Hanchen Jiang, Song- Chun Zhu, Zixia Jia, Ying Nian Wu, al. Seek in the dark: Reasoning via test-time instance-level policy gradient in latent space. arXiv preprint
2025a.
Jindong Li, Yali Fu, Li Fan, Jiahong Liu, Yao Shu, Chengwei Qin, Menglin Yang, Irwin King, and Rex Ying. Implicit reasoning in large language models: A comprehensive survey, arXiv preprint 02Xi::2509.02350, 2025b.
Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. Awq: Activation-awarc weight quantization for on-device Ilm compression and acceleration. Proceedings of machine learning and systems, 6:87-100, 2024.
Tianqiao Liu, Zui Chen, Zitao Liu, Mi Tian, and Weiqi Luo. Expediting and elevating large language model reasoning via hidden chain-of thought decoding. arXiv preprint arXiv:2409.0856,, 2024.
Xiaoxuan Liu, Lanxiang Hu, Peter Bailis, Alvin Cheung, Zhijie Deng, Ion Stoica, and Hao Zhang. Online speculative decoding. arXiv preprint arXiv:2310.07177 2023.
2023.
checked on, 4(7):2025, 2025a. AI Meta. The laaaa 4 herd: The beginning of new of natively multimodal ai innovation. hawpl///lllaae---tmmmmmmmam-mlle
https://ai. meta.
AI Meta. Llama 3.3 model card. https: http://thhhu.ccmmmttteeaaaiaa blob/maing mwoees//////soooooooomcoom httpa/////mooaaaaaaaaaaa////0010 3/MMDDE_33333__ CAR. 2025b.
Yansong Ning, Wei Li, Jun Fang. Naiqiang Tan, and Hao Liu. Not all thoughts generated equal:
Efficient Ilm reasoning via multi-turn reinforcement learning. arXiv preprint arXiv:2505.1182) 2025.
13

---

Published as a conference paper at ICLR 2026
OpenAl. Gpt-5 system card. System Card v2025-08-13, OpenAI, San Francisco, CA, August 2025a.
URL https::/ /wnn. openai..com. S-spptt--asyttteeeee card. paf.
OpenAl. Openai o3-mini. URL http:://opondd.ccmm//mexxdppeeep
2025b.
Zhuoshi Pan, Qianhui Wu, Huiqiang Jiang, Menglin Xia, Xufang Luo, Jue Zhang, Qingwei Lin, Victor Rühle, Yuqing Yang, Chin-Yes Yew Lin, al. Limlingua-2: Data distillation for efficient and faithful task-agnostic prompt compression. arXiv preprint arXiv:2403. 12968, 2024.
Jacob Pfau, William Merrill, and Samuel R Bowman. Let's think dot by dot: Hidden computation in transformer language models. arXiv preprint arXiv:2404.15758, 2024.
Qwen Team. Qwen2.5 technical report. arXiv preprint arXiv:2412.151155,00
Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URL
httpss://wett wta.eml.....itti github wtt.ll/////o//oooc...oo.. httpl////////////////3 qwq-32b/.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Di- rani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proof q&a bench- mark. First Conference on Language Modeling, 2024.
Jay Shah, Ganesh Bikshandi, Ying Zhang. Vijay Thakkar, Pradeep Ramani, and Tri Dao. Flashattention-3: Fast and accurate attention with asynchrony and low www-precision. Advances in Neural Information Processing Systems, 17766658-666885 2024.
Zhenyi Shen, Hanqi Yan, Linhai Zhang, Zhanghao Hu, Yali Du, and Yulan He. Codi: Compressing chain-of-thought into continuous space via self-distillation. arXiv preprint arXiv:2502.21074, 2025.
Dachuan Shi, Chaofan Tao, Ying Jin, Zhendong Yang, Chun Yuan, and Jiaqi Wang. Upop: Unified and progressive pruning for compressing lingun-language transformers. In International Confer- ence on Machine Learning. pp. 31292-31311 PMLR, 2023.
Dachuan Shi, Chaofan Tao, Anyi Rao, Zhendong Yang. Chun Yuan, and Jiaqi Wang. Crossget:
Cross-guided ensemble of tokens for accelerating lision-language transformers. In Forty-First International Conference on Machine Learning, 2024.
Dachuan Shi, Yonggan Fu, Xiangchi Yuan, Zhongzhi Yu, Haoran You, Sixu Li, Xin Dong, Jan Kautz, Pavlo Molchanov, and Yingyan Celine Lin. Lacache: Ladder-shaped kv caching for effi- cient long-context modeling of large language models. In Forty-second International Conference on Machine Learning. 2025.
DiJia Su, Hanlin Zhu, Yingchen Xu, Jiantao Jiao, Yuandong Tian, and Qinqing Zheng. Token assorted: Mixing latent and text tokens for improved language model reasoning. arXiv preprint arXiv:2502.03275. 2025.
Ziteng Sun, Ananda Theertha Suresh, Jae Hun Ro, Ahmad Beirami, Himanshu Jain, and Felix Yu. Spectr: Fast speculative decoding via optimal transport. Advances in Neural Information Processing Systems, 36:30222-302433 2023.
Jihoon Tack, Jack Lanchantin, Jane Yu, Andrew Cohen, Ilia Kulikov, Janice Lan, Shibo Hao, Yuan- dong Tian, Jason Weston, and Xian Li. Llm pretraining with continuous concepts. arXiv preprint arXiv:2502.08554, 2025.
Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. Commonsensegaa A question answering challenge targeting commonsense knowledge. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisties: Human Lan- guage Technologies, Volume (Long and Short Papers), pp. 4159-4158, 2019.
Wenhui Tan, Jiaze Li. Jianzhong Ju. Zhenbo Luo, Jian Luan, and Ruihua Song. Think silently, think fast: Dynamic latent compression of Ilm reasoning chains. arXiv preprint 25Xi:25005.1552 2025.
14

---

Published as a conference paper at ICLR 2026
Xiaoqiang Wang, Suyuchen Wang, Yun Zhu, and Bang Liu. System-1.3 1.5 reasoning: Traversal in language and latent spaces with dynamic shortcuts. arXiv preprint arXiv:2505.1866. 2025a.
Xinyuan Wang, Dongjie Wang, Wangyang Ying, Haoyue Bai, Nanxu Gong, Sixun Dong, Kumpeng Liu, and Yanjie Fu. Efficient post- mestinraiinng refinement of latent reasoning in large language models, 2025b.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- ery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv: ///71, 2022.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, and al. Chain-of-thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201. 1903, 2022.
David Wingate, Mohammad Shoeybi, and Taylor Sorensen. Prompt compression and contrastive conditioning for controllability and toxicity reduction in language models. arXiv preprint arXiv:2210.03166 2022.
Thomas Wolf, Lysandre Dehut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, et al. Transformers: State-of-the-an natural language processing. In Proceedings of the 2020 conference on empirical methods in natural language processing: system demonstrations, pp. 38-45, 2020.
Haoyi Wu, Zhihao Teng, and Kewei Tu. Parallel continuous chain-of-thoughi with jacobi iteration, 2025a.
Junhong Wu, Jinliang Lu, Zixuan Ren, Gangqiang Hu, Zhi Wu, Dai Dai, and Hua Wu. Lims are single-threaded reasoners: Demystifying the working mechanism of soft thinking. arXiv preprint (rXii:2508.034400 2025b.
Guangxuan Xiao, Yuandong Tian, Beidi Chen, Song Han, and Mike Lewis. Efficient streaming language models with attention sinks. arXiv preprint arXiv:2309. /7499, 2023.
Yige Xu, Xu Guo, Zhiwei Zeng, and Chunyan Miao. Softcot: Soft chain-of-thought for efficient reasoning with llms. arXiv preprint 2rXiv:2502.121::: 2025a.
Yige Xu, Xu Guo, Zhiwei Zeng, and Chunyan Miao. Softcot++: Test-time scaling with soft chain- of-thought reasoning. arXiv preprint arXiv:2505. 1484, 2025b.
Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Ad- vances in neural information processing systems, 36:11809-1182. 2023a.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference Learning Representations (ICLR), 2023b.
Yentinglin. Aime 2025 (american invitational mathematics examination 2025). Hugging Face dataset, 2025. URL
http://c//cccc//aeteeaeaeeeeeeee ittpee/s///eennnnnnnnniiiiii aime__
2025.
Xiangchi Yuan, Xiang Chen, Tong Yu, Dachuan Shi, Can Jin, Wenke Lee, and Saayan Mitra. Miti- gating forgetting between supervised and reinforeement learning yields stronger reasoners. arXiv preprint arXiv:2510.04454, 2025a.
Xiangchi Yuan, Chunhui Zhang, Zheyuan Liu, Dachuan Shi, Soroush Vosoughi, and Wenke Lee. Su- perficial self self-improved reasoners benefit from model merging. arXiv preprint 2rXiv:2503.02103, 2025b.
15

---

Published as a conference paper at ICLR 2026
Zhenrui Yue, Bowen Jin, Huimin Zeng, Honglei Zhuang, Zhen Qin, Jinsung Yoon, Lanyu Shang, Jiawei Han, and Dong Wang. Hybrid latent reasoning via reinforcement learning. arXiv preprint arXiv:2505.18454, 2025.
Boyi Zeng, Shixiang Song, Siyuan Huang. Yixuan Wang, He Li, Ziwei He, Xinbing Wang, Zhiyu Li, and Zhouhan Lin. Pretraining language models to ponder in continuous space, 2025.
Zhen Zhang, Xuehai He, Weixiang Yan, Ao Shen, Chenyang Zhao, Shuohang Wang, Yelong Shen, and Xin Eric Wang. Soft thinking: Unlocking the reasoning potential of Ilms in continuous concept space. arXiv preprint 25XX::2500515778 2025.
Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuur- mans, Claire Cui, Olivier Bousquet, Quoc Le, et al. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint 2205::2200.00625, 2022.
Hanlin Zhu, Shibo Hao, Zhiting Hu, Jiantao Jiao, Stuart Russell, and Yuandong Tian. Reason- ing by superposition: A theoretical perspective on chain of continuous thought. arXiv preprint arXiv:2505.12514 2025a.
Rui-Jie Zhu, Tianhao Peng, Tianhao Cheng, Xingwei Qu, Jinfa Huang, Dawei Zhu, Hao Wang, Kaiwen Xue, Xuanliang Zhang, Yong Shan, et al. A survey on latent reasoning. arXiv preprint arXiv2250..06003, 2025b.
16

---

Published as a conference paper at ICLR 2026
A USE OF LLMS DISCLOSURE
We employed GPT-5 (OpenAl, 2025a) from OpenAl to assist with language polishing in order improve the readability of the paper. We affirm that large language models were not misused intentionally in any part of this work. All intellectual contributions are attributed to the human authors, and the results presented in this paper are entirely the product of human research efforts.
B SUPPLEMENTARY DETAILS
B. SWIREASONING IMPLEMENTATION
Algorithm SWIREASONING
Input: Question 2i model M, max steps maxx coefficient co, coefficient (6, dwell window WE-L- max switches Cmax and answer budget B
Output: Answer 91:m
Init: Mode mp +- Latent, switch counter C +- O, injection queue Q 0, budget flag b -1
for = 1 to Tmax do
la < M M(11=-1);
Pt softmax(4)): H
H E. pelv] log Px(v]
II Entropy
4
if Q + 0 then
I1 Token injection (comerggenceehermnnttie prefix)
24 < Q-pop()
6
if = 0 then
break
8
if b > 0 then b- b-1
10
continue
11
if 1 then
12
H < H1; + 0
13
if m4-1 1 Latent and H, < H then
Mode switching (Sec. 3.3)
14
m, <- Explicit; H +- H3; At +- 0: C-C+1
15
else m,-1 = Explicit and He > and At 2 WE-L then
16
m. +- Latent; H + Hii At at+0
17
else
18
m. < m4-1; At 1++1
19
if m, Explicit and 1. S C S Cmax then
Switch count control (Sec. 3.4)
20
Q - [/thhnnnk]]]
If Convergence trigger
21
else m, Explicit and C Cmax then
22
Q []] //th///)//////////// n The final answer is"]]; b+ B
If Termination trigger
if m4 Explicit and At then
24
2t <- arg max Pe [v] or Sampling
25
else
26
ē: Pe(j E[u]
27
m, Latent and At then
I1 Thinking-related signal mixing
28
a₁ ag (1 an) T Janx
29
a₃ ēt (1 at ((thnk))
30
if Explicit and Al then
II "Tinkingg-elateer signal mixing
3 8o + (1 (13)) T
32
2, + 3 é + (1 3x) (/thinkk)
33
24 < &
If Soft embeddings feed as inputs
34
if x; = <EOS> then
35
break
36
Extract answer from In+ld
37 return
Alg. provides a detailed implementation SWIREASONING, where the implementation for mode switching is written in black and switch count control for token efficiency is outlined in blue.
17

---

Published as a conference paper at ICLR 2026
B.2 BENCHMARK DETAILS
We conduct evaluations on 11 reasoning benchmarks, including GSM8K (Cobbe et al., 2021), Math500 (Hendrycks et al., 2021), AIME 2024 (HugginggFacell4, 2024), AIME 2025 (Yentinglin, 2025) for mathematical reasoning; GPQA Diamond (Rein et al., 2024) for STEM reasoning: Hu- manEval (Chen et al., 2021), LeetCode-Conter
(Guo et al., 2024). MBPP (Austin al., 2021), LiveCodeBench (Jain et al., 2024) for coding reasoning; and 2WikiMultihoQAA (Ho et al., 2020), Commonsense@A (Talmor et al., 2019) for general reasoning.
GSM8K: We use the test set of 1,319 grade-school math word problems, designed to evalu- ate multi-step arithmetic reasoning in natural language. : https: htttsst/hugggiaaii htt.g/uggggggc....eoo/// co / datasets htt///ppnaa/////ea ggmök.
MATH500: curated set 500 problems from the MATH dataset, covering diverse high- school competition-level mathematics domains such as algebra, geometry. and number the- ory. https:///ugge ingfacc....... co/ /o tasets //uggiigggi ccH4 / /MATH--HSS TH- 500.
AIME 2024: Contains 30 problems from the 2024 American Invitational Mathematics Ex- amination, each requiring a concise numeric answer and reflecting competition-level dif- ficulty. https:/ htttps:://ugggaai httpg//ggggggc.cee///// co dataaatassssseeee HHgggggggggggleiiiieeeee aimm__ 2024.
AIME 2025: Contains 30 problems from the 2025 American Invitational Mathematics Examination, continuing the focus on competition-styll math reasoning with challeng- ing questions that test symbolic and logical skills. @ https: / ht/hhhgginngg iace face co/
datasets/yente in ingniig in/ a aime_ mm22_225..
GPQA Diamond: A high-quality subset of about 198 carefully verified questions, focusing on STEM disciplines including mathematics, physics, chemistry, biology, and computer science, designed evaluate expert-level factual knowledge and reasoning ability.
/ttps://huggingge wwwaace.....eeeeeeeeeeeeeeeeeeee sa//atasett httpt/s//henddddn////eeeeeeee
HumanEval: A set of 164 hand-written Python programming problems. Each problem provides a function signature and short docstring specification, and models are evalu- ated by executing generated code against unit tests.
@: bettps://huggingee
co/
datasets/opente oopeaai...... hhumaneval..
LeetCode-Contest: A benchmark for evaluating code LLMs, consisting of 180 algorithm problems of different difficulties from LeetCode contests. The evaluation is based on whether the generated solution passes the provided tests. @: htttps: h/tgg//gggggge.e.. co/dataseto Tech LeetCode- Conttst..
MBPP: We use the test set of 257 Python programming problems. Each problem includes natural language prompt, reference solution, and unit tests. The model is scored by running generated code on the tests. @ https:/ https://huggingg htt.ggggggagaccee.e///o/// co datasets/ pocgle-researe sescchddattttts/nnnnn
LiveCodeBench:
The dataset contains 279 problems, and the prompt instructs the model generate a correct Python program that matches the specification and passes all tests. @ https: htap:s:///hgggnggaggee LiveCodeBench-v5.
co dataaatss Primme http///iimmeeeeeeeeeeeeeee Intellect/
2WikiMultihopQA: We use the 2WikiMulihopQAA set with 200 problems from Long- Bench (Bai et al., 2024). The problems require combining evidence from multiple Wikipedia documents to reach the final answer. It tests cross-document reasoning rather than single passage lookup.
https:/ http:::////ugginggaacc. co/ ca/datsstss/////
zai SeilorrgLLongeene
We use the test set of 1,221 multiple-choice QA questions that require commonsense knowledge. Questions are constructed from ConceptNet relations and each question comes with one correct answer and several distractor options. https:/ huggingfer co /ataaaatttttttaeeeeeeeee htatttaa////oooooo/eee /t/c//oooon//e/ee soommnnsense_gaa
CommonsenseeA::
To provide LLMs with sufficient thinking space, following the same settings as Qwen3 (Yang et al., 2025), set the maximum generation length to 32,768 tokens for GSM8K, Math500, GPQA Dia- mond, HumanEval, LeetCode- Contest, MBPP, LiveCodeBench, 2WikiMultihoQAA, and Common- senseQA henchmarks, and 38,912 tokens for AIME 2024 and AIME 2025 benchmarks.
18

---

Published as a conference paper at ICLR 2026
We repeat the experiments eight times and report the average accuracy for both SWIREASONING and other baselines on the AIME 2024 and AIME 2025 benchmarks.
B.3 BEST PRACTICE FOR HYPERPARAMETER
Table 6: Hyperparameters for mode switching across datasets and models. W and Bo are fixed across all scenarios, while provides users with flexibility for adjustment depending the task.
Model
Hyperparameter Dataset
Qwen3-1.7B Qwen3-8B Qwen3-32B DeepSeek-Re RR--sstill- Distill-Llama-88
GSM8K
MATH500
AIME2024
AIME2025
GPQA Diamond
W (window size)
HumanEval
512 (fixed for all)
LeetCode-Contest
MBPP
LiveCodeBench
2WikiMultihopOA
CommonsenseeA
GSM8K
0.6
0.5
0.5
0.1
MATH500
0.5
1.0
0.5
0.5
AIME2024
0.5
0.9
1.0
0.65
AIME2025
0.3
0.9
0.9
0,7
GPQA Diamond
1.0
1.0
0.6
0.7
og (user-exposed)
HumanEval
0.5
LeetCode-Contest
1.0
MBPP
0.9
LiveCodeBench
0.9
2WikiMultihopOA
0.7
Commonsense(A
0.9
GSM8K
MATH500
AIME2024
AIME2025
GPQA Diamond
HumanEval
0.7 (fixed for all)
LeetCode-Coo Contest
MBPP
LiveCodeBench
2WikiMultihopQA
CommonsenseQA
In addition to Tab. 6, SWIREASONING operates as a straightforward and instant substitution for the model. -ene interface of Huggingface's transformers (Wolf et al., 2020) package. There are no model parameters or architecture changes, and no training was used in the experiments. For sampling-related hyperparameters and prompt templates, and DeepSeck-A R1's technical report (Yang et al., 2025; Guo et al., 2025) without modification.
the ones recommended by Qwen3
B.4 BROADER RELATED WORK
Efficient LLM Reasoning. In terms of improving reasoning efficiency, there are broader tech- niques including but not limited to KV cache compression (Han et al., 2023; Xiao et al., 2023; Cai et al., 2024; Shi al., 2025), prompt token compression (Wingate et al., 2022; Jiang et al., 2023; Pan et al., 2024: Shi et al., 2024), speculative decoding (Leviathan et al., 2023; Kim et al., 2023; Liu et al., 2023; Sun et al., 2023; Chen et al., 2024). traditional methods such quantization, pruning, distillation (Shi et al., 2023; Lin et al., 2024; Fu et al., 2024; Yuan et al., 2025b;a), and system-level optimizations such
(Dao al., 2022; Dao, 2023; Shah al., 2024). Learning-
19

---

Published as a conference paper at ICLR 2026
based generation methods, such as COCONUT (Hao et al., 2024) based supervised fine-tuning, and Long&Short (Ning et al., 2025) based on reinforcement learning, shorten the reasoning process by involving additional training.
SWIREASONING, however, targets a different axis of efficiency and is not aiming to surpass them. Instead, it saves tokens by dynamically alternating between latent steps and explicit steps and limit- ing the number of block switches. As such, is plug-and-pll play during inference and can be layered on top of the aforementioned techniques for multiplicative gains.
C SUPPLEMENTARY EXPERIMENTS
C.1 QUANTITATIVE ANALYSES OF EXPLORATION IN LATENT AND EXPLICIT MODES
(a) Concentration of token distributions
[b] Semantic similarity of tokan candidates
1.0
Average entroy
0.8
Average mitorit
0.6
0.15
0.4
0.10
0.05
0.0
100
200
300
500
100
300
500
Sample index
Semple index
Figure 6: Latent modes enable exploration with less concentrated token distributions and more semantically diverse candidates. In latent modes: (a) the average entropy of the next-token distributions indicates less con- centration, and (b) the average pairwise similarity among token candidates indicates more semantic diversity.
To quantify exploration in the latent and explicit modes, we analyze two complementary statistics with the Qwen3-88 8B model on Math500 as visualized in Fig. 6. First, to measure the differences in the concentration level of the next-token distributions between latent and explicit modes, we calculate the average Shannon entropy across steps for each sample. Fig. (a) shows that the entropy in latent mode 2.08x higher than in explicit mode on average (0.444 vs. 0.213), indicating that probability mass is spread across more candidate tokens instead of collapsing in latent mode.
Entropy alone does not guarantee semantic diversity, as the semantic distances of different token candidates can be close, even when the probability mass is spread across them. To measure the dif- ferences in the semantic similarity among token candidates between latent and explicit modes, we calculate the average pairwise cosine similarity of top-5 token candidates embeddings for each sample. Fig. (b) shows that the semantic similarity in latent mode is 63.8% of that in explicit mode average (0.118 0.185), showing that token candidates considered under latent modes are more semantically diverse. Taken together, these results quantitatively sub- stantiate that latent modes enable exploration with less concentrated token distributions and more semantically diverse candidates.
steps by using their
C.2 EFFICIENCY MEASURED IN TERMS OF LATENCY AND TFLOPS
We evaluate efficiency in terms of average wall-clock latency and TFLOPs over samples on MATH500 with Qwen3-8B, comparing CoT, CoT (Greedy), Soft Thinking, and SWIREASONING. Fig. 7 demonstrates that SWIREASONING consistently reaches a given accuracy with lower latency and fewer TFLOPs, and the advantage grows as the budget becomes tighter. Specifically, at 90% pass @1 1 accuracy, SWIREASONING uses 1.36x fewer TFLOPs and is 1.36x faster in wall-clock time than COT. At 80% accuracy, uses 2.18x fewer TFLOPs and 2.17x faster. These results confirm that SWIREASONING improves efficiency at fixed accuracies. especially under small budgets.
C.3 STATISTICS OF SWITCH COUNTS ACROSS PROBLEM DIFFICULTIES
To investigate the switching dynamics, we analyzed the distribution of switch counts across five benchmarks of varying difficulty. Fig. presents the box plots of switch counts for Qwen3-1.7B,
20

---

Published as a conference paper at ICLR 2026
100
Acc. 20%
300
20%6
Abc 50%
ADC 30%
2274
Accurscy 380
1,00x foweer
60
60
2,18
CCT
CoT
30
Sun Theeing
Theaao (OunC
Swth (0ung
20
20
Figure 7: Left: Accuracy vs. wall-clock latency. Right: Accuracy vs. TFLOPs. Dashed horizontal lines mark equal-accuracy targets (80%, 90%, 95%). Lower values on the x-axes are better.
Mocel
140
Owen331177 70
Owen3-45
Qwen 344
121
123
130
Average switch wotth courts
100
75
60-
20
OPAA Dammnd
AUPE 2034
AUFIC 2025
COMSE
OPgA Damond
20M
AUrE 2025
Hot 200
aPd niamord
XUME 2004
JUNE 1000
Figure 8: Statistics of switch counts across varying problem difficulties and model scales. The box plots show the distribution of the number of switches for Qwen3-1.7B. Qwen3-8B, and Qwen3-32B on five benchmarks.
Owen3-8B, and Qwen3-32B. The switching frequency demonstrates adaptability to problem diffi- culty. On benchmarks requiring complex reasoning such as AIME 2024 and AIME 2025, the switch counts increase substantially, with medians ranging from 44 69, and maximum counts reach 151. The results confirm that SwiReasoning functions as difficulty-awar reasoning approach.
C.4 ANALYSIS CHALLENGES IN SPECIFIC SCENARIOS
3D Surface Shortest Path
Question: A gecko is in room that is 12 feet long, 10 feet wide and 8 feet tall. The gecko is currently on side wall (10' by 8'), one foot from the ceiling and one foot from the back wall (12' by 8'). The gecko spots a fly on the opposite side wall, one foot from the floor and one foot from the front wall. What is the length of the shortest path the gecko can take reach the fly assuming that it does not jump and can only walk across the ceiling and the walls? Express your answer in simplest radical form.
Output:
To determine the shortest path the gecko can take to reach the fly, we consider the constraints:
The gecko is side wall (10' by 8') of 12' 10 8' room.
The fly is on the opposite side wall.
The gecko is one foot from the ceiling and one foot from the back wall.
The fly is one foot from the floor and one foot from the front wall.
21

---

Published as a conference paper at ICLR 2026
The gecko can only walk across the ceiling and walls, not the floor.
STEP 1: ASSIGN COORDINATES
Let's define the room in 3D coordinates:
The length (along the x-axis) 12 feet.
The width (along the y-axis) is 10 feet.
The height (along the z-axis) is 8 feet.
Assume:
The gecko is the left wall (x O), one foot from the back wall (y 1), and foot from the ceiling = 7). So its position is:
(x = 0, =
The fly is the right wall (x 12), foot from the front wall (y 9), and one foot from the floor (z 1). So its position is:
(x = 12, 9, 1)
STEP 2:
CONSIDER THE SHORTEST PATH
The gecko can only walk across the ceiling and walls. The shortest path will involve un- folding the room into flat 2D plane such that the gecko's path becomes straight line. common and effective approach is unfold the left wall, ceiling, and right wall into a single plane. This allows the gecko to move from its position on the left wall, across the ceiling, to the fly on the right wall.
STEP 3: TRANSFORM TO A 2D COORDINATE SYSTEM In this unfolded net:
The left wall is 10 feet wide (y-axis) and 8 feet tall (z-axis).
The ceiling is 12 feet long (x-axis) and 10 feet wide (y-axis).
The right wall is 10 feet wide (y-axis) and feet tall (z-axis).
When we unfold the left wall, ceiling, and right wall into one plane, the gecko's position and the fly's position can be mapped in this 2D plane. In this 2D representation:
The gecko's 's position is
=. the left wall.
TThe fly's position is at (y 9, = on the right wall.
However, since the gecko is the left wall, and the fly is the right wall, and the path goes across the ceiling, we can consider the horizontal distance as the length of the room (12 feet), and the vertical distance as the difference in heights from the gecko the ceiling (1 foot) and from the ceiling to the fly (7 feet), totaling feet.
So, the total path forms right triangle with: [Incorrect reasoning trace here that does adhere to "only walk across the ceiling and the walls" constraint]
One leg: 12 feet (length of the room)
The other leg: feet (vertical distance from gecko to ceiling + ceiling to fly)
STEP 4: COMPUTE THE SHORTEST DISTANCE
Using the Pythagorean theorem:
Distance V122 82
V144
64 =
208
V16
13
4V13
22

---

Published as a conference paper at ICLR 2026
FINAL ANSWER
4v13
Correct Answer: 2/113
While SWTREASONING demonstrates robust generalization across diverse domains, we identify a specific category of tasks, spatial reasoning with rigid topological constraints, which may be chal- lenging for latent-based methods. A representative case is the "3D Surface Shortest Path" problem from the Math500 benchmark shown above. In this problem, a gecko must find the shortest path on 3D room's 's surface. The reasoning trace reveals that SWIREASONING correctly identifies the high-level strategy: unfolding the 3D room into 2D plane. However, struggles to maintain the specific topological "only walk across the ceiling and the walls" constraint during latent transitions.
We attribute this behavior trade-off inherent latent reasoning:
Smoothing vs. Precision: Latent reasoning operates by smoothing probability distributions to enable diverse semantic exploration. While this is beneficial for logical reasoning, may be detrimental for tasks with precise constraints, such rigid geometric topology (e.g., defining exactly which wall edge connects to which).
Blurring Constraints: During the latent mode, the strict "only walk across the ceiling and the walls" constraint is blurred due to the probability mass spreading. As result, when the model switches back explicit mode, it may reason with an invalid path (e.g., calculating direct Eu- clidean distance through prohibited space), leading incorrect answer (4v13 vs. true 213)...
Overall, this suggests that while SWIREASONING excels at semantic and logical exploration, a few tasks requiring rigid geometric constraint satisfaction, where fuzzy latent exploration may interfere with precise geometric execution, could still be challenging.
C.5 ABLATION STUDY ON THE CONVERGENCE TRIGGERS W.R.T. TERMINATION ONES
Table 7:
Ablation study on the relative position of convergence and termination triggers. The convergence trigger is set different fractions of the termination count T (where T == Cmax). The results demonstrate that the default TT' offers balanced choice, avoiding the significant accuracy loss of earlier triggers while being more efficient than later ones.
Convergence w.r.t. Termination
C ==
T
C= JT
C
C=
C
C=
T
C
T
Cmax = 8
Accuracy (%) Generation Length
76.60 1428
79.20
81.00
84.20 2743
84.20
1817
2222
2743
Cmax 20
Accuracy (%) Generation Length
84.00
85.80
87.80 3597
88.20 4002
88.80
2788
3278
3943
To analyze the positions of convergence triggers w.r.t. termination ones, we conducted an abla- tion study using Qwen3-1.7B on the Math500 benchmark. We fixed the termination trigger two representative budgets (Cina = and Cuax 20) and varied the position of the convergence trigger relative to the termination step. Specifically, if T denotes the switch count termination (i.e., T = Cmax)), we tested setting the convergence trigger at E ( IT. presents clear trade-off between accuracy and token efficiency:
2T. T Tab.
3
Early convergence (TT, 42); triggering convergence too early significantly reduces generation length but leads notable drop accuracy (e.g., at Cmax 20, accuracy drops from 87.80% to 84.00% when moving from 7 to 177). This suggests that forcing the model to converge before sufficient exploration prevents it from finding correct solutions for harder problems.
Late convergence T2,, T7) delaying the convergence trigger yields marginal accuracy gains (e.g., +1 +1.00 +1.0% from 1 to 3T Cmax 20) but comes at the cost of increased token consumption.
23

---

Published as a conference paper at ICLR 2026
Balanced default T)::: the default setting of 2/2 3mmax effectively strikes balance. It captures the majority of the accuracy while maintaining reasonable token efficiency. Overall, C 2 serves as sweet spot for general-purpose reasoning under budget constraints.
C.6 ABLATION STUDY ISOLATING THE EFFECT LATENT UNDER SMALL BUDGETS
Table 8: Ablation study on isolating the effect latent under small token budgets. "SwiR w/o latent" retains the switch count controller but disables latent mode.
MATH 500
GPQA Diamond
AIME 2024
AIME 2025
GSM8K
Average
Method
Accurucy (%)
SwiR w/o latent 85.44
68.60
26.26
2.50
0.42
36.64
+0.00
SwiR
86.80
+1.36
72.40
+3.80
29.80
+3.54
5.83
+3.33
4.58
+4.16
39.88
+3.24
Generation Length
SwiR w/o latent
816
1207
1032
835
791
936
+0000
SwiR
816
1189
933
812
787
-3.10%
To further evaluate the effect of latent modes at small budgets, we used Qwen3-1.7B and compared the full SWIREASONING against variant where latent reasoning is disabled (SwiR w/o latent). In the "w/o latent" setting, the model is forced to remain in explicit mode, but retains the same switch count controller (functioning as step-limit controller for inserting the proposed convergence and termination triggers). We evaluated both methods under small token budgets.
Tab. shows that disabling latent leads consistent degradation performance across all bench- marks. Under lower or the same token usage, the full SwiReasoning achieves an average accuracy improvement of +3.24% over the ablation. On challenging benchmarks like AIME2025, the con- tribution of latent is particularly pronounced (+4.16%). These results show that the latent itself provides gains that cannot attributed to token controller mechanisms alone.
C.7 SUPPLEMENTARY COMPONENT- WISE ABLATION STUDY
Table Supplementary ablation studies on switch window mechanisms (including window size, symmetry vs. asymmetry) and signal mixing.
Method
GSM8K
Math500
Average
SwiR w/ window size 0
89.00%
89.20%
89.10%
SwiR w/ symmetry window size 2
89.31%
90.00%
89.66%
SwiR w/ symmetry window size 4
90.83%
89.20%
90.02%
SwiR w/ symmetry window size 8
90.60%
91.20%
90.90%
SwiR w/ symmetry window size
88.93%
92.00%
90.47%
SwiR w/ symmetry window size 32
90.14%
91.00%
90.57%
SwiR w/ asymmetry window size 64
89.69%
92.60%
91.15%
SwiR w/ asymmetry window size 128
90.45%
91.00%
90.72%
SwiR w/ asymmetry window size 256
89.76%
90.80%
90.28%
SwiR w/ asymmetry window size 512
90.83%
93.00%
91.92%
SwiR asymmetry window size 1024
90.83%
91.20%
91.01%
SwiR w/o mixing
90.44%
92.20%
91.32%
To quantify the marginal contribution each proposed mechanism, we conducted a component- wise ablation study using Qwen3-1.7B GSM8K and MATH500 benchmarks. We compared the default asymmetric window W = 512, W = 0) against variations with no window (W and symmetric windows of varying sizes. Results in Tab. 9 reveal that
24

---

Published as a conference paper at ICLR 2026
Necessity of window: Removing the window constraint completely leads to significant per- formance drop (Avg: 89.10% vs. 91.92%), due to rapid oscillations between modes preventing coherent reasoning chains from forming.
Benefit of asymmetry: The default asymmetric setting outperforms all symmetric configurations. While symmetric windows (e.g., size 8) improve over no window, they fail to match the peak per- formance of the asymmetric design. This validates our hypothesis that explicit reasoning requires a consolidation period (WE+L), while latent reasoning benefits from immediate exit upon confi- dence recovery (WL+E to reduce the risks of introducing spurious signals that may mislead the model.
Tab. also indicates that disabling the signal mixing results in a consistent accuracy decrease (Avg: 91.32% vs. 91.92%). This suggests that the signal mixing helps the model better orient its internal state during mode transitions and contributes positively to the final accuracy.
Table 10:
Supplementary ablation study on the maximum switch count Caxx. We evaluate the trade-off tween accuracy and generation length by varying Cmaz from unlimited.
Cmax
8
12
16
20
24
Accuracy
72.40%
81.00%
85.00%
86.80% 3280
87.80% 3597
87.80%
Generation Length
1189
2222
2914
3862
Cmax
28
32
36
40
44
Accuracy
89.00%
89.20%
99.80%
90.60%
91.80%
93.00%
Generation Length
4068
4186
4274
4318
4602
4924
We analyzed the effect of the switch count cap (Cmaz) on accuracy and generation length using Qwen3-1.7B on MATH500 in Tab. 10. The results indicate that
Efficiency control: As Cmas decreases (from oo down to 4), the generation length drops signifi-
cantly (from 4924 1 1189), demonstrating the controller's effectiveness curbing overthinking. This provides users with sufficient capabilities to control the intensity of thinking flexibly.
Accuracy trade-off: While extremely tight caps (e.g., 4 or 8) limit accuracy, moderate caps (e.g., 20-32) achieve performance competitive with close the unlimited setting (Ciaar oo) but with much lower token consumption. This confirms that the controller effectively makes use of partial reasoning trajectories to deliver reasonable answers.
C.8 DETAILED EVALUATION RESULTS UNDER VARYING TOKEN BUDGETS
We provide detailed evaluation results of Qwen3-8B in Tab. 11-15, Qwen3-1.78 in Tab. 16-20, and DeepSeck-RR L-Distill-Llama Tab. 21-25.
25

---

Published as a conference paper at ICLR 2026
Table 11; Evaluation results of Qwen3-8B on the GSM8K benchmark under varying token budgets. Rows sorted by generation length in descending order.
Method
Accuracy (%)
Generation Length
(Greedy)
SwiR
(Greedy)
CoT
95.60%
Col
CoT
94.77%
2123
CoT (Greedy)
95.15%
2115
Soft Thinking
95.38%
2073
Soft Thinking
95.07%
2033
Coll (Greedy)
92.65%
1934
CoT
91.81%
1926
SwiR (Ours)
94.84%
1879
Soft Thinking
92.12%
1865
SwiR (Ours)
95.14%
1761
SwiR (Ours)
94.391%
1585
CoT
79.90%
1553
CoT (Greody)
79.68%
1340
Soft Thinking
80.14%
1526
SwiR (Ours)
94.47%
1297
CoT
44.50%
990
CoT (Greedy)
45.791%
988
Soft Thinking
47.08%
988
SwiR (Ours)
93.70%
844
25.47%
(Groody)
25 .3.9%%%
Thinking
6.36%
(Greedy)
Thinking
6.22%
Table Evaluation results Qwen3-8B MATH150 benchmark under varying token budgets. Rows are sorted by generation length descending order.
Method
Accuracy (%)
Generation Length
CoT (Greedy)
96.40%
5311
SwiR (Ours)
98.40%
5183
C6T
96.00%
4985
Soft Thinking
96.00%
4934
Solt Thinking
95.40%
4733
CoT
95.60%
4729
(Greddy)
94.00%
SwiR (Ours)
95.80%
4266
SwiR
93.
87.
(Greedy)
Thinking
SwiR (Ours)
93.00%
SwiR (Ouns)
90.20%
3164
CoT
72.40%
CoT (Greedy)
72.80%
2890
Soft Thinking
70.20%
2865
SwiR (Ours)
85.80'
2387
CoT
46.20%
1922
Soft Thinking
44.80%
1898
CoT (Greedy)
43.00%
1873
SwiR (Ours)
78.40%
1368
CoT
24.20%
1024
Soft Thinking
25.00%S
1024
CoT (Greedy)
22.20%
1023
26

---

Published as a conference paper at ICLR 2026
Table 13: Evaluation results of Qwen3-8B on the GPQA Diamond benchmark under varying token budgets. Rows are sorted by generation length descending order.
Method
Accuracy (%)
Generation Length
SwiR (Ours)
61.11%
8359
Soft Thinking
59.59%
8153
Col
59.60%
8123
CoT (Greedy)
56.57%
7909
55.50%
(Greddy)
55.05%
7546
Thinking
55.05%
(Ours)
58
SwiR
57.07%
SwiR
CoT (Greedy)
33.84%
Soft Thinking
34.85%
4961
SwiR (Ours)
55.05%
4766
SwiR (Ours)
53.54
3603
Col (Greedy)
12.12%
3078
Solt Thinking
12.63'%
2959
CoT
10.61%
2861
SwiR (Ours)
46.96%
2117
Soft Thinking
2.53%
1753
CoT
2.52%
1743
CoT (Greedyy
2.02%
1723
SwiR (Ours)
47. 47%
1.27
CoT
0.00%
1024
Coll (Greedy)
0.00%
1024
Soft Thinking
0.00%
1024
SwiR (Oues)
39.39%
867
C6T
0.00%
512
Col (Greedy)
0.00%
512
Solt Thinking
0.00%
512
27

---

Published as a conference paper at ICLR 2026
Table Evaluation results Qwen3-8B on the AIME2024 benchmark under varying token budgets. Rows are sorted by generation length descending order.
Method
Accuracy (%)
Generation Length
SwiR (Ours)
79.17%
12491
Soft Thinking
67.92%
12271
CoT
75.83%
12077
CoT (Greedy)
70.00%
11680
SwiR (Ours)
10815
SwiR (Ours)
66.25%
10349
(Greedy)
Soft Thinking
CoT
(Ours)
SwiR (Ours)
Soft Thinking
CoT
38.75 18.75%
7109
CoT (Greedy)
36.67%
7033
SwiR (Ours)
45.42%
6093
Col
20.83%
4096
Soft Thinking
23.33%
4096
CoT (Greody)
26.67%
4056
SwiR (Ours)
25.42%
3589
CoT
5.83%
2048
CoT (Greedy)
10.00%
2048
Soft Thinking
3.75%
2048
SwiR (Ours)
12.08%%
1809
CoT
1.67%
1024
CoTl (Greedy
3.333%
1024
Soft Thinking
3.33%
1024
SwiR (Ours)
6.67%
818
CaT
0.83%
CoT (Greedy)
0.00%
512
Sont Thinking
3,33%
512
28

---

Published as a conference paper at ICLR 2026
Table Evaluation results Qwen3-8B on the AIME2025 benchmark under varying token budgets. Rows are sorted by generation length descending order.
Method
Accuracy (%)
Generation Length
SwiR (Ours)
70.00%
13911
Soft Thinking
68.33%
13665
CoT (Greedy)
60.00%
13292
CoT
67.50%
12924
SwiR (Ours)
(Ours)
58.33%
10596
Soft Thinking
(Ours)
CoT (Greedy)
SwiR (Ours)
Coll
Soft Thinking
36.25%
6772
CoT (Greedy)
33.333
6768
SwiR (Ours)
34.58%
6243
Col (Greedy)
13.33%
4096
CoT
13.339
4091
Soft Thinking
14.17%
4060
SwiR (Ouns)
21.67%
3608
CoT
7.50%
2048
CoT (Greedy)
6.67%
2048
Soft Thinking
6.25%
2048
SwiR (Ours)
11.25%
1999
CoT
1.67%
1024
CoTl (Greedy
0.00%
1024
Soft Thinking
3.33%
1024
SwiR (Ours)
6.67%
722
CaT
2.50%
512
CoT (Greedy)
0.00%
512
Soft Thinking
3,33%
512
29

---

Published as a conference paper at ICLR 2026
Table 16: Evaluation results of Qwen3-1.7B the GSM8K benchmark under varying token budgets. Rows are sorted by generation length descending order.
Method
Accuracy (%)
Generation Length
(Greedy)
SwiR (Ours)
90.8318
CoT (Greedy)
89.61%
Thinking
Soft Thinking
90.22%
1946
CoT
89.23%
Soft Thinking
89.841
1896
CoT (Greedy)
89.31%
1895
CoT
86.35%
1753
CoT (Greody)
87.64%
1744
Soft Thinking
87.49%
1736
SwiR (Ours)
89.23%
1695
SwiR (Ouns)
89.46%
1621
SwiR (Ours)
87.951%
1462
CoT
76.65%
1420
Soft Thinking
78.92%
1418
CoTl (Greedy)
78.17%
1407
SwiR (Ours)
88. 38%
1229
CoT
50.57%
967
CoT (Greedy)
48.52%
959
Soft Thinking
50.95%
958
SwiR (Ours)
86.80'
816
29
(Groody)
30.02%
Thinking
31.54%
82.26%
7.96%
(Greedy)
8.34%
Thinking
9.25%
Table Evaluation results of Qwen3-1.7B on the MATH1500 benchmark under varying token budgets. Rows are sorted by generation length descending order.
Method
Accuracy (%)
Generation Length
SwiR (Ours)
93.00%
4924
CoT (Greody)
91.00%
4799
CoT
92.00%
4780
Soft Thinking
90.50%
4721
Col
90.80'
4435
CoT (Greedy)
89.20%
4342
Thinking
89.00%
4288
SwiR (Ours)
87.80%
3862
(Greddy)
83
Thinking
(Ours)
SwiR (Ours)
SwiR (Ouns)
85.00%
2914
CoT
68.40%
2761
Soft Thinking
69.40%
2744
CoT (Greedyy
69.20%
2738
SwiR (Ours)
81 .0.00%
2222
CoT
46.40%
1857
CoT (Groody)
47.80%
1850
Soft Thinking
46.20%
1830
SwiR (Ouns)
72.40%
1889
CoT
25. 60%
1022
CoT (Greedyy
29.80%
1020
Soft Thinking
27.20%
1020
30

---

Published as a conference paper at ICLR 2026
Table 18: Evaluation results of Qwen3-1.7B on the GPQA Diamond benchmark under varying token budgets. Rows are sorted by generation length in descending order.
Method
Accuracy (%)
Generation Length
SwiR (Ours)
41.41%
9517
CoT (Greedyy
31.82%
9190
CoT (Greedy)
30.30%
8751
Solt Thinking
34.34%
8731
CoT
39.39%
8625
CoT
37.37%
8040
SwiR (Ours)
36.87%
7773
Solt Thinking
32.32%
7447
SwiR (Ours)
35.351
6792
SwiR (Ours)
37.88%
5856
SwiR (Ours)
36.87%
4766
CoT
17.17%
4758
Soft Thinking
16.67%
4463
CoT (Greedy)
12.12%
3770
SwiR (Ours)
37.37%
3497
Soft Thinking
8.08%
2915
Col
8.59%
2843
(Greddy)
7.58%
31.319
SwiR
27.27%
(Greedy)
2.53%
Thinking
0.51%
0.00%
CoT (Gready)
0.00%
1024
Soft Thinking
0.00%
1024
SwiR (Ours)
29.80%
933
Table 19: Evaluation results of Qwen3-1.7B on the AIME2024 benchmark under varying token budgets. Rows are sorted by generation length in descending order.
Method
Accuracy (%)
Length
Col (Greedy)
40.00%
12825
Swilt (Ours)
50.83%
12702
CoT
45.83%
11896
Soft Thinking
38.75%
10788
SwiR (Ours)
42.08%
10243
Soft Thinking
36.67%
9841
CoT
38.33%
9510
CoT (Greedy)
33.33%
9377
SwiRt (Ours)
38.75%
9350
SwiR (Ouns)
36.25%
8634
Soft Thinking
30.00%
7498
CoT (Greedy)
23.33%
7302
SwiR (Ours)
17.177%
7084
CoT
27.50%
SwiRt (Ours)
42%
$926
CoT (Greody)
10.00%
4096
Soft Thinking
16.67%
4096
SwiRt (Ours)
14.17%
CoT
CoT (Gready)
Thinking
SwiR (Ours)
7.50%
1887
CoT
1.25%
1024
CoT (Greedy)
0.00%
1024
Soñt Thinking
0.00%
1024
SwiR (Ours)
5.83%
812
31

---

Published as a conference paper at ICLR 2026
Table 20: Evaluation results of Qwen3-1.7B on the AIME2025 benchmark under varying token budgets. Rows sorted by generation length descending order.
Method
Accuracy (%)
Generation Length
CoT (Groedy)
33.33%
11408
SwiR (Ours)
38.33%
9944
CoT
33.33%
9733
Soft Thinking
36,67%
8904
Col (Greedyy
26.67%
8890
CoT
31.25%
8618
SwiR (Ours)
34.58%
8543
SwiR (Ours)
32.92%
81
Soft Thinking
33.33%
76.30
SwiR (Ours)
32.08%
7563
SwiR (Ouns)
30.83%
6761
Col
20.83%
6071
CoT (Gready)
20.00%
608
Soft Thinking
23. 23.33%
$738
SwiR (Ours)
.2%4
5145
CoT
14.17%
4096
CoT (Greedy
13.33%
3927
Soft Thinking
20.00%
3737
(Ours)
3311
(Greedy)
Soft Thinking
(Ours)
CoT
2.08%
Col (Gready)
Soft Thinking
0.00%
1024
SwiR (Ours)
4.58%
787
Table 21: Evaluation results of DeepSeek- R1 1--istill-Lllmm- 8B on the GSM8K benchmark under varying token budgets. Rows are sorted by generation length in descending order.
Method
Accuracy (%)
Generation Length
Soft Thinking
85.90%
2953
Soft Thinking
84.84%
2516
CoT (Gready)
85.826
2393
Solt Thinking
83.621
2266
Solt Thinking
77.41%
1741
CoT (Greody)
86.05%
1642
CoT
89.46%
1588
SwiR (Ouns)
90.07%
1665
CoT
88. 10%
1554
CoT
88.02%
1491
CoT (Greedy)
85.82%
1421
Col
86.13%
1404
SwiR (Ours)
88.55%
l349
SwiR (Ours)
89.46%
1312
CoT (Greedy)
85.44%
1307
Soft Thinking
70.36%
1279
SwiR (Ours)
87.26/%
1217
CoT
78.62'5
1191
(Greody)
79.53%
1092
SwiR (Ours)
86.43%
1071
Thinking
49.58
53.22%
(Greedy)
57.39%
SwiR (Ours)
83.62%
CoT (Gready)
28.8 28.81%
508
Soft Thinking
28.35%
508
SwiR (Ours)
70.96%
270
CoT
5.69%
256
CoT (Greedy)
6.52%
256
Solt Thinking
5.46%
256
32

---

Published as a conference paper at ICLR 2026
Table 22: Evaluation results of DeepSeck-R1- -I---sstll---aaa---- on the Math500 benchmark under varying token budgets. Rows are sorted by generation length descending order.
Method
Accuracy (%)
Generation Length
Thinking
(Greedy)
Thinking
SwiR (Ours)
92.00%
3837
Col CoT
89. 80%
3572
Soft Thinking
75.40%
3204
CoT (Greedy)
83.20%
3203
SwiR (Ours)
89.80%
3046
SwiR (Ours)
88.00%
2931
CoT
85, 35.20%
2828
SwiR (Ours)
86.40%
2722
CoT (Greedy)
79.60%
2622
SwiR (Ouns)
86 06.00%
2462
Solt Thinking
64.40%
2396
CoT
71 11.00%
2133
CoTl (Greody)
71.20%
2081
SwiR (Ours)
79.20%
1953
Soft Thinking
42.60%
1605
CoT
47.60%
L539
CoT (Greedy)
52.60%
1500
SwiR (Ours)
68.40%
1116
Col
12.20%
512
Thinking
10.20%
(Groody)
11.80%
57.40%
Table 23: Evaluation results DeepSeek- Biieck--R---sttii A1-Distill-Llmmee -Lamm--lmm-------- on the GPQA Diamond benchmark under vary- ing token budgets. Rows are sorted by generation length in descending order.
Method
Accuracy
Generation Length
Soft Thinking
33.33%
8593
CoT
46.46%5
7591
Soft Thinking
30.81%
7507
SwiR (Ours)
47.98%
7458
Col
46.461%
7236
SwiR (Ours)
45.45%
6635
SwiR (Ours)
44.44%
6038
CoT (Greedy)
31.82%
5854
SwiR (Ours)
41.92%
CoT (Greedy)
27.78%
$230
31.31%
4943
Thinking
16.67%
4510
SwiR (Ouns)
39.
(Greody)
23.73%
SwiR
(Greedy)
Thinking
6.57%
CoT
9.60%
2314
SwiR (Ouns)
25.76%
1840
CoT
2.53%
1.995
CoT (Greedy)
3.54%
1469
Solt Thinking
3.03%
1381
CoT (Greedy)
0.50%
CoT
1.01%
855
Soft Thinking
1.01% %
785
SwiR (Ouns)
29. 80%
673
CoT
0.00%
512
CoT (Greedyy
0.00%
512
Soft Thinking
0.00%
511
33

---

Published as a conference paper at ICLR 2026
Table 24: Evaluation results of CrrpSec--R-Distil-LLiieee on the AIME2024 benchmark under varying token budgets. Rows are sorted by generation length in descending order.
Accuracy (%)
Generation Length
SwiR (Ours)
45.00%
8179
CoT
43.75%
8145
CoT (Greedy)
30.00%
7840
SwiR (Ours)
44.17%
7555
CoT
41.25%
7330
CoT (Greedy
26.66/8
7107
Soft Thinking
34.17%
6956
SwiR (Ours)
40.42%
6803
SwiR (Ours)
41.25%
6645
SwiR (Ours)
39.58%
5876
Soft Thinking
32.08%
5871
CoT
28.33/%
5086
CoT (Greedy)
16.67%
4952
Soft Thinking
27.50%
4860
SwiR (Ours)
32.50%
4757
(Groddy)
6.67%
Thinking
SwiR
23.3318
SwiR
CoT
1.67%
Col (Greedy)
3.33%
Soft Thinking
3.33%
2045
Table 25: Evaluation results of DeepSeek-RI PeepSeck-R--Dsttllle RR---istill-LAA Distill-llama8BB on the AIME2025 benchmark under varying token budgets. Rows are sorted by generation length in descending order.
Accuracy
Generation Length
Soft Thinking
20.42%
8448
SwiR (Ours)
31.25%
6827
Soft Thinking
19.17%
6824
CoT
26.25%
6583
SwiR (Ours)
30.83%
6419
CoTl (Greedy)
30.00%
6293
SwiR (Ours)
30.00%
6230
CoT
25 25.00%
$724
SwiR (Ours)
29.17%
$721
SwiR (Ours)
26.67%
$229
CoT (Greedy)
26.67%
4967
CoT (Greedy)
23.33/
4370
Solt Thinking
14.17%
4170
19.58%
SwiR (Ours)
21.25%
4035
14.17%
Thinking
SwiR
CoT (Greedy)
16.67%
3.33%
Soft Thinking
0.00%
2048
CoT (Greody)
3.33%
SwiR (Ours)
7.08%
1777
34


Reviews:
[{"review_id": "8sN7g6TXom", "reviewer": "Reviewer_hrDS", "summary": "Summary\nThis paper introduces SWIREASONING, a training-free reasoning approach that integrates latent and explicit chain-of-thought (CoT) modes. Instead of committing to token-level reasoning at every step, the method forms latent reasoning states via weighted token embeddings, allowing the model to retain uncertainty and explore multiple hidden trajectories.\nA block-wise entropy reference is used as a switching criterion: when the entropy decreases (confidence rises), the model switches from latent to explicit reasoning immediately; when entropy increases, switching from explicit back to latent requires a dwell window to prevent oscillation. To signal mode transitions more explicitly, the method injects fused embeddings of  < think > / </ think > tokens during switch steps.\nAdditionally, switch counters and termination triggers are employed to avoid unbounded “over-thinking” and encourage timely final answer production.\nExperiments show that, particularly for smaller models (≤8B) and token-budget-constrained settings, SWIREASONING delivers higher reasoning accuracy and efficiency compared to standard CoT and soft-thinking baselines. Ablation studies on signal mixing and dwell window length further support the design choices.", "questions": "1 In small budget settings, the model often does not actually enter latent mode due to the dwell window, yet achieves strong gains. Can you provide ablations where latent reasoning is disabled but the early-exit and <think> conditioning remain, to isolate the effect of latent reasoning itself?\n\n2 Token efficiency does not necessarily imply inference-time efficiency. Can you report wall-clock latency or FLOPs vs CoT and SoftThinking to demonstrate practical gains?\n\n3 The SoftThinking baseline underperforms CoT in your ≤8B experiments, whereas the original paper shows gains mainly at ≥32B scale. Can you confirm your implementation matches the original, and provide results on a ≥32B model to ensure fairness?\n\n4 Could you clarify whether your latent update uses the full vocabulary distribution or a top-k truncation (as in SoftThinking)?\n\n5 Could you provide a systematic component-wise ablation (turning each mechanism on/off) to quantify the marginal contribution of:  (i) dwell window (incl. 0) and asymmetry, (ii) < think >/< /think > mixing, (iii) switch-count cap/termination triggers。", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "1. The approach does not require model updates and can be applied directly at inference time, making it practical for deployment on existing models.\n2.The entropy based switching between latent and explicit reasoning, combined with a dwell window to avoid oscillation, is conceptually simple and computationally lightweight.\n3. The method shows consistent gains over standard CoT on math reasoning benchmarks, particularly for ≤8B models under constrained token budgets, with ablation studies supporting key design choices.", "weaknesses": "1. Latent reasoning contribution unclear at small budgets\nIn short-budget settings, the method typically switches to explicit mode immediately and does not enter latent reasoning due to the dwell window. Yet it still outperforms CoT, suggesting the improvements largely come from early confidence-based stopping and < think >/< /think > embedding conditioning rather than latent reasoning itself. Ablation isolating these effects would clarify the true source of gains.\n2. Scope limited to math-focused tasks\nExperiments are concentrated on math reasoning, and it remains unclear whether the method benefits broader domains (e.g., multi-hop QA, code, commonsense reasoning). Reasoning dynamics can differ significantly across tasks.\n3. Large-model scaling and SoftThinking baseline discrepancy\nThe original SoftThinking results indicate improvements over CoT primarily for ≥32B models. Here, the reproduced SoftThink baseline is weaker than CoT on ≤8B models. Given that SwiReasoning is training-free, extending evaluation to ≥32B models would help verify baseline fidelity and clarify whether the proposed method scales beyond the small-model setting.\nHeuristic hyperparameters + no runtime reporting\n4.Heuristic switching and lack of runtime analysis\nThe switching logic, dwell window, and mixing schedule are manually designed. While intuitive, their tuning behavior across models and tasks remains unclear, and no wall-clock or FLOP analysis is provided, making deployment efficiency uncertain.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "vgfL5oaxqP", "reviewer": "Reviewer_dZdK", "summary": "This paper proposes SwiReasoning, a training-free framework for LLM reasoning that dynamically switches between explicit chain-of-thought and latent reasoning modes. The switching is guided by block-wise confidence estimated from entropy trends in next-token distributions. The method also includes a switch count controller to suppress overthinking and improve token efficiency. Experiments on mathematical and STEM benchmarks show consistent accuracy improvements and token efficiency gains across different model families and sizes.", "questions": "See weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The idea of dynamically switching between explicit and latent reasoning based on confidence signals is intuitive and addresses limitations of pure latent reasoning approaches.\n- The experimental evaluation is comprehensive across multiple models and benchmarks, including various token budget settings and Pass@k analysis that shows the method overall improves the performance with better efficiency.\n- The ablation studies analyze key design choices and provide insights into how different components contribute to performance.", "weaknesses": "- The method introduces many heuristic design choices and hyperparameters (window sizes W_E→L and W_L→E, signal mixing coefficients α₀ and β₀, switch count limits C_max, convergence/termination triggers), and it appears from the ablations that these are tuned on the test sets, which raises concerns about generalization and fair comparison.\n\n- It seems unclear whether the latent reasoning mode genuinely enables the claimed \"exploration\" of multiple reasoning paths, since Soft-Thinking simply weights tokens by probability, and the pretrained LLM likely still concentrates probability mass on similar natural language tokens. It would be beneficial to provide some analysis of the probability distributions within the latent tokens or other evidence that meaningful exploration is occurring beyond what explicit sampling would achieve.\n\n- There are quite a few citations that are wrong and appear to be LLM-hallucinated, even including the most important \"Soft-Thinking\" paper that this work is primarily built on top of - the citation in the draft is ```Bowen Zhang, Yanzhuo Li, Shuohang Wang, Yu Tu, Jason Wei, Ashish Vaswani, Shikun Liu, Chenlu Yu, Deli Chen, Hongyi Li, Zhihua Zhang, and Chen Liang. Soft thinking: Training-free\nlatent reasoning for large language models.``` while the actual one should be ```Zhen Zhang, Xuehai He, Weixiang Yan, Ao Shen, Chenyang Zhao, Shuohang Wang, Yelong Shen, Xin Eric Wang. Soft Thinking: Unlocking the Reasoning Potential of LLMs in Continuous Concept Space```.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "dWPG3dYIa0", "reviewer": "Reviewer_eVQp", "summary": "This paper introduces SwiReasoning, a training-free framework for improving reasoning in large language models by dynamically switching between explicit chain-of-thought (CoT) and latent reasoning modes. The core innovation lies in using entropy trends from next-token distributions as block-wise confidence signals to guide mode transitions: when confidence increases (entropy drops), the system switches to explicit reasoning to consolidate progress; when uncertainty persists (entropy rises), it switches to latent reasoning for exploration. Additionally, a switch count control mechanism limits the maximum number of mode transitions and enables early answer generation at natural checkpoints, suppressing overthinking and improving token efficiency.", "questions": "1. How is entropy computed during latent steps when no explicit token is selected? Is it from the softmax over the last hidden state projected through the LM head?\n\n2. In ablation Table 3, window size 512 is optimal. Given typical generation lengths (e.g., ~2000 tokens for GSM8K), does this mean most problems only switch 2–4 times? If so, is this really \"dynamic\" switching, or just a staged approach (latent -> explicit -> latent -> explicit) Can you show histograms of actual switch counts across problem difficulties?\n\n3. The paper claims \"Pareto-superior\" results. However, in some cases (e.g., AIME24 in Fig. 4), CoT appears to match or exceed SwiReasoning's accuracy at similar token budgets. Can you clarify where strict Pareto dominance holds vs. where there are trade-offs?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The insight that reasoning should adaptively switch based on confidence is intuitive yet underexplored in training-free settings, thereby making the idea and approach of the paper novel. \n\n2. The evaluation is comprehensive and spans multiple model families, scales (1.7B–8B parameters), training paradigms, and five diverse benchmarks. The consistency of improvements across all settings (13 out of 15 evaluations achieve highest token efficiency) provides strong empirical support.\n\n3. The methodology is well-structured with intuitive explanations, formal algorithmic descriptions (Algorithm 1), and effective visualizations (Figures 1–5). The paper is well written.", "weaknesses": "Enumerating a few weaknesses here\n\n1. While the convergence and termination triggers are intuitive, the specific thresholds (½Cmax for convergence, Cmax for termination) appear arbitrary. Why these specific fractions? How sensitive is performance to these choices? The paper doesn't ablate these design decisions.\n\n2. Limited failure case analysis and the modes of failure. It would be good to add the cases or categories of questions where the proposed approach fails. \n\n3. The results reported for soft thinking approach appear to be lower than standard CoT, which is contrary to the claims in the original Soft-thinking paper. This raises concerns about implementation correctness and whether comparisons are on equal footing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OGEfV2qZBz", "reviewer": "Reviewer_BFnL", "summary": "The paper proposes SWIREASONING, a training-free framework that dynamically switches between explicit and latent reasoning based on entropy-driven confidence signals in the next-token distribution. The key idea is that staying purely in latent space can diffuse probability mass and cause slow or noisy convergence, so the model alternates modes to balance exploration and exploitation. A second mechanism caps the number of thinking-block switches to reduce “overthinking” and save tokens. The authors conduct some experiments and show the performance.", "questions": "Please chek the weakness", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1.\tThe paper proposes a method that flexibly combines latent reasoning with explicit reasoning, effectively reusing intermediate embeddings as inputs to enable denser thinking steps.\n2.\tThe authors provide experiments on several standard reasoning benchmarks to demonstrate the applicability of the approach.", "weaknesses": "1.\tThe overall performance gains are relatively small, and the efficiency story is not fully convincing — a clearer, more direct metric is needed, and Appendix Table 8 suggests that long generations are still required to reach good accuracy.\n2.\tThe paper should further explain why switching between latent and explicit modes improves performance, and clarify whether (and how) the latent trajectories can be decoded or inspected in text for interpretability.\n3.\tThe experimental scope is limited; results on larger models and on other task types (e.g., coding), where latent reasoning may be less naturally applicable, would make the empirical claims more convincing.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "T45NKr2Pcx", "reviewer": "Reviewer_JcUf", "summary": "This paper presents **SWiReasoning**, a training-free framework that dynamically switches between **explicit (token-based)** and **latent (hidden-space)** reasoning based on entropy-derived confidence trends. A switch-count control prevents overthinking, improving both accuracy and token efficiency. Across multiple math and STEM reasoning benchmarks (GSM8K, Math500, AIME24/25, GPQA) and three reasoning LLMs (Qwen3-8B, Qwen3-1.7B, DeepSeek-R1), the method yields +1.5–2.8% Pass@1 improvement and up to +80% token efficiency gains without retraining.", "questions": "1. How robust is the entropy-based switching to unseen data or noisy prompts?  \n2. Could the switching mechanism be learned rather than manually tuned?  \n3. How does this compare to dynamic decoding methods such as COCONUT or Long⊗Short?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- **Practical and training-free**: Improves reasoning efficiency entirely at inference time.  \n- **Comprehensive experiments**: Consistent gains across models and reasoning benchmarks.  \n- **Well-analyzed switching mechanism**: Includes ablations on window size, confidence, and signal mixing.  \n- **Clear motivation**: Tackles overthinking and token inefficiency in CoT reasoning.", "weaknesses": "- **Moderate novelty**: Combines existing ideas (latent reasoning + token control) with minor algorithmic extensions.  \n- **Pipeline complexity**: The overall system involves multiple heuristic components (entropy tracking, switching windows, signal mixing), making the pipeline heavy and less theoretically elegant.  \n- **System-oriented nature**: The contribution is more of a practical inference-time system design than a fundamentally new reasoning principle.  \n- **Marginal improvement**: Accuracy gains are modest given the added complexity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)