SwiReasoning: Switch-Thinking in Latent and Explicit for Pareto-Superior
  Reasoning LLMs
Authors: name: Dachuan Shi, name: Abedelkadir Asi, name: Keying Li, name: Xiangchi Yuan, name: Leyan Pan, name: Wenke Lee, name: Wen Xiao
Organization: Microsoft

Recent work shows that, beyond discrete reasoning through explicit
chain-of-thought steps, which are limited by the boundaries of natural
languages, large language models (LLMs) can also reason continuously in latent
space, allowing richer information per step and thereby improving token
efficiency. Despite this promise, latent reasoning still faces two challenges,
especially in training-free settings: 1) purely latent reasoning broadens the
search distribution by maintaining multiple implicit paths, which diffuses
probability mass, introduces noise, and impedes convergence to a single
high-confidence solution, thereby hurting accuracy; and 2) overthinking
persists even without explicit text, wasting tokens and degrading efficiency.
To address these issues, we introduce SwiReasoning, a training-free framework
for LLM reasoning which features two key innovations: 1) SwiReasoning
dynamically switches between explicit and latent reasoning, guided by
block-wise confidence estimated from entropy trends in next-token
distributions, to balance exploration and exploitation and promote timely
convergence. 2) By limiting the maximum number of thinking-block switches,
SwiReasoning curbs overthinking and improves token efficiency across varying
problem difficulties. On widely used mathematics and STEM benchmarks,
SwiReasoning consistently improves average accuracy by 1.5%-2.8% across
reasoning LLMs of different model families and scales. Furthermore, under
constrained budgets, SwiReasoning improves average token efficiency by 56%-79%,
with larger gains as budgets tighten.