Fractured Chain-of-Thought Reasoning
Authors: name: Baohao Liao; name: Hanze Dong; name: Yuhui Xu; name: Doyen Sahoo; name: Christof Monz; name: Junnan Li; name: Caiming Xiong
Published: 2025-05-19 11:30:41

Abstract: Inference-time scaling techniques have significantly bolstered the reasoning
capabilities of large language models (LLMs) by harnessing additional
computational effort at inference without retraining. Similarly,
Chain-of-Thought (CoT) prompting and its extension, Long CoT, improve accuracy
by generating rich intermediate reasoning trajectories, but these approaches
incur substantial token costs that impede their deployment in latency-sensitive
settings. In this work, we first show that truncated CoT, which stops reasoning
before completion and directly generates the final answer, often matches full
CoT sampling while using dramatically fewer tokens. Building on this insight,
we introduce Fractured Sampling, a unified inference-time strategy that
interpolates between full CoT and solution-only sampling along three orthogonal
axes: (1) the number of reasoning trajectories, (2) the number of final
solutions per trajectory, and (3) the depth at which reasoning traces are
truncated. Through extensive experiments on five diverse reasoning benchmarks
and several model scales, we demonstrate that Fractured Sampling consistently
achieves superior accuracy-cost trade-offs, yielding steep log-linear scaling
gains in Pass@k versus token budget. Our analysis reveals how to allocate
computation across these dimensions to maximize performance, paving the way for
more efficient and scalable LLM reasoning.