Scaling Speculative Decoding with Lookahead Reasoning
Authors: name: Yichao Fu; name: Rui Ge; name: Zelei Shao; name: Zhijie Deng; name: Hao Zhang
Published: 2025-06-24 17:48:10

Abstract: Reasoning models excel by generating long chain-of-thoughts, but decoding the
resulting thousands of tokens is slow. Token-level speculative decoding (SD)
helps, but its benefit is capped, because the chance that an entire
gamma-token guess is correct falls exponentially as gamma grows. This
means allocating more compute for longer token drafts faces an algorithmic
ceiling -- making the speedup modest and hardware-agnostic. We raise this
ceiling with Lookahead Reasoning, which exploits a second, step-level layer of
parallelism. Our key insight is that reasoning models generate step-by-step,
and each step needs only to be semantically correct, not exact token matching.
In Lookahead Reasoning, a lightweight draft model proposes several future
steps; the target model expands each proposal in one batched pass, and a
verifier keeps semantically correct steps while letting the target regenerate
any that fail. Token-level SD still operates within each reasoning step, so the
two layers of parallelism multiply. We show Lookahead Reasoning lifts the peak
speedup of SD both theoretically and empirically. Across GSM8K, AIME, and other
benchmarks, Lookahead Reasoning improves the speedup of SD from 1.4x to 2.1x
while preserving answer quality, and its speedup scales better with additional
GPU throughput. Our code is available at
https://github.com/hao-ai-lab/LookaheadReasoning