Scaling Up, Speeding Up: A Benchmark of Speculative Decoding for Efficient LLM Test-Time Scaling
Authors: Shengyin Sun
Yiming Li
Xing Li
Yingzhao Lian
Weizhe Lin
Hui-Ling Zhen
Zhiyuan Yang
Xianzhi Yu
Chen Chen
Mingxuan Yuan
Chen Ma
Venue: ICLR 2026 Poster (2026)
Keywords: Speculative Decoding
Test-Time Scaling
Primary area: datasets and benchmarks

Abstract: Test-time scaling has emerged as a powerful paradigm for enhancing the reasoning capabilities of large language models (LLMs) by allocating additional computational resources during inference. However, this paradigm is inherently inefficient due to the generation of different reasoning traces, leading to significant computational overhead. Speculative decoding offers a promising avenue for mitigating this inefficiency, yet its efficacy in the structured and repetition-rich context remains unexplored. To bridge this gap, we introduce the first comprehensive benchmark designed to evaluate speculative decoding methods in LLM test-time scaling. Our benchmark provides consistent experimental protocols across representative test-time scaling paradigms (e.g., Best-of-N sampling and multi-round thinking), enabling a fair comparison of three major categories of speculative decoding: model-based, training-based, and N-gram-based methods. Extensive experiments reveal that simple N-gram-based methods effectively capture repetitive patterns, demonstrating unique potential in accelerating test-time scaling. This phenomenon demonstrates the value of integrating N-gram-based methods with model-based or training-based approaches to benefit both repetitive and diverse reasoning in test-time scaling. We hope this benchmark spurs further research on speculative decoding for test-time scaling, enabling faster and more practical reasoning in LLMs through better handling of repetitive and diverse reasoning paths. Code available at <https://github.com/sunshy-1/SpecTTS-Bench>.

Published as a conference paper at ICLR 2026
SCALING UP, SPEEDING UP: A BENCHMARK OF SPEC- ULATIVE DECODING FOR EFFICIENT LLM TEST-TIME SCALING
Shengyin Sun"", Yiming Li Xing Li2, Yingzhao Lian", Weizhe Lin", Hui-Ling Zhen', Zhiyuan Yang", Xianzhi Yu?, Chen Chen', Mingxuan Yuan', Chen Ma" 11
City University of Hong Kong, Huaawi Technologies
shengysun4 --mmy.. cityu. sityy.edu.... hk, 1i Li.yiming3ehuave com
Yuan. "uan.Mingxuant Mingxuan@huawe com, chenmaecityu. .hk
ABSTRACT
Test-time scaling has emerged as powerful paradigm for enhancing the reasoning capabilities of large language models (LL.Ms) by allocating additional computa- tional resources during inference. However, this paradigm is inherently inefficient due to the generation of different reasoning traces, leading to significant computa- tional overhead. Speculative decoding offers promising avenue for mitigating this inefficiency, yet its efficacy in the structured and repetition-rich context remains unexplored. To bridge this gap. introduce the first comprehensive benchmark designed to evaluate speculative decoding methods in LLM test-time scaling. Our benchmark provides consistent experimental protocols across representative test-time scaling paradigms (e.g., Best-of-N sampling and multi-round thinking), enabling fair comparison of three major categories of speculative decoding: model-based, raining-based, and N-gram-based methods. Extensive experiments reveal that simple N-gram-bas based methods effectively capture repetitive patterns, demonstrating unique potential in accelerating test-time scaling. This phenomenon demonstrates the value integrating N-gram-based methods with model-based baased approaches to benefit both repetitive and diverse reasoning in test-time scaling. We hope this benchmark spurs further research on speculative decoding for test-time scaling, enabling faster and more practical reasoning in LLMS through better handling of repetitive and diverse reasoning paths. Code available htttt:::: //githttbbb httpshhw..oooooccccooommmocooooh www.s/sunsshsssyyeeeee Sttnaahyyy/ppeeTS-Reee...
INTRODUCTION
In the pursuit of more capable large language models (LLMs), the test-time scaling (Snell et al.
2025 Wang al. 2023) Madaan al.) 2023 Tian et al. 2025 Yao al.] 2023) has emerged as key performance driver complex tasks. Unlike training-tine time scaling, which relies on larger models or datasets, test-time scaling offers complementary strategy that can be applied already- deployed models without retraining. This approach enhances performance by allocating substantial computational resources during inference, allowing the model "think" longer and more deeply. Dominant paradigms in test-time scaling include Best-of-N (BoN) sampling (Snell et al. 2025) and multi-round thinking TTin al. (2025). These paradigms have demonstrated remarkable success in complex reasoning (Lightman et al. 2023 Cobbe et al. 2021 Li et al. 2022). However, this enhanced performance comes at a steep price. Generating multiple full-length responses or iterative reasoning chains incurs prohibitive computational overhead, leading to significant latency bottleneck that severely limits their use in real-time, interactive scenarios.
To unlock the potential of test-time scaling for practical use, it is imperative mitigate its associated latency. Speculative decoding (Liu et al.] [2025] Huang et al.) (025) has emerged as a leading technique achieve this goal. This approach accelerates autoregressive generation by using a fast
Equal contribution.
Corresponding author.

---

Published as a conference paper at ICLR 2026
Table 1: Illustration of intra-turn and inter-turn redundancy under the test-time scaling paradigm.
Let be the least prime number for which there exists positive inleger such that n² + is divi- sible by p². Find the least positive integer m such that m + is divisible by p².
Question
Turn 1: Compute 8² 512. Then 512 mod 17: since 17 x 30 510, we have 512 (mod 17).
Therefore, x 8 = 4 x 2 = 8 (mod 17). Thus, 4 x 8 x 1 17A = x 17k (mod 289). So, n 50 + 8 x 17k: (mod 289). Hence, n² - 1 = 51 + 136k (mod 289). We need this to be 0 (mod 289). so 51 + 1364 = 0 (mod 289). Dividing both sides by 17, we get 3 + 81 = 0 (mod 17), Le..
Intra-turn Redundancy
Wait, earlier 1 said that 4 x 17k 8 17ks mod 289. Let me check that again. x 8".
Compute 8 512. Then 512 mod 17: since 30 510, we have 512 (mod 17).
Turn 1: <Lhink><a /a Okay, so I need to find the least prime number P such that there's posit- ive integer n where p² divides n² + 1. Then, once find that p. have to find the smallest positive integer m such that p" divides m + 1 Afright, let me start by understanding the problem ...
Inter-turn Redundancy
Turn 2: <think>in Okay, so I need to find the least prime such that there's a positive integer with p² dividing n² + Then, once find that p. I need find the smallest m such that m² + divisible by p" AnisnF let me recall that
draft mechanism to generate candidate tokens, which are then efficiently verified in a single pass by the larger target model. This process substantially reduces wall-clock time while remaining lossless. On the other hand, the repetitive nature of test-time scaling creates uniquely favorable conditions for speculative decoding to excel. particularly for retrieval-based solutions (Hu et al. 2025) Oliaro et al.) 2024). These retrieval-based approaches are inspired by N-gram models, which predict the next token based on a specific sequence of prior tokens, such as pairs (bigrams) or triples (trigrams). As a model explores different reasoning paths, it frequently outputs highly repetitive token sequences. These can include recurring logical phrases (c.g.. "Let's consider the case where..."). standard boilerplate code, or common transitional expressions (as shown in Table TT. This repetitive structure suggests that N-gram-base based methods (Hu et al., 2025 Oliaro et al., 2024), which dynamically cache and reuse recently generated token sequences, exceptionally well-suited to this scenario.
Despite promising acceleration, existing literature lacks comparative analysis to determine which speculative decoding architecture best exploits computational redundancies in test-time scaling. Current benchmarks evaluate these methods in generic tasks (Xia et al. 2024), overlooking the unique characteristics of advanced reasoning paradigms. Consequently, critical questions remain unanswered: Can a simple, adaptive N N-gram mechanism outperform a more complex, pre-trained draft model in scenario ripe with repetition? How do the trade-offs between these methods-flen flxibiis,leiiblitty, training cost, and real-time adapiabiity-manne mynnifest when accelerating test-time scaling frameworks? Without a dedicated benchmark to investigate these specific dynamics, the community cannot make informed decisions about how to efficiently scale up inference-tine reasoning.
To address this critical gap, we introduce the first comprehensive benchmark designed to systemati- cally compare different speculative decoding methods for accelerating test-time scaling. Specifically, our work provides a head-to-heal comparison of speculative decoding methods (dctailed in Section covering nearly all of-te-tf (SOTA) techniques in this domain. We aim to elucidate the trade-offs among these approaches and, most importantly, test the hypothesis on the unique effective- ness of using N- N-gram patterns to accelerate test-time scaling. Our observations are summarized as follows. (i) The method that relies on training an auxiliary draft model shows promising acceleration potential, but its inference behavior is inherently coupled the training process, limiting adaptability across diverse reasoning scenarios. (ii) Methods leveraging N-gram patterns excel at capturing redundancy in test-time scaling: in particular, those that store and reuse N-gram patterns during generation achieve progressive acceleration by reusing past computations, enabling efficiency gains that accumulate across generation steps. However, this advantage is sensitive to sampling temperature. (iii) A brand new hybrid method called SAMIEAGLE-] is tested (detailed in Section 3.3). Hybrid speculative decoding uniquely combines methods that train an auxiliary draft model for semantic alignment with those leveraging N-gram patterns to capture repetition, unlocking distinct potential for reasoning under eest-time scaling. However, current hybrid strategies remain heuristic, failing to effectively harness the semantic richness of draft models and the reusability of N-gram patterns. We hope these findings can provide practical guidance for the design of speculative decoding strategies in test-time scaling, balancing adaptability, efficiency, and robustness for complex reasoning tasks.
3.3
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
Test-time scaling for enhanced reasoning. The paradigm of test-time scaling enhances model performance by allocating more computational resources inference time. This principle can be realized through two main avenues (Zhang et al., 2025): (1) specialized training that endows a model with an inherent capability to dynamically scale its computation, or (2) external, training- freee frameworks that guide a fixed model. The former approach, inherent test-time scaling, is embodied by the development of "slow thinking" or System reasoning models. Models such as OpenAl ol series (OpenAl et al., 2024) are explicitly trained to internalize multi-step reasoning. Complementing this inherent capacity are external frameworks, which apply an outer loop of control to a base model, further amplifying its reasoning performance without modifying its parameters. This paper focuses on analyzing two foundational categories of such frameworks: (1) Sampling-basee -based methods such as BoN sampling (Snell et al., (025) and Seff-Consistency (Wang et al. 2023). (2) Iterative refinement
methods such as Self-Refine (Madaan et al., (203)) and multi-round thinking Tian et al. 2025). More
details about these frameworks can be found in Appendix A.I.I
Speculative decoding for inference acceleration. Speculative decoding has emerged as a SOTA solution for reducing L.LM inference latency losslessly. The landscape of speculative decoding can be classified into three families: (1) Model-based speculative sampling which uses a smaller,
general-purpose language model the drafter (Leviathan et al. 2023 Chen al., 2023). (2) Training- speculative decoding which trains specialized draft model (Cai et al.) 2024 Li et al.
2024a 2025c] |Sun et al.] (026)) (3) N- ---gram--assed method which builds a "draft" by retrieving token sequences from a dynamic cache of recent outputs (He et al.) 2024 Zhao et al. 2024, Hu et al.
2025 Oliaro et al. 2024). While these methods have been benchmarked on generie tasks (Xia et al.
2024), their interplay with test-time scaling frameworks remains largely unexplored. The highly structured and often repetitive nature of reasoning chains generated by methods like BoN presents a unique environment where the trade-offs between these different speculative decoding families are unknown. Our work directly addresses this gap by systematically evaluating their performance in these latency-critical, redundancy-rics rich reasoning scenarios. Additional relevant work is discussed in Appendix A.1.2
3 BENCHMARK CONSTRUCTION
This section outlines the three core components of our experimental setup: the test-time scaling frameworks accelerate the evaluation datasets, and the speculative decoding algorithms compared. The overall framework is illustrated in Figure [
Redurdancy Tact-trae Sealin
#2 Speciaa five Decodig
Muttifurm Thiskig
60) Moderbaaed
Trankig
Jolle Rood
Purk
Drcht
Liiger 3.5
Alfer
DOg Iil NO
Twoh
Prstandaccy Patters
BAME Tood
The To Spprr ONT alli metten
Totes
Correct
Arcwer
Figure I: The framework of speculative decoding for efficient test-time scaling. (#I) An overview of test-time scaling frameworks and their inherent redundancies. (#2) Illustrations of three categories of methods. Model-based method uses a smaller, homologous model as draft model to generate speculative token sequences. Training braining-based method employs additionally trained draft model using both input tokens and hidden features. N-gram-batt based method leverages tools such as suffix automata retrieve repeated patterns for draft generation.
3

---

Published as a conference paper at ICLR 2026
3.1 BENCHMARKED TEST-TIME SCALING FRAMEWORKS
This work selects two particularly representative frameworks in test-time scaling, namely BoN and multi-round thinking, as the basis of our benchmark.
3.1.1 BoN
The BoN (Snell et al. 2025 is a foundational technique designed to improve output quality by leveraging a generate and-rant mechanism. Instead of relying on a single, potentially flawed output from the model, BoN operates in two distinct stages:
Generation:
Given request, the LLM is prompted to generate N independent candidate solutions or reasoning paths. This is typically achieved by sampling from the model's output distribution multiple times with non-zero temperature ensure diversity. The collection of these N samples represents a broad exploration of the potential solution space.
Inlectionn/eeiiicaation After generating the N candidates, verifier module is used to and select the best solution. The verifier can range from simple, heuristic- based checker to a more sophisticated, separately trained reward model. In its most canonical form, mroces--based verifier might assess the correctness of each intermediate step in a solution, rather than just evaluating the final answer. The candidate with the highest score then chosen the final output.
3.1.2 MULTI-ROUND THINKING
Multi-round thinking TTan et al. 2025) improves reasoning through iterative refinement. This method inspired by the human cognitive process of reconsideration, enhances a model's 's reasoning by prompting it to reflect on and improve its own previous answers over rounds, as follows:
Initial Response (Round 1): The LLM generates an initial answer to the user's prompt. This represents the model's initial intuitive response.
Iterative Refinement (Rounds 2 to M): The process is repeated for pre-defined total of M rounds. In each subsequent round (from 2 to M), the model is prompted to refine its previous output. The new prompt is constructed by augmenting the original question with the answer from the preceding round. For example, the prompt for the 2nd round takes the following structure:
[original Question]
The assistant's previous answer is: {Round 1 Answer)
Please re-answer.
This mechanism forces the model to reconsider its initial reasoning path, allowing it to self-correct errors and elaborate on its response with each iteration.
3.2 BENCHMARKED DATASETS
Having defined the computational frameworks that expand reasoning effort at inference time, we now turn the datasets used for evaluation. The choice of datasets is critical ensure that our benchmark measures reasoning capabilities across diverse set of challenging problems. To this end, we have selected four widely-recognized benchmarks that have become staples in recent literature for evaluating the limits of LLM reasoning capabilities. Our evaluation suite is composed of 120 samples extracted from the following datasets:
AIME 2024 and AIME 2025: The American Invitational Mathematics Examination (AIME) (AMME 2025) features challenging competition-level mathematics problems that demand creative and rigorous problem-solving skills. We use the official problems from the 2024 and 2025 competitions, utilizing the full set of 30 problems from each.
MATH-500; A subset of the MATH dataset (Hendrycks et al. 202 a comprehensive benchmark of mathematics problems ranging from pre-algebra to calculus and beyond. It is the standard for evaluating mathematical reasoning in LLMs. Given the considerable size the full 500-problem dataset, selected the first 30 problems to ensure focused and manageable evaluation.
4

---

Published as a conference paper at ICLR 2026
The GPQA (Rein al. 2023) consists of graduate-Ievel, Google-proof Q&A problems across diverse domains, designed to test internalized knowledge and complex reasoning. The full dataset contains 448 expert-level questions, representing challenging collection. To maintain consistency with other evaluation subsets, we selected the first 30 problems from this dataset.
GPQA:
This curated suite serves as rigorous testbed for evaluating the acceleration effects of speculative decoding within the computationally intensive BoN and multi-round thinking frameworks, enabling us to quantify efficiency gains in high-stakes reasoning scenarios.
Table Summary of speculative decoding methods, including their key ideas, categories (model-, training, N- N-gram-bust (rrummbasedd, speculation structures (Linear or Tree), ability to reuse repetitive content during generation, and its compatibility with Greedy decoding and speculative Sampling.
Drafting
Verification
Methods
Key Idea
Category
Speculation
Greedy
Sampling
SpS Chen et al. 1025
Specularive decoding with smaller homolo- gous models rejection sampling. Introducing truinable lightweight regressive heads on intermediate layers.
Linear
Model-hased
EAGLE-3
Training-based
PLD Suxena 2023
Generates drafts by reusing N-gram pat- extracted from the input prompt.
Lincar
X
/
x
N-gram-based
REST Ho et al. 2024
Drafting based
external datastore for
Tree
/
Nigran hused
retrieving likely next tokens.
Lookahead Fuu
Generating multiple N-gram candidates (ex- tracted from generation) in parallel.
N-gram-based
PIA Zhao et al. 2024
Speculative decoding via Trie-structured context eaching.
Tree
N-gram-bused
SAM Hu
2025
Generating drafts using suffi automaton
Linear
N-gram-hused
derived from the generation history.
Recycling Luo et
2025
Generating drafts by recycling top K next tokens from history.
Tree
/
gram bused
3.3 BENCHMARKED SPECULATIVE DECODING ALGORITHMS
We benchmark a diverse set of methods, which can be broadly categorized into three groups: model- based speculative sampling, training-based speculative decoding, and N-gram-based methods. This collection of methods encompasses almost all existing speculative decoding techniques compatible with reasoning models, capturing the full spectrum of current SOTA solutions. We provide summary of the above methods in T Detailed descriptions of each method are presented below.
Model-based Method
Speculative Sampling (SpS) (Chen al. 2023): As a pioneering speculative decoding method, SpS employs a smaller, faster draft model to generate sequence of candidate tokens. The larger target model then validates these tokens in parallel using a rejection sampling mechanism.
Training-based Method
EAGLE-3 (Li et al. 2025c): This is an advanced speculative decoding approach that dispenses with separate draft model. Instead, it attaches lightweight trained auto-regressive heads to the target model's intermediate layers to generate drafts.
Nased Method
Prompt Lookup Decoding (PLD) (Saxena 2023): : PLD identifies N-grams from the prompt that match the sequence generated so far and uses them speculative candidates. PLD is particularly effective in tasks where the output is likely to contain substrings from the input.
Retrieval- basdd Speculative Decoding (REST) (He et al. (002)): REST utilizes a datastore of example sequences and retrieval model to find continuations relevant to the current context. These continuations serve as the speculative draft, which is then verified by the target model.
Lookahead (Fu et al. 2024): A parallel decoding algorithm generating and validating multiple N-grams simultaneously. Unlike single-sequence speculative decoding, Lookahead maintains multiple continuations and uses tree structure verify candidates in single forward pass.
5

---

Published as a conference paper at ICLR 2026
PIA (Zhao al. 2024):
This method utilizes trie structure to manage and retrieve contextual information from the prompt and decoding context efficiently. It supports tree-based speculative decoding, where multiple candidate token sequences generated and verified in parallel.
SAM (Hu et al.) 2025): SAM employs suffix automaton maintain the contextual history of the generated sequence. This data structure allows for the effective identification of previously generated token sequences (suffixes) that best match the current context.
Recycling (Luo et al., 2025): Recycling enhances speculative decoding by caching the top-K most probable subsequent tokens for each token verified by the target model. This cache, constructed from the highest logits observed during past generation steps, serves as high-quality source of draft candidates for future occurrences of the same token.
Hybrid Method
SAM with EAGLE-3 (SAM/EAGLE-3)) (Hu et al. 2025;: This approach a hybrid strategy that dynamically switches between the SAM and EAGLE-3 frameworks. When SAM identifies matched suffix that is too short to be beneficial for speculation, the system defaults to EAGLE-3.
4 SPECULATIVE DECODING FOR REASONING UNDER TEST-TIME SCALING
4.1 EXPERIMENT SETUP
We investigate the effect of speculative decoding methods in reasoning models DeepSeek- -1-ccc--R---tiiii -Distill- Llama-8B (DSL.-BB) (DeepSeek-Al et al., 2025) and Qwen3-8B (QW3-8B) (Yang et al. (025).
employing the widely-used test-time scaling paradigms multi-round thinking (with rounds) and BoN (with 4 trajectories). We evaluate methods on the reasoning datasets AIME24/25, MATH500, and GPQA (detailed in Section (.2). The evaluation pipeline from prior work (Xia et al. (2024) is adopted to ensure fair comparison. For the reasoning models, we use the f data type and apply greedd//toochsticc decoding with batch size of 1, following the original settings of the evaluated methods. The evaluation metrics include: Mean Accepted Tokens (MAT), which measures the average number of tokens accepted per speculative decoding step, and the Walltime Speedup Ratio (Speed), which quantifies the inference efficiency gain relative to vanilla autoregressive decoding. The evaluated methods including model-baset speculative sampling (c.g.. SpS), training- traniing-based speculative decoding (e.g.. EAGLE-3), and N-gram-baxe methods (c.g., PLD, REST, Lookahead, PIA, SAM, and Recycling). More implementation details are provided in the Appendix A.2
4.2 DISCUSSION ON RESULTS
The performance of different methods under multi-round thinking is shown in Table 3 As DSL-8B does not have a smaller model variant, results for the combination of SpS and DSL-8B are not provided. For QW3-8B, we evaluate SpS using Qwen3-0.6B (QW3-0.6B) (Yang et 2025) as the draft model. Additionally, under the speculative sampling, methods such as Lookahead do not support non oon-gredy decoding, hence their non- -reeddye variants are not included in Table 3 Table Iillustrates the performance of speculative decoding methods under the BoN. Since BoN requires sampling to generate diverse reasoning traces, we do not evaluate greedy decoding in this context.
4.2.1 TRAINING-BASED METHOD
The adaptability of the training- method is constrained by its training process. For the maining-based method, we evaluate the SOTA EAGLE-3, which supports reasoning model acceleration. Other training-basee methods, such as EAGLE-1/ 1/2 (Li et al., 2024b a) and Medusa (Cai et al.] 2024), not included in comparison because they do not provide corresponding draft models for reasoning models. For DSL.-BB, we used a draft model trained by the EAGLE-3 teans while for QW3-8B, we use draft model provided by the Tengyunw tean] As shown in Table 3 and T EAGLE-3 achieves considerable speedup across different scenarios. However, we also observe some performance deviations from expectations. Specifically, the average speedup ratio
(T = 0) in Table decreases by 34% when running EAGLE-3 on DSL-8B compared to QW3-8B
acc.

---

Published as a conference paper at ICLR 2026
Table 3: Performance comparison of speculative decoding methods for reasoning models under
multi-round thinking framework with different temperature Best Second Best )
Bench
AIME24
AIME25
MATIIS00
GPQA
Overall
Model
Method
MAT
Speed
MAT
Speed
MAT
Speed
MAT
Speed
MAT
Speed
AR
1,00
1.00x
1.00
1.00x
1.00
1.00x
1,00
1.00x
1.00
1,00x
EAGLE-3
2.26
1.56x
2.24
1.53x
2.63
2.86x
2.46
1.61
2.35
x.93
SAM[EAGLE-3]
3,91
3.09
4.50
3.85
4.94
x.11x
7.00
4.70x
4.72
3,97
DSL-8B
SAM
2.64
2.41x
3.05
2.96x
2.60
2.14x
3.57
3.20x
2.93
2,66x
Recycling
2.98
2.08x
2.98
2.05
2.97
2.18x
3.02
2.08
2.99
2.10x
(T =
PLD
2.14
1.76x
2.38
1.89
2.25
1.71x
2.65
2.02
2.33
L.84X
REST
1.33
1.00x
1.35
0.95
1.37
1.08
1.31
1.00
1.34
1.01
Lookahead
2.21
1.59
2.26
1,53
2.28
1.55
2.44
1.62
2.28
x.57
PIA
1.84
1.51x
1.96
1.63
1.91
1.48%
2.15
1.63
1.95
1,56x
AR
1.00
1.00x
1.00
1.00
1.00
1.00x
1.00
1.00
1.00
1.00
EAGLE-3
2.21
1.31x
2.17
1.50x
3.21
3.02
2,45
1.61
2.33
1.91 X
SAMIEAGLE-3]
2.54
1822
2.48
1.89
3.28
3.13x
2.86
2.15
2.66
2.29
DSL-8B
SAM
1.84
1.65x
1.84
1.65x
1.93
1.71%
1.93
1.73
1.87
1.69%
(T' = 0.6)
Recyeling
2.84
1.91
2.83
1.93x
2.88
2.03x
2.81
1.98
2.84
1.96x
REST
1.35
1.00x
1.37
0.97
1.38
1.03x
1.35
1.00x
1.00x
PIA
1.58
1.26x
1.58
1.27x
1.63
1.39x
1.68
1.36x
1.61
1.33x
1.00
x.00
1.00
1.00x
1.00
1.00
1.00
1.00x
EAGLE-3
4.31
2.80x
4.37
2.87x
4.50
3.04
4.39
2.91
4.38
2.91
SAM[EAGI
4.40
3.08
4.47
3.18x
4.82
3.47
5.90
4.17%
4.76
3.49
SAM
2.15
1.97
2.27
2.07
2.18
1.95
3.19
3.11x
2.37
2.28
QW3-8B
Recycling
3.02
2.08
2.98
2.13x
3.00
2.20x
3.07
2.17x
3.01
2.15x
(T =
PLD
1,95
1.61x
1.97
1.66x
1.91
1.63x
2.49
2.04
2,05
1.74x
SpS
7.40
0.93
7.50
0,96
6.69
0.77%
6.33
0.83
7.07
0.87x
REST
1.39
1.12x
1.14x
1.42
1.16x
1.33
1.09
1,38
1.13X
Lookahead
2.14
1.54x
2.12
1.54x
2.10
1.58x
2.28
1.69
2.15
1.59
PIA
1.83
1.48x
1.96
1.68
1.89
1.63x
2.08
1.72x
1.93
1.63x
AR
1.00
1.00x
1.00
1.00
1.00
1.00x
1.00
1.00
1.00x
EAGLE-3
4.11
2.61x
2.71x
4.32
2.87
4.03
2.68
4.16
2.73x
SAM[EAGLE
3.92
2.68x
3.93
2.72
4.19
2.93x
3.98
2.82
3.97
2.79
QW3-8B
SAM
1,91
1.73x
1.95
1.78x
1.93
1.74x
2.09
1.86x
1.96
1.78x
(T = 0.6)
Recycling
2.88
2.00x
2.92
2.04x
2.90
2.11x
2.92
2.08
2.90
2.06x
SpS
6.22
0.91
6.54
0.95
6.17
0.80x
6.32
0.86
6.34
0.88x
REST
1.41
1.08x
1.41
1.10x
1.42
1.12x
1.36
1.03
1,08
PIA
1.69
1.40X
1.70
1.45x
1.71
1.45x
1.75
1.48x
1.71
1.44x
(2.91x~11.3xx) Moreover, the overall MAT (T = 0) of EAGLE-3 on DSL--BB in Table 3 is approximately half of that on QW3-8B (4.38 -2335). Similar trends are observed in the temperature sampling setting and Table 4 These differences exceed the typical performance variation that can be attributed intrinsic model factors such as differences in vocabulary or architectural design. We hypothesize two potential reasons for this phenomenon: (i) As noted by the Tengyunw team, they used 600k samples train the draft model for QW3-8B, which is larger than the dataset used by the EAGLE-3 team for training the DSL-8B draft model. A larger training set can improve the aligument between the draft and target models' output distributions, leading to better performance. (ii) The original EAGLE-3 evaluation focused on short-generation scenarios. We conjecture that the training process for the draft model may not adequately model uin-gennrration settings (for example, larger position indices may receive insufficient training updates), leading to performance degradation in long-genan -enggenerttion scenarios of the reasoning model.
Training- trasedng-based method is insensitive to temperature. As the temperature T increases, the target model's 's output distribution becomes more uniform, making it more challenging for the draft model accurately predict the subsequent tokens compared greedy decoding. Despite this increased difficulty, the EAGLE-3 demonstrates considerable robustness to temperature variations. For instance, under the multi-round thinking paradigm, when the 7 is raised from 0 to 0.6, the average speedup of EAGLE-3 on QW3-SB decreases by only 6%. This relative stability suggests that the training-based method has the potential to generalize across different levels of decoding stochasticity.
7

---

Published as a conference paper at ICLR 2026
Table 4: Performance comparison of speculative decoding methods for reasoning models under BoN
framework with different temperature T Best Second Best
Bench
AIME24
AIME25
MATH1500
GPQA
Overall
Model
Method
MAT
Speed
MAT
Speed
MAT
Speed
MAT
Speed
MAT
Speed
AR
1.00
1.00x
1.00
1,00x
1.00
1.00
1.00
1.00x
1.00
1,00x
EAGLE-3
2.19
1.17x
2.18
1.43x
3.09
2.71x
2.37
1.52x
2.31
1.74x
SAMIEAGLE-3]
2.44
1.53x
2.41
1.72x
3,31
3.05
2.72
1.96x
2.56
2.10x
DSL-8B
(T' = 0.6)
SAM
1.89
1.62x
1.86
1.62x
1.97
1.77
2.00
1.75x
1.91
1.70x
Recycling
1.91
2.80
1.86x
2.88
2.04
2.80
1.93x
2.82
1,9xx
REST
1,36
0.99
1.38
0.99
1.40
1.07x
1.33
0.98
1.37
1.01
PIA
1.53
1.18x
1.55
1.21
1.62
1.35x
1.68
x.37
1.57
1.20x
AR
1.00
1.00
1.00
x.00
1.00
1.00
1.00
1.00
1.00x
EAGLE-3
4.09
2.63x
4.17
2.64x
4.30
2.85
4.00
2.64x
4.12
2.70x
SAM[EAGLE-3]
3.81
2.66x
3.91
2.72x
4.18
2.99
3.87
2.68
3.91
2.77x
QW3-8B
SAM
2.05
1.88x
2.04
1.88
2.17
2.01
2.19
2.00x
2.10
x
(T' 0.6)
Recycling
2.90
2.03x
2.88
1.99x
2.90
2.06x
2.91
2.04x
2.89
2.03
SpS
6.05
0.93
6.22
0.95
6.01
0.80x
5.87
0.86x
6.05
0.88x
REST
1.41
1.12x
1.40
1.10x
1.42
1.11x
1.36
1.07x
1.40
1.10x
PIA
1.67
1.38x
1.70
1.41x
1.72
1.44x
1.76
1.45x
1.70
1.42x
Takeaway 1: While the training- based methods demonstrate promising acceleration potential, their performance is inherently tied to the training process, raising practical concerns about their adaptability diverse reasoning scenarios.
4.2.2 N-GRAM-BASER METHOD TOKEN N-GRAM
Among token N-gram methods, which identify repeated patterns at the token level, SAM most efficiently captures redundant patterns. For token N-gram methods, we assess SAM, PLD, REST, Lookahead, and PIA. Overall, SAM achieves the best performance among these token N-gram methods, even rivaling the training-based method EAGLB-3. For instance, under the greedy decoding setting with multi-round thinking, SAM achieves 38% higher overall speedup ratio than EAGLE-3 on DSL-8B (2.66 + x.93x). This advantage primarily stems from the high efficiency of SAM in suffix matching, as well as the repetitive patterns inherent in reasoning traces. Other methods, such as PLD, PIA, and Lookahead, also demonstrate considerable potential, achieving around 1.5x speedup across various test-time scaling paradigms. However, due to the higher time complexity of their N-gram matching mechanisms, they still fall short of SAM in acceleration. REST performs relatively poorly. We hypothesize this is because the external chat datastore used by REST has limited overlap with reasoning datasets, resulting in fewer effective token reuse opportunities.
Sampling temperature significantly affects token N-gram-based methods. We note that token N- N-gram methods cxhibit notable sensitivity to sampling temperature. Taking SAM as an example, under the multi-round thinking paradigm, when the temperature increases from 0 to 0.6, its overall speedup on QW3-8B drops by approximately 22% (from 2.28x to 1, and on MAT it decreases from 2.37 to 1.96, representing a reduction of about 17% This highlights key trade-off: while these methods are particularly effective at leveraging the repetitive patterns in reasoning model outputs under time scaling, their acceleration benefits tend to decrease as output diversity increases.
Takeaway Token N-gram methods excel at capturing redundancy in reasoning traces under test-time scaling, yet this advantage comes with sensitivity to sampling temperature variations.
4.2.3 N-GRAM-BASED МETHOО PROBABILISTIC N-GRAM
Probabilistic N-gram method Recycling, which captures the co-occurrence of top-K next tokens through logits, achieves the highest MAT among N-gram-based methods. As shown in Table 3 Recycling achieves performance comparable to the strongest retrieval-based method SAM. Under greedy decoding, Recycling slightly lags behind SAM in speedup ratio, yet attains higher MAT. We attribute this to Recycling's 's higher computational complexity in draft generation. Specifically, Recycling employs tree-structured speculative verification, resulting in significantly
8

---

Published as a conference paper at ICLR 2026
higher computational latency compared to SAM's linear verification. Moreover, Recycling speculates 81 tokens, which is nearly twice the number of tokens speculated by SAM (40 tokens), which further contributes to its overall computational overhead.
Probabilistic N-gram method exhibits low sensitivity to temperature sampling. As the tem- perature increases from 0 to 0.6 in the multi-round thinking paradigm, Recycling's overall speedup ratio on QW3-8B decreases only from 2.15 to 2.06- decline of less than 5%. In contrast, SAM's speedup drops by 22% under the same temperature shift. We hypothesize that this robustness stems from Recycling's ' preservation of approximate next-token prediction logits, specifically the top-k vocabulary distribution, which enables substantial overlap between its tree-based speculative tokens and the target model's 's sampled tokens even as temperature varies.
Takeaway 3:
The probabilistic N-gram method offers superior stability and higher draft to- ken acceptance rates among N N-gram-bases methods, yet its computational bottleneck prevents acceptance gains from translating into proportional speedup.
4.2.4 HYBRID APPROACH: INTEGRATING TRAINING-BAND AND N-GRAM METHOD
Hybrid speculative decoding can leverage the strengths of different methods. For the hy- brid approach, we conduct evaluations on SAMEEAGLE-3], which integrates EAGLE-3 and SAM (detailed hybrid strategies are provided in Section 3.3 As shown in Table 33, S SAM[EAGLE-] achieves the highest overall speedup ratio across all scenarios. Under the greedy decoding setting. the SAM[EAGLE-3] demonstrates significant improvements over individual methods. Within the multi-turn thinking paradigm, SAMJEAGLE-3) delivers speedup gains 20% and 53% compared to EAGLE-3 and SAM on QW3-8B, respectively. The reason behind this improvement is straight- forward: SAM[EAGLE-3] preserves EAGLE-3's powerful token speculation capabilities while addressing its limitation in capturing repetitive patterns that emerge during dynamic generation.
Hybrid speculative decoding retains the limitations of its components. While inheriting the strengths of SAM, SAM[EAGLE-3] also retains its key drawback, namely, high sensitivity to sampling temperature. When the sampling temperature increases from 0 to 0.6, the speedup ratio of SAM[EAGLE-3] under the multi-round thinking paradigm decreases by 42% on DSL-SB x3.97x x97299 x) and by 20% on QW3-8B (3.49 x -2.79x), respectively. Furthermore, as observed in Table and Table [ the performance gain of SAM[EAGLE-3] over EAGLE-3 is less significant. This indicates that under temperature sampling with T' = 0.6, the accuracy of using SAM for speculative decoding not satisfactory, leading to the overall performance of SAMIEAGLE-33 being largely dependent on that of EAGLE-3. Despite certain limitations, the consistent SOTA performance of SAM[EAGLE-3] across various evaluation scenarios underscores its practical effectiveness. By integrating the strengths of different methods, the hybrid method achieves well-balanced performance across various settings, highlighting its potential for reasoning with test-time scaling.
Takeaway 4: Hybrid speculative decoding uniquely combines the semantic alignment strengths of training-based methods with the repetitive pattern capture capabilities of N-gram methods, unlocking distinct potential for reasoning with test-time scaling.
4.2.5 MODEL-BASER METHOD
Model-based method SpS generates high-quality drafts, but faces computational bottlenecks. As shown in Table 3 and Table 4 SpS achieves the highest MAT. For example, in the multi- thinking scenario with QW3-8B (T = 0), SpS's overall MAT is 1.49 times that of the second-best method. This superiority primarily stems from SpS employing smaller, homogenous model as the draft model, which ensures a high degree of alignment between the output distributions of the draft and target models. However, this high MAT does not translate into proportional speedup. The primary bottleneck lies in the computational cost of generating drafts. Despite being smaller, such as a 0.6B model, the draft model is still too heavy compared to the target models like 8B, particularly under tight latency constraints. As a result, the time saved by reducing target model evaluations is largely offset by the overhead of running the draft model, which limits the overall acceleration.

---

Published as a conference paper at ICLR 2026
Takeaway 5: Model-based method SpS achieves high MAT by leveraging a well-aligned draft model, but its acceleration is limited due to the relatively small size difference between the draft model (e.g., 0.6B) and the target model (e.g., 8B).
Tum
FF
Tuzn
Tim
Tum 2
Spucdap Ruilo
2.00
2,67/
2.06
2
2.04+
2.19+
2
2.104
1.35-
Kecyeling
SAM]EADLE-S
BAGLB-
Recycling
SAM[RAGLE-]
(a)
(h)
2,40 278 TTx
Tum
525
2d
2.7
2004
Speedap
2.104
3:224
1334
1.4+
BAGLE-3
SAM
Recycling
SAMJEAGLE
(c)
Figure 2: Speedup ratio across different turns under different settings across various temperature T. (a) Multi-round thinking 0). (b) Multi-round thinking (T 0.6). (c) BoN (T 0.6).
4.2.6 ANALYSIS SPEED SCALING OVER TURNS
Token N-gram methods achieve progressive acceleration. Figure compares the acceleration ratios of different methods across reasoning turns. A key observation that Token N-gram methods exhibit steadily increasing acceleration ratio as the number turns grows. As illustrated in Figure SAM and PIA show acceleration ratio increases of 33% (2.13x x2 and 45% (1.45x - in the second turn over the first, respectively. This trend demonstrates their ability to effectively identify and exploit repetitive patterns that naturally emerge during test-time scaling. Specifically, by retrieving and reusing relevant intermediate results or reasoning traces from prior turns, these methods reduce the number of required decoding steps in subsequent turns, leading to progressively greater efficiency gains over time. In contrast, other methods exhibit nearly constant acceleration ratios across turns, indicating that each turn processed independently without leveraging reusable patterns from prior decoding steps. Further analysis can be found in Appendix A.3
Takeaway 6: Token N-gram methods PIA and SAM achieve progressive acceleration by reusing past computations, enabling efficiency that scales across turns.
CONCLUSION
In this paper, we introduce the first benchmark for speculative decoding in accelerating LLM test-time scaling. Specifically, we evaluate a diverse range of methods, including model-based training- based, and N-gram-bar based approaches, under prominent frameworks such BoN sampling and multi-round thinking. Through extensive experiments, we find that simple N-gram-based methods show strong potential in accelerating test-time scaling by effectively capturing repetitive reasoning patterns. This highlights the value of integrating N- N-gram-based methods with other categories of approaches, thereby balancing acceleration for both repetitive and diverse reasoning in test-time scaling. We hope this benchmark serves as a foundation for advancing speculative decoding techniques tailored to test-time scaling, fostering more efficient and practical L.LM reasoning.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This work was supported by the Early Career Scheme (No. CityU 21219323) and the General Research Fund (No. CityU 11220324) of the University Grants Committee (UGC), the NSFC Young Scientists Fund (No. 9240127), and the Donation for Research Projects (No. 9220187 and No. 9229164).
ETHICS STATEMENT
This benchmark's dataset collection adheres to strict ethical principles and responsible research practices. All data sources are derived exclusively from publicly available materials for academic research purposes only, with commercial applications intended. We maintain full compliance with copyright laws and respect intellectual property rights of original content creators. We have implemented comprehensive screening procedures ensure datasets are free from harmful content, including discriminatory material, violent imagery, and personally identifiable information. This research initiative aims to advance scientific knowledge and contribute positively societal welfare while protecting individual privacy and promoting social responsibility in Al research.
REPRODUCIBILITY STATEMENT
To ensure reproducibility of our work, we provide comprehensive implementation details for bench- mark construction in Section 3] Section 1.11 and Appendix [A.2 All open- LLMs used in this study are accessed through Hugging Face (https: /wwgg i face. co/) ) and we strictly adhere their respective terms and conditions for use. To facilitate the reproduction of our experiments, the code and data are provided in
com sunshyunssssuo
REFERENCES
AIME Aime problems and solutions. 2025. URL
https:://attooprobbb...o.o emso vi com/ wi ki / Index. wnn...ppp/// ATMMP rob ewsendseee... ut ions
Simon A. Aytes, Jinheon Back, and Sung Ju Hwang. Sketch-of-thought: Efficient Ilm reasoning with adaptive cognitive-inspired sketching. In arXiv:2503.05179, 2025.
Macicj Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, and Torsten Hoefler. Graph of thoughts: Solving elaborate problems with large language models. In Thirty-Eighth AAAI Conference on Artificial Intelligence, 2024.
Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D. Lee, Deming Chen, and Tri Dao. Medusa: Simple IIm inference acceleration framework with multiple decoding heads. In arXiv:2401.10774 2024.
Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre, and John Jumper. Accelerating large language model decoding with speculative sampling. In 2xXi::3302.013188 2023.
Qiguang Chen, Libo Qin, Jinhao Liu, Dengyun Peng, Jiannan Guan, Peng Wang, Mengkang Hu, Yuhang Zhou, Te Gao, and Wanxiang Che. Towards reasoning era: A survey of long chain-of- thought for reasoning large language models. In arXiv:2503.09560 2025.
Xiaoshu Chen, Sihang Zhou, Ke Liang, and Xinwang Liu. Distilling reasoning ability from large language models with adaptive thinking. In arxivv2404.09170, 2024.
Jeffrey Cheng and Benjamin Van Durme, Compressed chain of thought: Efficient reasoning through dense representations. In 2rXiv:24/2.131177 2024.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Rejchiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. In arXiv:2/ 10. 14168, 2021.
11

---

Published as a conference paper at ICLR 2026
DeepSeek-AL, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, al. Deepseek- rl: Incentivizing reasoning capability in Ilms via reinforcement learning. In 12X9::2501.12948, 2025.
Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. GPTQ: Accurate post-training quantization for generative pre-trained transformers. arXiv:2210.17323, 2022.
Yichao Fu, Peter Bailis, Ion Stoica, and Hao Zhang. Break the sequential dependency of Ilm inference using lookahead decoding. In arXiv:2402.022057 2024.
Tingxu Han, Chunrong Fang, Shiyu Zhao, Shiqing Ma, Zhenyu Chen, and Zhenting Wang. Token- budget-aware Ilm reasoning. In arXiv:2412.18857, 2024.
Jitai Hao, Yuke Zhu, Tian Wang, Jun Yu, Xin Xin, Bo Zheng, Zhaochun Ren, and Sheng Guo. OmniKV: Dynamic context selection for efficient long-context llms. In Proc. Int. Conf. Learning Representations, 2025.
Shibo Hao, Sainbayar Sukhbaatar, Dilia Su, Xian Li, Zhiting Hu, Jason Weston, and Yuandong Tian. Training large language models to reason in continuous latent space. In arXiv:2412.06769 2024.
Zhenyu He, Zexuan Zhong, Tianle Cai, Jason Lee, and Di He. REST: retrieval-based speculative decoding. In NAACL, 2024.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In Joaquin Vanschoren and Sai-Kit Yeung (eds.). Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks, 2021.
Yuxuan Hu, Ke Wang, Xiaokang Zhang, Fanjin Zhang, Cuiping Li, Hong Chen, and Jing Zhang. Sam decoding: Speculative decoding via suffix automaton. In ACL, 2025.
Zongle Huang, Lei Zhu, Zong Yuan Zhan, Ting Hu, Weikai Mao, Xianzhi Yu, Yongpan Liu, and Tianyu Zhang. MoESD: Unveil speculative decoding's 's potential for accelerating sparse moe. In Advances in Neural Information Processing Systems, 2025.
Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodehench: Holistic and contamination free evaluation of large language models for code. In Proc. Int. Conf. Learning Representations, 2025.
Ayeong Lee, Ethan Che, and Tianyi Peng. How well do 1lms compress their own chain-of- thainhhf-thoughh?? a token complexity approach. In arXiv:2503.01 141, 2025.
Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In Proc. Int. Conf. Machine Learning. 2023.
Yiwei Li, Jiayi Shi, Shaoxiong Feng, Peiwen Yuan, Xinglin Wang, Yueqi Zhang, Ji Zhang, Chuyi Tan, Boyuan Pan, Yao Hu, and Kan Li. Speculative decoding for multi-sample inference. In arXiv:2503.05330, 2025a.
Yuetai Li, Xiang Yue, Zhangehen Xu, Fengging Jiang, Luyao Niu, Bill Yuchen Lin, Bhaskar Ramasubramanian, and Radha Poovendran. Small models struggle to learn from strong reasoners. In arXiv:2502. /2/ 143, 2025b.
Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. Eagle-2: guage models with dynamic draft trees. In Proc. Conf. Empirical Methods in Natural Language Processing, 2024a.
Faster inference of lan-
Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. Eagle: Speculative sampling requires rethinking feature uncertainty. In Proc. Int. Conf. Machine Learning, 2024b.
Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. Eagle-3: Scaling up inference accelera- tion of large language models via training-time test. In arXiv:2503.0/884, 2025c.
Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser. Rémi Leblond, et al. Competition-lewa code generation with alphac.ee. In arXiv:2203.078/4 2022.
12

---

Published as a conference paper at ICLR 2026
Guosheng Liang, Longguang Zhong, Ziyi Yang, and Xiaojun Quan. Thinkswitcher: When to think hard, when to think fast. In arXii:2505.14418: 2025.
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In arXiv:2305.20050, 2023.
Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. AWQ: Activation-aware weight quantization for on-device Ilm compression and acceleration. In Proceedings of Machine Learning and Systems, volume 6, pp. 87-100, 2024.
Qingwen Lin, Boyan Xu, Guimin Hu, Zijian Li, Zhifeng Hao, Keli Zhang, and Ruichu Cai. Cmets: A constrained monte carlo search framework for mathematical reasoning in large language model, 2025a.
Weizhe Lin, Xing Li, Zhiyuan Yang, Xiaojin Fu, Hui-Ling Zhen, Yaoyuan Wang, Xianzhi Yu, Wulong Liu, Xiaosong Li, and Mingxuan Yuan. Trimr: Verifier-based training-free thinking compression for efficient test-time scaling, In arXiv:2505.1715: 2025b.
Ryan Liu, Jiayi Geng, Addison J. Wu, Ili Sucholutsky, Tania Lombrozo, and Thomas L. Griffiths. Mind your step (by step): Chain-of-thought can reduce performance on tasks where thinking makes humans worse. In arXiv:2410.21333 2024a.
Tianyu Liu, Qitan Lv, Hao Li, Xing Gao, and Xiao Sun. Logitspee: Accelerating retrieval-based speculative decoding via next next token speculation. In arXiv:2507.01444, 2025.
Zirui Liu, Jiayi Yuan, Hongye Jin, Shaochen Zhong, Zhaozhuo Xu, Vladimir Braverman, Beidi Chen, and Xia Hu. KIVI: A tuning-free free asymmetric 2bit quantization for kv cache. In arXiv:2402.02750, 2024b.
Chenwei Lou, Zewei Sun, Xinnian Liang, Meng Qu, Wei Shen, Wenqi Wang, Yuntao Li, Qingping Yang, and Shuangzhi Wu. Adacot: Pareto-optimal adaptive chain-of of-thuughtt triggering via reinforcement learning. In arriiv:505./1180 2025.
Xianzhen Luo, Yixuan Wang, Qingfu Zhu, Zhiming Zhang, Xuanyu Zhang, Qing Yang, and Dongliang Xu. Turning trash into treasure: Accelerating inference of large language models with token recycling. In ACL, 2025.
Wenjie Ma, Jingxuan He, Charlie Snell, Tyler Griggs, Sewon Min, and Matei Zaharia. Reasoning models can effective without thinking. arXiv:2504.09858, 2025.
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 2023.
Meta. Efficient speculative decoding for llama at scale: Challenges and solutions. In wwXiv:2508.08122 2025.
Ramesh Nallapati, Bowen Zhou, Cicero Nogueira dos santos, Caglar Gulcehre, and Bing Xiang. Abstractive text summarization using requence-to-sequenee to-sequence mns and beyond. In 100.0v:602.00033. 2016.
Gabriele Oliaro, Zhihao Jia, Daniel Campos, and Aurick Qiao. Suffixdecoding: Extreme speculative decoding for emerging ai applications. In arXiv:241 1. 1049955, 2024.
OpenAL, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, et al. Openai ol system card. In arXiv:2412.16722 16720, 2024.
Rui Pan, Yinwei Dai. Zhihao Zhang, Gabriele Oliaro, Zhihao Jia, and Ravi Netravali. Specreason: Fast and accurate inference-time compute via speculative reasoning. In arXiv:2504.07891 2025.
13

---

Published as a conference paper at ICLR 2026
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. Gpqa: A graduate-leve google-proot proof Q&A benchmark, 2023.
arxiv. org/ abs//ab//200 2021b//22222
Ranajoy Sadhukhan, Jian Chen, Zhuoming Chen, Vashisth Tiwari, Ruihang Lai, Jinyuan Shi, Ian En- Hsu Yen, Avner May, Tianqi Chen, and Beidi Chen. MagicDec: Breaking the latency-throughpit tradeoff for long context generation with speculative decoding. In arXiv:2408. 1009, 2025.
Apoory Saxena. Prompt lookup decoding, November 2023. URL https://github com/
apoorvumang/prongt http://rrmmt--ockuuupppo www.--oooccccdiiineee--a decoding/
Amrith Setlur, Matthew YR Yang, Charlie Snell, Jeremy Greer, Ian Wu, Virginia Smith, Max Simchowitz, and Aviral Kumar. E3: Learning to explore enables extrapolation of test-time compute 2025.
for llms. In arXiv:2506.00900
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute optimally can be more effective than scaling model parameters. In Proc. Int. Conf. Learning Representations, 2025.
Woomin Song, Saket Dingliwal, Sai Muralidhar Jayanthi, Bhavana Ganesh, Jinwoo Shin, Aram Galstyan, and Sravan Babu Bodapati. Accelerated test-time scaling with model-free speculative sampling. In arXiv:2506.0444 04708, 2025.
Gaurav Srivastava, Shuxiang Cao, and Xuan Wang. Towards reasoning ability of small language models. In wwxxvv2502.//556 2025.
Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Na Zou, Hanjie Chen, and Xia Hu Stop overthinking: A survey on efficient reasoning for large language models. In arXiv:2503. 16419, 2025.
Jiankai Sun, Chuanyang Zheng, Enze Xie, Zhengying Liu, Ruihang Chu, Jianing Qiu, Jiaqi Xu, Mingyu Ding, Hongyang Li, Mengzhe Geng, and et al. A survey of reasoning with foundation models. In arXiv: 1562, 2023.
Lin Sun, Guangxiang Zhao, Xiaoqi Jian, Yuhan Wu, Weihong Lin, Yongfu Zhu, Linglin Zhang, Jinzhu Wu, Junfeng Ran, Sai er Hu, and al. EnyRI-32B-Previne Boosting accuracy with branch-merge distillation. In arXiv:2503.04872. 2025.
Shengyin Sun, Yiming Li, Renxi Liu, Weizhe Lin, Hui-Ling Zhen, Xianzhi Yu, Mingxuan Yuan, and Chen Ma. Revisiting judge decoding from first principles via training- free distributional divergence. In arXiv:2601.04700 2026.
Xiaoyu Tian, Sitong Zhao, Haotian Wang, Shuaiting Chen, Yunjie Ji, Yiping Peng. Han Zhao, and Xiangang Li. Think twice: Enhancing LLM reasoning by scaling multi-round test-time thinking. In arXiv:2503. 19855, 2025.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In Proc. Int. Conf. Learning Representations, 2023.
Yibo Wang, Li Shen, Huanjin Yao, Tiansheng Huang, Rui Liu, Naiqiang Tan, Jiaxing Huang, Kai Zhang, and Dacheng Tao. R1-compress: Long chain-of-thought compression via chunk compression and search. In arXiv:2505.1683 16838, 2025a.
Zili Wang, Tianyu Zhang, Haoli Bai, Lu Hou, Xianzhi Yu, Wulong Liu, Shiming Xiang, and Lei Zhu. Faster and better Ilms via latency-aware test-time scaling. In arXiv:2505.19634 2025b.
Han WU, Yuxuan Yao, Shuqi Liu, Zehua Liu, Xiaojin Fu, Xiongwei Han, Xing Li, Hui-Ling Zhen, Tao Zhong, and Mingxuan Yuan. Unlocking efficient long-to- to-sort 11m reasoning with model merging. In arXiv:2503.20641 2025.
Heming Xia, Zhe Yang. Qingxiu Dong, Peiyi Wang, Yonggi Li, Tao Ge, Tianyu Liu, Wenjie Li, and Zhifang Sui. Unlocking efficiency in large language model inference: comprehensive survey of speculative decoding. In ACI. 2024.
14

---

Published as a conference paper at ICLR 2026
Zeyu Xing, Xing Li, Hui-ling Zhen, Xianzhi Yu, Mingxuan Yuan, and Sinno Jialin Pan. Large reasoning models know how to think efficiently. In ES-FoMo III: 3rd Workshop on Efficient Systems for Foundation Models, 2025.
Siyuan Yan, Mo Zhu, Guo qing Jiang, Jianfei Wang, Jiaxing Chen, Wentai Zhang, Xiang Liao, Xiao Cui, Chen Zhang, Zhuoran Song, and Ran Zhu. Scaling laws for speculative decoding. In 0tti::2505007858. 2025.
Yang, Anfeng Li, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, et al. Qwen3 technical report. In wwxXi:2500509388 2025.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In Advances in Neural Information Processing Systems, 2023.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, and et al. DAPO: An open-source Ilm reinforcement learning system at scale. In arXiv:2503. 14476, 2025a.
Zhouliang Yu, Yuhuan Yuan, Tim Z. Xiao, Fuxiang Frank Xia, Jie Fu, Ge Zhang, Ge Lin, and Weiyang Liu. Generating symbolic world models via test-time scaling of large language models. In arXiv:2502.04728, 2025b.
Lifan Yuan, Ganqu Cui, Hanbin Wang, Ning Ding, Xingyao Wang, Boji Shan, Zeyuan Liu, Jia Deng, Huimin Chen, Ruobing Xie, Yankai Lin, Zhenghao Liu, Bowen Zhou, Hao Peng, Zhiyua Liu, and Maosong Sun. Advancing I1m reasoning generalists with preference trees. In Proc. Int. Conf. Learning Representations, 2025.
Zhihang Yuan, Yuzhang Shang, Yang Zhou, Zhen Dong, Zhe Zhou, Chenhao Xue, Bingzhe Wu, Zhikai Li, Qingyi Gu, Yong Jae Lee, Yan Yan, Beidi Chen, Guangyu Sun, and Kurt Keutzer. Llm inference unveiled: Survey and roofline model insights. arXiv:2402.16363 2024.
Yu Yue, Yufeng Yuan, Qiying Yu. Xiaochen Zuo, Ruofei Zhu, Wenyuan Xu, Jiaze Chen, Chengyi Wang, Tiantian Fan, Zhengyin Du, Xiangpeng Wei, Gaohong Liu, Juncai Liu, Lingjun Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Ru Zhang, Xin Liu, Mingxuan Wang, Yonghui Wu, and Lin Yan. VAPO: Efficient and reliable reinforcement learning 2025.
for advanced reasoning tasks. In arXiv:2504.05/1
Qiyuan Zhang, Fuyuan Lyu, 7exu Sun, Lei Wang, Weixu Zhang, Wenyue Hua, Haolun Wu, Zhihan Guo, Yufei Wang, Niklas Muennighoff, Irwin King, Xue Liu, and Chen Ma. A survey on test-time scaling in large language models: What how, where, and how well? In arXiv:2503.242335 2025,
Yao Zhao, Zhitian Xie, Chen Liang, Chenyi Zhuang, and Jinjie Gu. Lookahead: An inference acceleration framework for large language model with lossless generation accuracy. In Proc. ACM Conf. Knowledge Discovery and Data Mining, 2024.
15

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 ADDITIONAL RELATED WORK
A.1.1 TRAINING-FREE FRAMEWORKS FOR TEST-TIME SCALING
Models can acquire reasoning abilities through specialized training that teaches them to generate long chains of thought internally (Chen et al., [025] [un et al.] 2023] Yu al., 2025a Yue et al. 2025). Complementing these trained capabilities are external test-time scaling frameworks. These techniques implement an outer control loop around base model to enhance its reasoning performance without altering the model's parameters (Yu et al.) 2025b Yuan al. 2025 Setlur et al., 2025). The advantage of these external methods lies in their universal applicability: they can augment any model's 's performance, including those that already incorporate built-in reasoning capabilities. This work examines two primary categories of external training-free scaling approaches:
The most prominent approach in this category is BoN sampling (Snell ct al., 2025), where model generates N candidate outputs, and verifier or the model itself selects the most plausible one. A more sophisticated variant, Self-Consistency (Wang et al. 2023). generates multiple reasoning paths and selects the final answer by majority vote, demonstrating significant improvements in arithmetic and commonsense reasoning. These methods operate on the principle that model is more likely to generate correct reasoning path than an incorrect one, and sampling multiple paths increases the probability of finding a valid solution.
Sampling-Based Methods:
This class of methods involves a multi-step process where the model progressively improves its own output. For instance, Self-Refine (Madaan et al. 2023) enables model to generate an initial response, critique it, and then use that feedback to produce a refined answer in subsequent pass. A more streamlined approach is multi-round thinking (Tian et al.) 2025), which simplifies the feedback loop. Instead of generating an explicit critique, it provides only the final answer from the previous turn as context, prompting the model to re-evaluate its conclusion. This method is designed to break "cognitive inertia" by discarding the prior reasoning chain, forcing the model to approach the problem anew.
Iterative Refinement:
Other search algorithms, such Tree-of-Thoughts (ToT) YYao 2023). Graph-of-Thoughh (Besta et al. 2024), and those based on Monte-Carlo Tree Search (MCTS) (Lin et al.] 2025a). can be understood sophisticated hybrids of these foundational principles. For instance, ToT explores multiple candidate reasoning steps (a form of sampling) and evaluates them to construct a solution tree, while MCTS iteratively builds and refines a search policy. By combining multi-candidate generation with progressive construction and evaluation, they represent a powerful but more complex class of techniques built upon the core ideas of sampling and refinement.
A.1.2 EFFICIENT REASONING
While proven effective for unlocking the advanced reasoning capabilities of LLMs, test-time scaling paradigms are inherently resource-intensive. This significant increase in computational overhead and inference latency presents a substantial challenge to their practical deployment in real-world, time-sensitive applications. Efficient reasoning offers promising path to alleviate this burden by generating smarter, more concise reasoning with less computation (Wu et al. 2025 Sui et al.) 2025 Xing al., 2025). Research in this area can be organized into three main categories. (i) Model-basce efficient reasoning (Wang et al. [2025a] Lou et al. 2025), which aims to optimize full-length reasoning models into more concise variants or directly train compact models specialized for fast and effective reasoning. (ii) Reasoning output-based efficient reasoning (Hao et al., 2024| Pan et al., 2025 Lin
al. 20256), which dynamically reduces the computational work during inference by adaptively shortening the reasoning chain and eliminating unnecessary steps. (iii) Input prompts-based efficient reasoning (Ma et al. 2025 Liang et al., 2025), which enhances efficiency by tailoring the reasoning process according to the properties of the input, such its specified length constraints.
However, the speculative decoding methods for accelerating test-time scaling examined in this paper are fundamentally orthogonal to the aforementioned efficient reasoning methodologies (Liu et al. 2024a), representing a distinctly different paradigm for performance optimization. Recent studies have also explored speculative approaches within the context of test-time scaling, including but not limited to speculative acceleration analyses for test-time scaling (Wang et al.) (025b), logit- N-gram speculation (Song et al. 2025). and multi-sample inference speculation (Li et al. 2025a).
16

---

Published as a conference paper at ICLR 2026
However, unified and standardized benchmark for evaluating speculative decoding within the test- time scaling setting is still lacking, which precisely constitutes the focus of our work. Fundamentally, speculative decoding constitutes a lossless acceleration technique that preserves the original model's output distribution exactly, ensuring that both the quality and inherent characteristics of generated reasoning remain completely unchanged throughout the acceleration process. This preservation of distributional fidelity stands in stark contrast to efficient reasoning approaches, which typically achieve speedup by introducing modifications the reasoning process itself, whether through aggressive
model quantization that reduces numerical precision (Frantar et al. 2022, Lin et al., 20244] Hao et al.
2025 Liu al. 2024b), output compression techniques that sacrifice detail for brevity (Cheng & Durme 2024 Hao et al. 2024), structured prompt engineering that constrains the reasoning pathway (Han et al. 2024 Lee et al. [025| Aytes et al. 2025), or architectural modifications that streamline computational flow (Li et al. 2025b Srivastava et al. 20255] Sun et al., 2025; Chen et al. (024). More critically, speculative decoding operates exclusively at the inference execution level without fundamentally altering the underlying reasoning methodology, maintaining complete fidelity to the original model's cognitive processes, decision-making patterns, and output generation mechanisms. This approach ensures that aspect of the original model's 's performance characteristics, reasoning depth, or solution quality is compromised or sacrificed in pursuit of computational efficiency, making it particularly attractive acceleration strategy for scenarios where maintaining exact model behavior is paramount.
A.2 IMPLEMENTATION DETAILS
All experiments are conducted on a server equipped with 16-core CPU, utilizing Py Torch 2.1.1 with CUDA 12.1. The overall evaluation pipeline follows the previous work (Xia et al. 2024). For greedy decoding with temperature T = 0, we employ exact match verification between the draft model and target model outputs ensure generation consistency. For sampling scenarios where temperature T + O, we utilize speculative sampling (Chen et al. 2023 for verification, which maintains mathe- matically proven identical output distributions to the target model. Given that reasoning models may generate extremely long outputs during their internal deliberation process, we implement specific constraints to ensure practical evaluation while preserving the models' reasoning capabilities. We set the maximum output token limit to 15,000 tokens and establish a thinking budget mechanism that allows up to 80% of the maximum token allocation for internal reasoning processes. For the DeepSeek-RR S-1-DistillLLaame designated termination tokens /nnnn</ttt ated token to compel the model to conclude its reasoning phase and provide the final response. For the Qwen3 series models, we employ a consistent termination strategy by inserting the official stop- thinking instruction starting from the 12.000th generated token, following the protocol specified in their technical report (Yang et al. 2025). Specifically, we employ the designated stop-thinking instruc- tion, i.e., Cons i dddring the Timi ted time by the user, have to give the so ut on based on the thinking di ly now, /n</think> \n)nn to grace- fully terminate the reasoning phase. Additionally, the detailed information of the open-source trained models used in the experiments is shown in Table
model, we implement a forced thinking termination strategy where \n\y n are inserted starting from the 12,000th gener-
Model
Type
DeepSeck RR--i---DD-LLaae. Disail-Llam--51
Target Model
htter 1eta:1//hoe mttarr//ugciittee co co/ Sttpeetssseeeeee--------0--
Qweni-48 4B
larget Model
814 254 57 Hegg 1in 100 207 Vasi. Weei 4B
Qwen3-8R
Tager Model
12 29 hi-g
Owen3 140
Taget Model
444 20g
Oweni-0.68
Draft Modsl
Usens 3,36
EAGE E3 EDDDeppecc-RR B.n
Draft Model
-D1--Dist--.. 1 -11aM6-
EXGLE Qwen3 4B
Draft Model
Draft Model
EXGLLE3-QeaB--148
Draft Model
BLL 27/./////gga.ooo ing Lale 200 Any
Table 5: Information about the open source trained models.
A.3 FURTHER ANALYSIS
In this section, we further analyze speculative decoding in terms of computational overhead of key modules, the potential of hybrid strategies, performance models of different scales, acceleration across reasoning turns, and visualization of inference acceleration process, offering insights into its practical efficiency and limitations.
17

---

Published as a conference paper at ICLR 2026
Potential analysis of hybrid speculative decoding. We present the average accept length across different suffix matching lengths and temperature settings in Figure The results indicate that SAM demonstrates strong potential when the matching length exceeds 10. For instance, under greedy decoding (7' = 0). SAM achieves an average accept length greater than 9 at a suffix matching length of 11, which nearly twice the average accept length of EAGLE-3. However, we also observe notable fluctuations in SAM's average accept length as the matching suffix length increases. This sensitivity suggests that the performance of speculative decoding can vary significantly depending on the hybrid strategy (based on the matched suffix length). It is promising to design more refined hybrid strategies to unlock SAM's potential for onn-sequencc matching test-time scaling, which will further improve the performance of the hybrid speculative decoding.
SAM Recycling
EAGLE-
SAM Recycling
EAGLE-S Rocyding
SAM EAGLES
Recycling
Kome Aecooe 1 Lengn
BAGLE-S
SAM
SAM
18
Matching Length
Masching Length
(a)
(b)
Figure 3: Average accept length under different suffix matching lengths across various temperature T.
(a) T = 0. (b) T 0.6.
Takeaway 7: SAM's capability to capture redundant patterns in the test-time scaling paradigm remains underexploited in hybrid speculative decoding, highlighting the need for more refined and dynamic hybrid strategies to fully unlock its potential.
Time consumption proportions across modules in speculative decoding. Figure 4 iillustrates the proportion of time consumed by each stage within PIA, EAGLE-3, Recycling, and SAM. During draft generation, the model produces sequence of tokens based on the current state. The decoding stage processes this draft sequence to compute representations and logits for each position. In the verification phase, the model determines which tokens in the draft are correct by comparing predicted probabilities given by the target model. Finally, the updating stage accepts the verified correct tokens and updates the model state accordingly, preparing for the next iteration. As shown in the Figure 4 PIA achieves the lowest proportion of draft generation time, which benefits from its pruning strategy for the N-gram draft tree. In contrast, EAGLE-3 exhibits the highest draft generation time proportion, reaching up 35.7%, as it requires invoking an additional draft model that is more time-consuming than N N-gram methods. Recycling demonstrates the highest update time proportion among all methods, attributed to its need to dynamically maintain top-k next token list throughout the generation process. For SAM, the majority of time consumption is concentrated in the target model's 's decoding phase,
Verify
Update
Dmai
Verify
Updaic
PIA
PSA
EAGLE-S
11.7%
EAGLE-3
Recycling
Recycling
73.%%
Percestage
Percestage
(a)
(b)
Figure 4: Analysis of time consumption proportions across modules in speculative decoding with different model sizes. (a) QW3-8B. (b) QW3-14B.
18

---

Published as a conference paper at ICLR 2026
Table 6: Performance comparison of speculative decoding methods on reasoning models with different
scales across various temperature T Best Second Best
Beuch
AIME24
AIME25
MATH500
GPQA
Overall
Model
Method
MAT
Speed
MAT
Speed
MAT
Speed
MAT
Speed
MAT
Speed
Under the Multi-tura Thinking Paradigm
AR
1.00
1,00x
1.00
1.00x
1.00
1.0x
1.00
1.00x
1.00
1.00x
EAGLE-3
3,45
2.28%
3.48
2.22%
3.54
2,38x
3.31
2.20x
3,44
2.27%
SAMIEAGLE-3
3,40
2.37x
3.40
2.28x
3.50
2.55x
3,47
2.39%
3,43
2,40
QW3-4R
SAM
1.99
1.92x
1.94
1.95
1.82x
2.19
x.99
2.01
1.89
(T' - 0.6)
Recycling
REST
2.92
2.20
1.12%
2.91
2.11X
2.92
2.38x
1.15%
2.94
1.35
2,25%
1.04%
2.92
1.40
2.24x
1.10x
PIA
1.71
1.45%
1.70
1.39x
1.69
1.42x
1.78
1.46x
1.72
1.43x
AR
1.00
1.00x
1.00
1.00x
1,00
1.00x
1.00
1.00x
1.00
1.00x
EAGLES3
3.28
2.26x
3.38
2.31
3.36
2.27%
3.05
2.10%
3.28
2.23%
QW3 14B
SAMIEAGLE-3J
2.24x
2.29x
2.31
3.12
2.15%
3.21
(2' = 0.6)
SAM
1.84
1.70X
1,86
1.72X
1.85
1.66X
1,90
1.73x
1,86
1.70x
Recycling
2.10%
2.08x
2.89
2.17x
2.13%
2.86
2.12x
REST
1.41
1.17x
1.40
1.17%
1.42
1.17x
1.33
1.11X
1.39
1.15%
SpS
5.78
1.22
5.96
1.25x
5.73
L1IX
4.59
1.02X
5.59
1.14x
PIA
1.62
1.43x
1.69
1.44x
1.66
1.45x
1.69
1.46%
1.66
1.45%
Under
Poradigm
AR
1.00
1.00x
1.00
1.00x
1.00
1.00x
1.00
1.00%
1.00
1.00
EAGLES3
2.24%
2.23%
3.52
2.31
3.33
2.11x
3.44
2.22x
SAMIEAGLE-3]
3.37
2.37x
3.37
2.33x
3.58
2,61
3.38
2.36%
3.40
2.42x
SAM
2.07
2.32
2.07x
2.18
2,00x
QW3-1B
Rocycling
2.92
2.14x
2.91
2.11x
2.92
2,31x
2.96
2.20x
2.93
2.19%
(T = 0.6)
REST
1.42
1.11x
1
1.44
1.16x
1.38
1.03%
1.41
1.10x
PIA
1.70
1.40%
1.72
1.41x
1.75
1.50x
1.83
1.51
1.74
1.46x
AR
1.00
1.00x
1.00x
1.00
1.00x
1.00
1.00x
1.00
1.00x
EAGLES3
3.28
2.27%
3.31
2.27%
3.33
2.27x
3.04
2.07x
3.24
2.22%
SAMIEAGLE-3!
3,21
2.33x
3.22
2.34x
3.35
2.47
3.10
2.20x
3.21
2.33x
QW3-14B
SAM
1.98
1.84x
1.99
1.87x
2.09
1.93x
2.05
1.85%
2.01
1
(T - 0.6)
Recycling
2,86
2.12x
2.86
2.10x
2.89
2.20x
2.87
2.15%
2.87
2.15x
SpS
5,76
1.22x
5.76
1.24x
5.62
1.09
4.87
1.04%
5,53
1.14x
REST
1.40
1.16x
1.40
1.16x
1.42
1.18x
1.34
1.1X
1.39
1.15x
PIA
1.64
1.42x
1.69
1.44x
1.70
1.46x
1.73
1.45%
1.68
1.45x
which benefits from its advanced suffix automaton data structure. This approach finds the longest suffix that matches the sequence in suffix automaton at each step of the generation with an average O(1) time complexity, while the linear drafting strategy further reduces computational overhead, significantly minimizing the time spent in other components.
Takeaway 8: N-gram- methods, such as SAM, incur lower draft generation time overhead, enabling a greater share of computational resources to be allocated to the decoding phase.
Analysis of speculative decoding on models with different scales. In addition to evaluating Qwen3-8B models, we further investigate the effectiveness of speculative decoding methods across different model scales (as shown in Table 6), including the smaller Qwen3-4B and larger models such as Qwen3-14BB Among them, SAM[EAGLE-3] achieves the best performance across all model sizes, demonstrating the potential of hybrid speculative decoding. We also observe that EAGLE shows slight performance degradation when accelerating 4B/14B models compared to the 8B scenario. This stems from the training process, the EAGLE draft models for 4B/14B come from a different team AngelSlin[ highlighting how training-based speculative decoding methods are highly dependent on the draft model's training quality. Benefiting from the increased model size, we find that the SpS for the 14B model shows significant improvement compared to the 8B model. This is primarily because the 14B target model is proportionally much larger relative the 0.6B draft model than the 8B model is, enabling more effective speculation.
Analysis of overall performance. Figure presents a comprehensive radar chart comparing various speculative decoding methods on the Qwen3-8B model under multi-turn thinking with greedy decoding, displaying both overall MAT and Speedup Ratio across multiple datasets. The most striking observation is that SpS demonstrates exceptional performance with its MAT scores essentially encompassing all other methods, which can be attributed to the high alignment between SpS's s draft
https://hugginggfacs w 1m1im
19

---

Published as a conference paper at ICLR 2026
Moan Accepted
Speedup
MATH5500
MATRESO
GRQA
EAGLE-1
SAM
REST
Becycling
Figure 5: The performance various speculative decoding methods on Qwen3-8B under the multi- turn thinking framework with temperature T O.
model and target model. This comprehensive coverage underscores that there remains substantial untapped potential in model alignment for training- based speculative decoding methods. Meanwhile, our novel hybrid combination SAM[EAGLE-3] exhibits considerable promise, achieving the second- highest MAT performance while securing the highest speedup ratio among all evaluated methods. This dual excellence validates our hypothesis regarding necessity of combining repetition pattern capture through the N-gram-basee methods (SAM component) and semantic alignment through the training- approaches (EAGLE-3 component) in test-time scaling acceleration. The success of SAM[EAGLE- demonstrates that strategically combining different speculative decoding paradigms can yield superior performance compared to individual approaches, leveraging both the efficiency of pattern-base based prediction for repetitive sequences and the semantic understanding of alignment- based methods for complex reasoning. Moreover, our other findings are summrized as follows:
While speculative decoding relies on training an auxiliary draft model demonstrates promis- ing acceleration potential, its performance inherently tied to the training process, raising practical concerns about its adaptability to diverse reasoning scenarios.
Methods leveraging N-gram patterns excel at capturing redundancy within test-time scaling, yet this advantage is accompanied by the sensitivity to sampling temperature.
Hybrid speculative decoding uniquely combines methods that train auxiliary draft model for semantic alignment with those leveraging N-gram patterns to capture repetition, unlocking distinct potential for reasoning under test-time scaling.
Methods that store N- N-gram patterns during generation can achieve progressive accelcration by reusing past computations, enabling efficiency that scales across turns.
In hybrid speculative decoding, the acceleration from reusing repetitive token sequences is limited by suboptimal integration strategies, calling for more dynamic approaches to realize its potential under test-time scaling.
Methods leveraging N- N-gram patterns incur lower draft generation time overhead, enabling greater share of computational resources be allocated to the decoding phase.
Analysis of generalization to other domains. assess the generalizability of our findings, the evaluation is extended to additional domains, including code generation LiveCodeBench (Jain et al.) 2025) and summarization CNN/Daily Mail (Nallapati et al. 2016). The results are given Table can be observed that the code and summarization datasets, SAMIEAGLE-3) consistently achieves the highest speedup across nearly all scenarios. Moreover, the N N-gram-bart based methods can even outperform the training- based method in some cases. For example, both SAM and Recycling achieve higher acceleration ratios than EAGLE-3 when applied to DSL-8B on the BiveCodeBench. These observations consistent with the findings reported before. This indicates that redundancy is not
20

---

Published as a conference paper at ICLR 2026
Table Performance comparison of speculative decoding methods for reasoning models under the
multi-round thinking framework at different temperatures 7 on other domains
Best Second Best
(a) Experiments on LiveCodeBench.
Model
DSL-8B
0)
DSL-8B (7
0.6)
QW3-8B (T
QW3-8B
0.8)
Method
MAT
Speed
MAT
Speed
MAT
Speed
MAT
Speed
EAGLE-3
2.12
1.64x
2,00
1.33%
4.37
288x
4,02
2.58x
SAMIEAGLE-31
4.17
3.34x
2.30
1800
4.46
3.27%
3.70
2,51x
SAM
2.90
2.79x
1.72
1.44x
2.19
1.92x
1.77
1,58x
Recycling
2.91
2,02x
2.69
1,71x
2,90
1.98X
2.83
1.92x
PLD
2.24
1.81x
1.94
1.57x
SpS
5.93
5.16
0.85%
REST
1.33
1.11x
1.37
1,02X
1.35
1.14
1.37
1.09
Lookahead
2.15
1.57x
2,08
1.53x
PIA
1,04x
AR
1.00
1.00x
1.00
1.00%
1.00
1.00x
1.00
1.00x
(b) Experiments on CNN/Daily Mail.
Model
DSL-8B (T' 0)
DSL-8B (T
0.6)
QW3 SB (T'
QW3-8D (T 0.6)
Method
MAI
Speod
MAT
Speed
MAT
Speed
MAT
Speed
EAGLES3
4.76
2.83x
4.60
2.68%
4.11
2.63x
SAMIEAGLE-]]
4.92
3,05x
4.70
2,79%
4.31
2.77%
4.17
2.61x
SAM
1.53
1.45x
1.39
1.33x
1,55
1.4xx
1,52
1.42x
Recyeling
2.50
1.71x
PLD
1.47
1.32x
1.46
1.30x
Sp8
3.57
0.51x
3.44
0.54x
REST
1.29
1.00x
Lookahead
1.59
1.29x
1.61
1.30x
PIA
1.61
1,42x
1,48
1.30%
1.58
1,39X
1.55
1,36x
1.00x
1.00
1.00x
Table 8: Error analysis of Qwen3-8B on the multi-round thinking Best
Second Best
(a) Experiments with Qwen3-8B (I' 0).
Danaset
AIME24
AIME25
MATH500
GPQA
Overall
Mechod
MAT
Speed
MAT
Speed
MAT
Speed
MAT
Speed
MAT
Speed
EAGLESS
431+0000
2.7-0003x
4.300002
287+0001
50-0000
301+003x
4.39-0000
2.000002
38800000
SAMILAGLE-3]
0007
3,43
4.7210.33
3.43
Recycling
299240.0
2:03=000x
2.020002
2.7.00000x
299:0001
2115000x
3.0=0001
2.14=000
30005:001
2:10=6005
1.0::00000
1.00=0000x
1,00=0.00
100+0.000
100:0:00
1.000000
1.00=0000
1.0=00000x
10:+000000
1.00:+00
(b) Experiments with Qwen3-8B (7 0.6).
MATHE500
GPQA
Mechod
MAT
Speed
MAP
Speed
MAI
Speed
MAT
Speed
MXI
Speed
EAGLE-3
4.12:0.01
2.00000X
0.21=0.02
2.71:0.01x
4.34-1022
2.85=000x
4.02=0.02
2.00-0.0%
4.16+0.01
2000000
SAMIENGLE-31 31
18910004
26110006
3.9110011
2.7010.0xx
4 11810.11
3.95
2.7210.9X
3.9200.22
1.7410.85
Recycling
280400011
1.5-=000xx
200-0002
1.97-0006X
200+0001
2.05==006x
2-9-0001
200=600XX
2.00-00.00
2,0000000
AR
1.00:0.000
1.0000.0x
1.0000.00
1.0010.00K
100:0.000
1,00:0.0x
1.00:0.00
1.0:10.000
1000100000
1.00:10000 00x
specific to mathematical reasoning but a universal phenomenon in generation tasks. The consistent superiority of hybrid methods across domains further validates our benchmark's core insights.
Analysis of error bars. We conduct error analysis on representative subset of methods (EAGLE-3, SAM]EAGLE-3J, and Recycling) by running them three times. The results in Table demonstttte that the impact of randomness is minimal, with standard deviations consistently below 0.1 across tested configurations (Qwen3-8B, multi-round thinking).
Analysis of varying batch sizes. As our work primarily serves as a benchmark, to ensure objectivity and fairness, we intentionally avoid altering the logic of the evaluated methods' original implementations. However, many existing codes are not well-suited for batched inference. For example, the official implementation EAGLE-3 does not support batched inference; the SAM method's suffix automaton does not support batched writing support batched adjacency matrix construction. Given these architectural constraints, implementing varying batch sizes would require significant modifications to the original codebases. We maintain that our bingle-batch results serve as a reliable proxy for performance in moderate- batch regimes due to the memory-bout -ound nature of L.LM decoding. Speculative decoding improves hardware utilization by
lookup; and Recycling does not
21

---

Published as a conference paper at ICLR 2026
Table 9: Performance comparison with varying batch sizes on DSL-8B ( Best
Second Best
(a) Batch Size =
Dataset
AIME24
AIME25
MATH1500
GPQA
Method
MAT
Speed
MAT
Speed
MAT
Speed
MAT
Speed
EAGLE-3
3.30
1.52x
3.34
1,56x
3.10
1.46%
3.08
1.41x
SAM-Single
1.81
1.33x
1.87
1.44X
2.91
2.72%
2.01
1.58x
SAM-Cross
186
1.34%
2.11
1.62x
3.19
2.89%
2.21
1.73x
(b) Batch Size = 8
Datasel
AIME24
AIME25
MATH1500
GPQA
Method
MAT
Speed
MAT
Speed
MAI
Speed
MXI
Speed
EAGLEE3
3,34
1.1x
3.37
1.88
3.13
1.11X
3,06
1.03x
SAM Single
SAM-Single
1.75
1.06x
1.11
1.8X
2.52
2.24%
1.92
1.36x
5AM-Croes
L82
112X
192
1,20X
3.12
2.61x
1.99
1.34%
verifying multiple tokens in single forward pass, thereby reducing the frequency of memory accesses and overall latency. From a roofline model perspective (Yan et al., 2025 Yuan et al., 2024), while extremely large batch sizes may eventually shift the bottleneck to computation where verification overhead could negate gains, increasing the batch size within the memory-bound region before reaching the compute-bound consistently yields inference acceleration. Crucially, for N-gram-based methods, the cost of draft generation is negligible, making the speedup highly positively correlated with the MAT. Consequently, the high MAT observed in our single-bate batch experiments serves as a strong indicator of potential speedups in batched scenarios. We focused on single- baghe-batchha inference to isolate these intrinsic algorithmic capabilities from the uneven system-level engineering (Meta) 2025
Sadhukhan et al., 2025) required to optimize batched execution across different baselines. Despite these challenges, we implement preliminary batched versions of EAGLE-3 3 and SAM for exploration under the BoN scenario. For SAM, to enable batched inference, we modify the suffix automaton's 's query mechanism: instead of maintaining a single shared automaton, we construct one for each reasoning path, and draft tokens across branches are cross-looked-up for matches. While this design makes batching possible, it inevitably increases the query complexity beyond O(1), representing a necessary compromise under the current framework. We test BoN with N == 4 and N = on subset of the four reasoning tasks using DSL-8B, and the results are shown in Table (where SAM-Single the original implementation using a single automaton per reasoning path, and SAM-Cross the modified version that supports cross-branch token look-ups). The results suggest that increasing the batch size indeed tends to trigger compute-bous bound behavior. For instance, EAGLE-3 achieves an average speedup of around 1.5 1.5x at batch size 4, which is notably lower than its MAT and, in some cases, even slightly behind the N-gram-basse based method SAM. This observation consistent with our earlier findings, indicating that redundancy plays crucial role in test-time scaling acceleration. In practice, when the batch size increases, the speculative size typically needs to be reduced to avoid becoming compute- compute-boun;; the observed slowdown is therefore likely due to the current testing configuration, which does not carefully calibrate this trade-off. In addition, the parallelization of BoN inherently reduces overlapping redundancy across different inference paths, which further explains the diminishing speedup. This reinforces our conclusion that redundancy should be preserved and fully utilized in speculative decoding. Our benchmark therefore substantiates a key practical insight that BoN should be applied along the sequential dimension, while the batch dimension should be reserved for handling different queries maximize throughput (as leaving the available redundancy unexploited would be an unacceptable waste).
Analysis of acceleration across turns. Tables 13 present detailed acceleration ratios and MAT values across different turns for various model sizes under different test-time scaling paradigms, serving as a complement to Section 4.2.6 The data reveals a particularly encouraging pattern for methods capable of caching historical generations, such as SAM and PIA, which demonstrate increasing acceleration ratios as the number of turns progresses. This scaling up and speeding up trend represents a crucial advantage for test-time scaling acceleration, as it suggests that the computational benefits compound over longer reasoning sequences. The ability to leverage previously generated content not only improves efficiency in individual turns but creates cumulative effect where each subsequent turn becomes progressively faster, making these eaching-enables approaches especially valuable for complex multi-turn reasoning tasks that require extended computational processes.
22

---

Published as a conference paper at ICLR 2026
Table 10: Performance comparison of speculative decoding methods across turns under the multi- round thinking framework with different temperature ''' on DSL-8B and QW3-8B,
Bench
AIME24
AIME25
MATH500
GIQA
Overall
Model
Method
Tum
MAT
Speed
MAT
Speed
MAT
Speed
MAT
Speed
MAT
Speed
AR
Tumn
Tum
1.00
1.00
1.00x
1.00
1.00
1.00
1,00x
1,00X
1.00
1,00%
1,00x
1.00
1.00
1.00x
1.00x
1.00
1.00
1.00x
1.00%
EAGLE-3
Turn
Tum
2.12
2.39
1.70x
1.00x
2.30
2.18
1.51x
1.79x
2,68
2.60
3,27
2,83x
2.59
2.41
2.55x
1.60x
2.40
2.27
2.41x
1.4xx
SAMTEAGLE-31
Tim
3.47
2.71x
4.34
3.50%
4.28
6.91
x
4.44
3.71x
Tum
4.61
4.18x
5.53x
5.22
7.17
4.25x
5.17
4.79%
SAM
Turn
Tum
2.38
3.04
x
2.24x
3.81
2.64
3.89x
2.51x
2.25
3.26
3.60x
1.86%
3.48
3.61
2.75%
3.31x
3.38
2.67
3.35%
2.47%
DSL-88
Rocycling
Tum
295
2.11x
2.96
x
2.96
2.15x
3.04
2.09
2.97
x.09
(T' = ()
Tum
3.03
x.07x
3.02
2.15%
2.99
2.24%
2.99
2.11%
3.01
2.15x
PLD
Tum
2.10
1.76x
2.23
1.75x
2.18
1.57%
2.60
1.97%
2.255
1.76x
2.19
1.80x
2.06x
2.11%
2.75
2.44
1.98%
REST
Tum Tum
1.32
1.34
1
x
1.35
1.34
0,96x
0.96x
1.38
1.35
1.09
1.08
1.31
1.31
1.01x
1.02
1.34
1.34
1.02X
1.02
Lookahead
Tum
2.19
x
2.24
1.52x
2.26
1.53x
2.47
1.64x
2.28
1.57%
Turn
2.24
x.58
2:29
1.51x
2.30
1.59%
2.38
1.58
2.29
1.56x
PIA
Tumn
1.62
1.38x
1.70
1.41%
1.68
1.28
2.01
1.51%
1:74
1.38x
Tum
2,36
1,84x
2.38
2.15x
2.47
2.33%
2:44
1.81x
2.32
2,04K
Tur
1.00
1.00
1.00x
1.00x
1.00
1:00x
1.00
1.00x
AR
Tum 2
1.00
1.00x
1.00
1.00%
1,00
1.00x
1.00
1.00%
1.00
1.00x
EAGLE-3
Tum
2.26
1.40k
2.22
1.45%
3,42
3,20x
2.39
1.63%
2.36
1.98%
Tum
2.16
1.47x
L.70x
3.15
2.56
1.97x
2.30
2.12%
SAM[EAGLE-3]
Tum
Tur
2.51
2.58
1.76x
2.08
2.42
2.55
2.16x
1
3.35
3.21
4.06x
3.03
3.12
2.72
2.08%
2.52%
2.60
2.72
2.76x
2.23x
DSL-8B
SAM
Tur
1.73
1.52x
1.73
1.53x
1.74
1.50x
1.85
1.68%
1.76
1.56x
(T' = 0.6)
Tumn
202
1.99
1.98
1.89X
2.13
2.38
2.08
1.81x
2.03
2.02%
Recycling
Tumn
282
1,88x
2,80
1.91x
2.84
2,02
2.78
1.977
22.81
1.95%
Tum
2.86
2.00x
2.87
1,96x
2.91
2.09
2.86
1.98x
2.87
2.01
REST
Tum
Tur
133
1.35
0.99
0.99
1.37
1.37
0.99x
0.99x
1.40
1.03x
1.36
1.36
0.99
0.99
1.37
1.37
1.00x
1.00x
PIA
Tum
Tum
1.73
147
1.41x
1.18x
1.47
1.44%
1.20x
1.48
1.79x
1,23x
1.57
1.90
1.29%
1.76
1.50
1.55x
1.23x
Tur
1.00
1.00x
1.00
1.00x
1.00
1.00x
1.00
1.00x
1.00
1.00x
AR
Tum
1.00
1.00x
1.00
1.00x
1,00
1.00x
1.00
1.00x
1.00
1.00%
EAGLE-3
Tum
4.2
2.73x
4.31
2.84%
4.44
2.99x
4.27
2.84x
4.30
2.86
Tum
4.45
3.01x
4.47
3.00x
4.62
3.10x
4.63
2.95%
4.52
3.02%
SAM[EAGLE-3
Torn
4.05
2.79
4.13
2.96x
4.60
3.23
5.15
3.84%
4.38
3.22
Tum
4.98
3
4.98
3.77x
5.36
4.25
7.82
5.19%
5.44
4.29
194
1.77x
2.08x
1.70x
2.91
3.01x
2.13
2.13%
SAM
Tum
2.58
2.54x
2.71
2,70x
2.92
2.75x
3.88
3,27
2.87
2.83x
Recycling
Tum
2.7
2.10x
2.94
2.14x
3.00
2.18%
3.05
2.15%
2.99
2.15x
QW3-8B
Tum
3.08
2.11x
3.03
2,14x
2.20x
3.09
2.17x
2.16x
(7' ()
PLD
Tumn
1.84
1.52x
1.84
1.54x
1.84
1.54%
2.41
2.00x
1.95
1.65%
Tum
2,14
1.82
2.18
L,93x
2.07
1.83x
2.65
2.07x
2.24
1.91x
SpS
Tum
Turn
8.40
6.87
0.88
1.00x
7.09
8.14
0.94x
1.00x
7.01
6.52
0.77
0.78x
6.10
6.90
0.82x
0.79
6.68
7.80
0.85x
0.89x
REST
Tum
1.40
1.14x
1.39
1.14%
1.42
1.17x
1.34
1.09%
1.38
1.14x
Tum
1.38
1.10x
1.39
1,14x
1,41
1.14%
1.32
1.07x
1.38
1.11x
Lookahead
Tum
Tum
2.13
2.16
1.57x
2.12
2.11
1.54x
1.52%
2.13
2.05
1.58x
1.53
2:29
2:25
1.70x
1.60%
2.16
2.15
1.55x
1.59%
PIA
Tum
Tim
218
1.64
1.35x
1.84x
2.43
1.71
2.23x
1.49%
2.40
1,66
2.15x
1.41x
2.72
1.87
2.17%
1. 1.58%
2.38
1.72
2.10%
1.45x
AR
Tum
Tum
1.00
1.00
1.00x
1.00x
1.00
1.00
1.00x
1.00x
1,00
1.00
1.00x
1,00x
1.00
1.00
1.00x
1.00%
1.00
1.00
1.00x
1.00x
EAGLE-3
Tum
4.08
2.60x
4.13
2.63x
4.30
2.85
3.99
2.63%
4.11
2.69
Tumn
4.16
2.66x
2.88x
2.94x
4.14
2.78%
4.25
2.82x
SAMIEAGLE-3]
Tumn
3.84
2.59x
3,86
2.61x
4.03
2.75x
3.90
2.71%
3.89
2.67x
Tum
4.06
2.95x
4,04
2.98x
4.54
3.33%
4.14
3.05x
4.12
3,09K
QW3-8B
SAM
Tum
108
182
2.01x
2.14
1.83
1.68x
2.01x
2.28
2.20x
1.58
2.38
1.96
1.77x
2.09x
2.18
1.85
1.66x
2.08x
(T 07=00 0.6)
Recyeling
Tum
287
2,02x
2.91
2.0xx
2,89
2.11x
2.90
2.06x
2.89
2.06x
Tum
291
2.92
2.04%
2.11x
2.96
2.12%
2.92
2.06x
SpS
Tum
6.06
0.90
6.41
0.94x
5.97
0.79x
5.74
0.84x
6.07
0.86x
Tom
6.47
0.93x
6.73
0.98x
6.53
0.83%
7.65
0.84x
6.79
0.89x
REST
Tum
Tum
140
1.41
1.06x
1.09x
1.40
1.10x
1.09x
1.42
1.12x
1.12x
1.37
1.35
1.04x
1.40
1.39
1.07
1.09%
PIA
Tumn
1.56
1.32x
1.58
1.96x
1.58
1.32x
1.63
1.40x
1.59
1.35x
Tum
1.91
1,59x
1.89
1.62X
2.02
1.79%
2.02
1.70x
1.94
1.68K
23

---

Published as a conference paper at ICLR 2026
Table 11: Performance comparison of speculative decoding methods across turns under BoN frame- work with different temperature T on DSL-8B and QW3-8B.
AIME24
AIME25
MATH500
GPQA
Ovenal
Model
Method
Turn
MAT
Speed
MAT
Speed
Speed
MAT
Speed
MAT
Speed
Tur
1,00
1,00x
1.00
1.00k
1.00
1.00x
1,00
1,00x
1.00
1,00x
Tum 2
1.00
1.00x
1.00
1.00x
1,00
1.00x
1.00
1.00x
1.00
1.00x
AR
Tum
1.00
1.00x
1.00
1.00x
1,00
1,00x
1.00
1.00x
1.00
1.00x
Tum
1.00x
1.00
1.00x
1,00x
1.00
1.00x
1.00
1.00x
Tum 1
2.23
1.49%
2.22
1.52x
3.19
3.19x
2.39
1.88%
2.36
2,07
Tum
2.22
1,36x
2.18
1.57%
3.31
3.14x
2,42
1.68x
2.34
1.97%
EAGLE-3
Tum
2.21
1.24x
2.18
1.58%
2.89x
2.51
1.78x
2.33
1,92x
Tur
2.12
1.11x
2.12
1.58x
2.88
2,95x
2.20
1.60x
2.21
1.86x
Tur
2,41
1.68x
2.43
1.83x
3.63
3.36x
2.72
2.15%
2.57
2.28%
Tum
2.45
1.72x
2.43
1,83x
3.15x
2.74
2.19x
2.588
2.26x
SAM[FAGLE-3)
Tum
2.41
1.25%
2.41
1.75x
3.29
3.37x
2.70
2.00%
2.54
2.90%
Turn
2.49
1.79
2.49
1.79k
3.24
3,33x
2.73
2.11x
2.57
2.34x
Tur
1.79
1.54x
1.75
1.52x
1.77
1.47x
1.87
1.65x
1.79
1.54%
Tum
1.64x
1.85
1.61x
1.80x
2.01
1.74x
1.91
1.70x
DSL-8B
SAM
(7 0.6)
Turn
1.92
1.69x
1.90
1 1
2.10
2,02x
2,05
1.78%
1.96
1
Turn
1,97
1.73x
1.95
1.70%
2.12
2.14x
2.10
1.79%
2.01
1.85x
Turn
2.81
2.83
1.90k
2,03x
2.78
2.01x
2.82
1.97x
Tur 2
2,82
1.92x
2.79
1.86x
2.89
2.06x
2.82
1.93x
2.82
1.95x
Recycling
Tum 3
2.81
1.97x
2.80
1.89%
2.89
2.02
2.80
1.87%
2.81
1.94%
Tur
2.81
1.91x
2.80
1.86x
2.89
2.09x
2.81
1.91x
2.81
1.95%
Tum 1
1.01x
1.39
1.02x
1.09x
1.33
1.02x
1.37
1.04x
Turn 2
1,36
0.99x
1.37
0.97
1.39
1.08x
1.34
0.99%
1.37
1.01x
REST
Tur
1,38
1
1.58
0.99
1.41
1,09x
1.32
0.99x
1.37
1.02K
Turn
1.37
1.01x
1.08x
1.33
0.99x
1.36
1.01x
Tum
1.45
1.15%
1.45
1.20x
1.49
1.24x
1.58
1.35
1.48
1.24x
Tum 2
1.56
1.17x
1.58
1.20%
1.58
1.36x
1.66
1.37
1.58
1.28%
PIA
Tum
1,56
1.29x
1.58
1.21x
1.47x
1.73
1.38%
1.62
1.34%
Tum
1.56
1.17x
1.61
1.27x
1.72
1.49x
1.76
1.39%
1.63
1.34x
Tur
1.00
1.00x
1.00
1.00x
1.00
1.00x
1.00
1.00x
1.00
1.00k
Tur
1.00
1.00x
1.00
1.00x
1.00
1.00x
1.00
1.00x
1.00
1.00x
AR
Tum
1.00x
1.00
1.00x
1.00x
1.00
1.00x
1.00
1.00x
Tum
1.00
1.00x
1.00
1.00x
1.00
1.00x
1.00
1.00%
1.00
1.00x
Tum
4.04
2.58x
4.10
2.61%
4.32
2.87x
3.99
2.62%
1.09
2.68K
Turn
4.11
2.65
4.15
2.66x
2,84x
4.02
2.62x
4.12
2.70%
EAGLE-3
Tur 3
4.10
2.67x
4.20
2.65x
4.25
2.84x
3.98
2.65%
4.13
2.71x
Tur 4
4.12
2.65%
4.22
2,67x
1.34
2.84x
4.02
2.67%
4.16
2.72
Tur
3.81
2.56x
3.90
2.57x
4.06
2.77x
3.93
2.66x
3:90
2.68%
Tum
381
2.68x
3.91
2.78x
2,93x
3.88
2.72x
3.91
2.78%
SAM[EAGLE-3
Tum 3
3.79
2.66x
3.91
2.76x
4.24
3.18%
3.83
2.67%
3.90
2.83x
Turn
3.84
2.79x
3.91
2.83x
4.28
3,21x
3.85
2.70%
3.93
2.90%
Tum 1
1.11
1.82
1.66x
1.59x
1.93
1.79x
1.84
1.68%
Tur 2|
2.02
1.85x
2.02
1.88%
2.18
2.00x
2.21
2.00x
2.08
1.94x
SAM
Tum
2.15
1.95x
2.14
2,00x
2.39
2,31x
2.31
2.07
2.21
2.10%
Tur
2.27
2.09x
2.23
2.10x
2.44x
2.42
2.20x
2.32
2.22x
QW3-8B (T 0.6)
Tur
2.91
2.02x
2.88
2.01x
2.87
2,08%
2.90
2.02x
2.90
2,04x
Tum 2
2.90
2.04x
2.87
2.01x
2.89
2.04x
2.91
2.05
2.89
2.04x
Recycling
Tum
2.89
2.04x
2.88
1.98%
2.91
2.06x
2.91
2.06x
2.89
2.04x
Tum
2.89
2.08x
2.87
1.88x
2.07x
2.90
2.03
2.89
2.03x
Turn
6.06
0.91
6.41
0.5x
5.97
0.80x
5.74
0.83%
6.07
0.88*
Tur 2
6,26
0.95x
6.23
0.96x
5.92
0.78x
5.84
0.85%
6.09
0.88*
SyS
Turn
5.85
0.91x
6.15
0.44
0.82x
6.01
0.85x
6.02
0.88
Turn
6.04
0.94x
6.11
0.9x
6.07
081x
5.90
0.83x
6.03
0.88x
Tum 1
1.41
1.12x
1.40
1.10x
1.42
1.12x
1.37
1.07%
1.40
1.10x
Tum
1.40
L.1x
1.40
1.10%
1.43
1.12x
1.37
1.07x
1.40
1.10%
REST
Tum
1.1xx
1.41
1.1xx
1.12x
1.35
1.09x
1.40
1.11x
Turn
1.41
1.14x
1.40
1.10%
1.43
1.10x
1.35
1.06x
1.40
1.10%
Tum
1,55
1.32x
1.57
1.32x
1.52
1.30x
1.64
1.40x
1.57
1.33x
Turn
1.68
1.40x
1.72
1.46x
1.43x
1.77
1.47%
1.72
1.44K
PIA
Tur
1.72
1.39%
1.72
1.40x
1.82
1.54x
1.79
1.48%
1.75
1.46x
Tum 4
1.73
1.44x
1.81
1.53x
1.84
1,53x
1.85
1.48x
1.80
1.50x
24

---

Published as a conference paper at ICLR 2026
Table 12: Performance comparison of speculative decoding methods across turns under BoN frame- work with different temperature T on QW3-4B and 14B..
AIME24
AIME25
MATH500
GPQA
Overall
Model
Method
Turn
MAT
Speed
MAT
Speed
Speed
MAT
Speed
MMT
Speed
Turn 1
1.00
1.00x
1.00
1.00x
1.00
1.00x
1,00
1,00x
1.00
1.00x
Turn 2
1.00
1.00x
1,00
1.00x
1.00
1.00
1.00
1.00x
1,00
1.00x
AR
Turn 3
1.00
1.00x
1.00
1.00x
1.00
1.0x
1.00
1.00x
1.00
1.00x
Tumn 4
1.00x
1.00
1.00x
1.00x
1.00
1.00x
1.00
1.00x
Tumn 1
3.44
2.24x
3.45
2,21
3.49
2.30%
3.29
2.10%
3.42
2
Turn 2
3:43
2.20x
3.50
2.26x
3.52
2.30x
3.31
2.11x
3/44
2.22
BAGLE-3
Turn
2.23x
3/16
2.24%
2.30x
3.36
2.11x
3/45
2.22
Turn 4
3.45
2.27x
3.50
2,22x
3.57
2.32x
3.35
2.14x
3.44
2.24x
Turn 1
3.38
2.38
3.27
2.18%
3.37
2,31x
3.33
2.24x
3.33
2.28
Turn 2
3.33
2.29%
3.34
2.31x
3:55
2.58x
3.38
2.36x
3.37
x.89
SAM[EAGLES F.3]
Turn
3.39
2,36x
3,43
2,44x
2.79x
3,42
2.38
3.44
2.50%
Turn 4
3.39
2.46%
3,44
2
3.79
2,86x
3.41
2.52%
3,47
2.58%
Turn 1
1.85
1.72x
1.87
1.68%
1.81
1.64x
2.04
1.83x
1.89
1.72%
Turn
1
2.12
1.93x
2.13x
2.31
2.09
2.18
2.02
QW3-4B (7 0.6)
SAM
Turn 3
2.25
x.03x
2.25
2,04x
2.37
2.28x
2.47
2.14x
2.31
2.13x
Tum 4
2.35
2.17x
2.33
2.15%
2.54
2.49%
2.55
2.28
2.41
2.28%
Turn
2,92
2.16x
2.92
2.14x
2,31x
2.94
2.20%
2.92
2.20x
Turn 2
2,92
2.13x
2.92
2.13x
2.92
2,30x
2,93
2.20%
2.92
2.20%
Rocycling
Turn 3
2.93
2,09%
2.92
2.14x
2.92
2,31x
3.03
2.19%
2.95
2.19x
Turn 4
2.91
2.18x
2,90
2,06x
2.93
2.33x
2.94
2.23x
2.92
2.21x
Turn 1
1.14x
1.42
1.10x
1.16x
1.38
1.04x
1.41
1.11x
Turn 2
1.42
1.07x
1.4
1.10x
1.44
1.16%
1.38
1.08x
1.41
1.10x
REST
Turn 3
1.42
1.09x
1.41
1.11x
1.44
1.16x
1.38
1.02x
1.41
1.09
1.11
1.11x
1.16x
1.36
1.05%
1.41
1.11K
Turn 1
1.57
1.34x
1.59
1,34x
1.57
1.34x
1.70
1.45%
1.60
1.37%
Turn 2
1.71
1.41x
1.75
1.47x
1.75
1.52x
1.84
1.51x
1.76
1.48K
PIA
Turn 3
1.77
1.38x
1.75
1.43x
1.83
1.58x
1.89
1.53x
1.80
1,48x
Turn 4
1.77
1.51x
1.78
1.43x
1.86
1.61x
1.89
1.57x
1.81
1.54x
Turn 1
1.00
1.00x
1.00
1.00x
1.00
1.00x
1.00
1.00%
1.00
1.00x
Turn 2
1.00
1.00x
1.00
1.00k
1.00
1.00X
1,00
1.00%
1.00
1.00%
AR
Turn
1.0x
1.00
1.00k
1.00x
1.00x
1.00
1.00x
Turn 4
1.00
1.00x
1.00
1.00x
1.00
1.00x
1.00
1.00x
1.00
1.00x
Turn 1
3.30
2.28x
3.29
2.28x
3.32
2.25%c
3.01
2.07
3.23
2.22%
Turn 2
3.28
2.28x
3.29
2.27x
3.34
2.26x
3.05
2.07x
3.24
2.22%
EAGLE-3
Turn 3
2.27x
3.33
2.26x
2.29
3.05
2.08
3.25
2.22x
Turn 4
3.28
2.26xx
3.31
2,27x
3.33
2.29
3.04
2.08%
3.24
2.22x
Turn 1
3.15
2,24x
3.17
2.24x
3.16
2.19x
3.05
2.13%
3,14
2.20
Turn
2.30x
3.19
2.31x
2.16x
3.10
2.21x
3.19
2.32x
SAMJEAGLE-3]
Turn 3
3.21
2.37x
3.25
2.38
3.42
2.63x
3.09
2.16%
3.22
2.39
Turn 4
3.27
2.42x
3.28
2.44x
3.51
2,76%
3.18
2.29
3.29
2.48%
Turn
1.75
1.62x
1.75
1.63x
1.52x
1.81
1.66x
1.76
1.61x
Turn 2
1.96
1.85x
2,00
1.87x
2.08
1.96x
2.05
1.85%
2.01
1.88%
SAM
Turn 3
2.11
1.88x
2.11
1.99x
2.24
2.15%
2.17
1.92%
2.14
2.01
Turn 4
2.14
2.01x
2.17
2.06x
2.49
2.39x
2.25
2.02x
2.22
2.13%
QW3-14B
(T 10-00 0.6)
Turn
2.12x
2.85
2.12x
2.18x
2.84
2.13x
2.85
2.14x
Turn 2
2.86
2.12x
2.85
2.11x
2.90
2.23x
2.87
2.15x
2.87
2.16x
Recycling
Turn 3
2,86
2.13x
2.87
2,08x
2.88
2.19%
2.89
2.16%
2.87
2.14%
Turn
2.87
2.13x
2.85
2.10x
2.23x
2.88
2.16x
2.87
2.16x
Torn 1
5.78
1.23x
5.79
1.25x
5.54
1.0xx
4.47
1.02%
5,42
1.14x
Turn 2
5.76
1.21x
5.84
1.24%
5.64
1.09x
5.02
1.04%
5.60
1.14x
8p88
Turn 3
5.72
1.22x
5.69
1.23x
5.58
1.09x
5.11
1.03x
5.54
1.14x
Turn 4
5.79
1.23x
5:72
1.25x
1.10x
4.90
1.04x
5.55
1.15%
Turn 1
1.40
1.18x
1.40
1.16x
1.45
1.18%
1.33
1.11x
1.39
1.16x
Turn 2
1.38
1.14x
1,40
1.16x
1.41
1.17x
1.35
1.11%
1.39
1.15%
REST
Torn
1,40
1.16x
1.19x
1.33
1.10x
1.39
1.15%
Torn 4
1.40
1.17x
1.40
1.18%
1.41
1.18x
1.33
1.11%
1.39
1.16x
Turn 1
1.51
1.34x
1.54
1.34x
1.53
1,31x
1.59
1.37%
1.54
1.34x
Turn 2
1.64
1.43x
1.70
1.45x
1.72
1.49%
1.73
1.44x
1.69
1.48x
PIA
Turn
1.46x
1.72
1.45x
1.56
1.53%
1.81
1.50x
1.74
1.49%
Turn 4
1.73
1.48x
1.81
1.59x
1.82
1.59%
1.81
1.50x
1.78
1.54x
25

---

Published as a conference paper at ICLR 2026
Table 13: Performance comparison of speculative decoding methods across turns under multi-round thinking framework with different temperature T on QW3-4B and QW3-14B.
Bench
AIMH-24
AIME25
MATH500
GPQQ
Ovenal
Model
Method
Turn
MAT
Spead
MAT
Speed
Speed
MAT
Speed
MAT
Speed
Turn 1
1.00
1.00x
1.00
1.00x
1.00
100%
1,00
1.00x
1.00
1.00
AR
Turn 2
1.00
1.00x
1.00
1,00x
1.00
1.00c
1,00
1,00x
1,00
1.00%
EAGLE-3
Turn
3.39
2.21x
3,42
2.20x
3.48
2.32x
3.27
2.15x
3.38
2.22x
Torn 2
3.57
2.44x
3.59
2.38
3.64
2.50x
3.41
x
3.56
2.43x
SAM[ENGLEAS 3
Turn I
3.30
2.25x
3.32
2.20
3.35
2.36x
3.40
2.30%
3.33
2.28
Turn 2
3.58
2.74x
3.54
2.60%
3.90
3.12x
3.66
2.75
3.62
2.81%
(7 0.65
QW3
SAM
Turn
Tum 2
2,25
2.35x
2.13
1.85
2.17x
1.72x
2.26
2,31x
1.67x
2.10
2:42
x.33
2.21
1.91
1.77
2.29
Recycling
Turn 2
Tumn 1
2.92
2.92
2.29
2.17x
2.93
2.89
2.18%
2.10x
2.91
2.93
2.46x
2.35x
2.96
2,93
2.44x
2.21
2.91
2.93
2.35
2.21
REST
Torn
Turn 2
1.41
1.41
1.13%
1.11x
1,42
1,41
1.10x
1.09
1.45
1.41
1.15%
1.15%
1.32
1.37
1.04%
1.07%
1,41
1.39
1.11%
1.11%
PIA
Turn
Turn
1.59
1.35x
1.93
1.60
1.65%
1.31%
1.55
1.86x
1.29x
2.17
1.67
1.38%
2.00
1.60
1.33%
1.78%
AR
Turn 2
Turn 1
1.00
1,00
1.00x
1.00x
1.00
1,00
1.00x
1.00x
1,00
1.00
1.00x
1.00X
1.00
1,00
1.00%
1,00%
1.00
1,00
1.00%
1.00%
EAGLE33
Turn
2.25K
2.21x
3.01
2.06x
3.23
2.19K
Tum 2
3.34
2.34x
3,45
2.47x
3.51
2.40%
3.16
2.18
3.37
2.34%
SAM]EAGLE-3]
Turn 2
Turn 1
3.11
3.29
2.45x
2.16x
3.19
3.37
2/18K
2.21x
3.54
3.20
2.62X
2.17x
3.10
3.03
2.50x
2.07
3.13
3,36
2.52%
2.15%
QW3-148
SAM
Turn
Tum 2
1.74
2.02
1.95x
1.62x
1.77
1.99
1.64%
1.94x
2.28
2.21%
1.49
2.14
1.80
1.68
1.98
2.06
1:75
2.02
1.59%
(T
Recycling
Turn
2,86
2.10x
2.85
2,08
2,86
2.15%
2.84
2.11%
2.85
2.11%
Turn
2,86
2.87
2.12x
2.24x
2.87
2.20x
2.88
2.17%
SpS
Turn I
5.78
1.21x
5.79
1,23x
5.54
L.0%
4.47
1.01
5,42
1.13x
Turn 2
5.78
1.25x
6.21
1,29
6.04
1.13x
4.96
1.05
5.87
1.17%
REST
Turn
Turn
1.41
1.17x
1.17x
1.41
1,40
1.17x
1.17x
1.43
1.17x
1.17x
1.34
1.31
1.12x
1.09
1.39
1.39
1.16x
1.15x
PIA
Turn I
Turn 2
1.51
1.84
1.34x
1.68X
1.90
1.56
1.67x
1.37x
2.02
1.52
1.85%
1.31X
2,07
1.57
1.36x
1.83%
1.54
1.92
1 1.34x
1.76x
Visualization of inference acceleration process. To demonstrate the effectiveness of different speculative decoding approaches in test-time scaling scenarios, we present the responses of DSL-8B with various speculative decoding methods (EAGLE-3, SAM, SAM[EAGLE-3) under multi-turn thinking framework. In our visualization, red underlined text indicates tokens successfully predicted by the draft model, while blue underlined text represents tokens generated directly by the target model. The analysis reveals distinct characteristics and limitations of each approach. EAGLE-3's draft generation fundamentally constrained by its maximum tree depth, which inherently limits its capacity for long-sequence modeling. While this constraint enables efficient tree-based speculation for shorter sequences, it becomes a significant bottleneck when dealing with extended reasoning chains typical in test-time scaling applications. In contrast, SAM demonstrates superior capability in generating longer draft sequences in a single speculation round, significantly exceeding EAGLE-3's maximum generation length. However, this extended generation capacity comes at the cost of reduced MAT, resulting in substantially more actual decoding steps compared to EAGLE-3. This trade-off between draft length and acceptance efficiency often diminishes the overall acceleration benefits. The hybrid approach SAM[EAGLE-]] strategically combines the strengths of both methodologies, achieving an optimal balance between long-sequence modeling capability and high acceptance rates. This hybrid method inherits SAM's 's extended draft generation capacity while maintaining EAGLE-3's superior acceptance efficiency, making it particularly well-suited for test-time scaling scenarios.
Responses DSL-8B with EAGLE-3 under Multi-turn Thinking (Target, Draft) Turn 1:
Oxay, 80 need to fiqure out the profit for the math club from their bake sale. Hm,, profit is usually calcuiated tetel revenue minus total costs, right? So, should probably start by calculating how much monay they made from selling the cookies, cup- eakes,
of baking them. An)aFFist, let down all the given inCormat Moorinnya They sold 54 cookies at three Toz $1 each, An They sold 20 cupeskes at 52 eag3. Ap- They sold 35 hrownies at 81 each. The 5otal past to bake all these Items was $15, AnyoAIright, so I need to Lind the total revenue and then subtrsct the Lotal cost to get the profit. szofit.Amlnles. me break it down Izen. by item, \a\nStars Ing wish the cookies: 54 cookies sold three for $1. So, hew much did they make Erom
brownles, and then auht ract. the
26

---

Published as a conference paper at ICLR 2026
cookies? (nnkeel three cockies cost $1, then one which 18 metellymmtelyyy 80.333, But maybe it's batter to caloulate 15 in whole dol Tirst. N950,, 54 cookies divided by bookies per dollar la 18 dollars. So, they made divided by 518 From cookies. nkait, t, let veri that cookies 5 for 91, sxpeaxes: 20 cupoakes Indeed 18, 80, booklea contribute 518 to the revenue A $40,
would cost 1/3 dollar,
82 eech. So,, bring in 540. /al (ilhen, brownies: 35 brownies sold at 31 each. So, 35 times 81 is $35. So, browni
seems
AnlaSS, cupcakes
mo add up 5he revenbes: www.mooossn/ysooooiiioo. $18ynknCupcakes: 340\nkrBroww Browr les: 9391r\nTotal revenue
535 the resenue. AnlnNow,
40 35, An\nLet that: 40 18 58, and 53 35 1s 93, 30, Lotal revenuee $93. $93. NANoNow the solal cost nn),88, prorit 93 15, \mhn99 minus 15 18 78, 80, the piofis la 578, An'nnall, let me double check my mistake. wwwnStarniinnnnnnnn
compute
bake these thema
calculations to make suze dida's
cookies:
by 3 15 18. That seems right. Ceatt.nnaaee Vnlatupeskes: 20 times 2 is 40, That's 5 corract. An/pBrowmiess 16 58, plus 35
35 times 18 35, That's attalggtforward.... (nlaarrd/yyyTotall AnyyTotal revenuet
93. Yea, that adds up. InnnTotal cost is given $15, 80 subtracting that From 93 gives 78. Hom, that seems like 5 lct, but let ne think 1E did everything ecsrectly. ly. nalnne per item
is the these items, So, that's the total cost, not per item. 8o, yes, subtracting $15 from 893 Is correct. /alnSo, profit 51 is 18, 20 eupeaken at 52 15 40, 35 brownics at 31 is 35, 80, 18 40 i5 58, plus 35 1a 93. 8o, sotal revenue 1a 393. AnynTotal cost $15, 80 profts 18 93 15, which 78. Se, yea, that seems cosrect, VaNai don't think made mistake here. So, the profit (76.)n)a+++2na Answer.s <nswe Inche math club's s prefit was (boxed78 dollars. Selass..nn.. ta</think>la ( determine the profit for total revenue from selling gookies, cupcakes, and brownies, and then subtract the totel cest oF baking
the total
8152
problem says
them $15 bake
978, Awentes
me think again:
for
club from their bake sale, nead calculate the
items. items.\n)ni. \n)ni, **Revenue from cockies*<:
cookies
for $1 Number of sets gold: M (frac543 PeRevenue from cupcakeas
Vinn Revenue from cookies: V 18
cupoakes 8old each. ia Revenue from cupsakes: VI 20 (iimes 2 43 colless.Aapa **Veveu From 35 brownies sold at 51 each, la Xevenus from brownies: \ 35 itines Total Revenue from cupcakes Revence from brownies}a - Totel revenue {( 18 40 35
[times 18 dollars.
brownics
dollars. Sallars.nnni (n/n4.
Revanue from cookias
V dellarn. (a\n5. **Totel cost*+:|n
Ectal cost to these items W28
An
s*Profiteesly Profit = Total revenue Total cost - Prof17 # - 15 = 78 001- Tass, laynThue, the prolit for the mach club was b/' dollara.
Turn 2:
sell ing cookles. They 54 nookies for $1. So, the number of sets sold is 54 dividad by 3, which equala 18. miltiplying 16 by 31 gives ravenue of from anokiea, ea. AnNnNext, T'1l calcu
caleulate
ravenue from cup- oakes, They sold 20 eupeakes $2 each, Mult hallippying 20 $2 resulca in revenue of $40 From cupcakes. \nlathen, I'l1 calculase the zevenue Ezom brownies. They sold 35 brownies 31 esch. Multiplying the revenuag: $18 (cookien) + 540 (cupeakes) + 635 (brownies) equala a totol revenue Itema
835 from brownies. (aladdiing all
Find the Fract the total cost From the Ectal revenua: 393 515 equala 5 profit of 578 An) Tore, She math elub made profit of (ooxe 781] dol ara.
$93. \nnnThe
determi The profil for the math club from their bake sale, need Lo calculate the Lotal sase from selling cookies, iee,, supcekes, and brownies, and then subtract the total cost of baking these items. (n)ml. *eVeenue from cookies**:yy in
cookies sold (s1.)n Sumber of sets sold: \ ( frac543 18 \/ Revenbe from cookiess \ 18 \0362,
for
[Limes 18
cuposkea sold 122 aach. Revenue from cupoakes: VI 20 \e1nes 2 40 del Tollars.Innns **Revenue from brownlesee 35 brownles sold at (31 each, \n Revenue From brownles: \ Lmme Total
From cupcakes**
Revenue from cookies Revenue from cupcekes Revence from brownies)a Totel revenue | 18 40 35 total
- dollars. Aaind. AvTotal al
1615, \615.Amk66 An)n6. exProf vn Profit = Total revenue Total cost I Prof12 * \1 93.9 15 78 001-
\I dollars. (r\n5. **Tatel cost**1|n
these itoms
latnThue,
mach club (ooooxe788) dollars.
Responses DSL-8B with SAM under Multi-turn Thinking (Target, Draft) b Turn 1:
Okzy, 50 profit 18 usually leu abed as total revenue mi noa total costa, right? So, T shouls probably stazt by calcuiat Lnnulating how much money Chey mlad [rom sell ing the cookles, eug cakes, brownies, and then subtrect the
to figure out profit
their bake sale. Hnm.
of baking them. \nlaFisst, let note Jown all the given informstior: (n)p- They sold 54 cookies at three for $1 each. \n- They 20 eupeakes at $2 each, An= They sold 35 browries at $1
\n= The bake all these Ltera was $15. AnyaAlrigh Light, 90 I need Lo Eind the Colal reveriee and Chen subtract the total cost. get the Prraft.Annnna..
break it. down 15em by 1tom, Aninstars- Ing wish the cookles: 54 cookles sold three For $1, So, how much did they make from cookies? (okkie///a/nlel1.. if three cuckies cost 51, then one cookle would cost 1/3 of a dol laz, which is appreximately 30. 333. maybe it's better calculate it in whole dol-
27

---

Published as a conference paper at ICLR 2026
per dollar is dollars. So, they made 918 from cookies. ae..s..o M,, let verify that cookies for $1, 30 54 divided is indeed 18, So, bookies ribut $18 Lo the reverie, AnYaNeet, cupcakes: pakes
first. An/n55, cockies divided by
cup straight forward, ad. An)nSö, brownies AnynThe, brownlear 35 brownles sold at $1 each, 80, 35 times $1 la me add
92 each,
times 92 1a 64D. That
cakes bring In 540. $40.)01
zevenues: Anlaeos (niotal revenue 18 + 40 + 35. Anlauet me com- les: SiByaanCup=aaeee Capppaaeeee #40hm)marommiee 935
the revenue, AnValow,
puta that: 18 4D 15 58, and + 935
total $15, 50, profit 16 total revenue minus total DOst, An/n3o, profil 53 15.3n)n3 minus 13 78. 50, the protiL 378. nankaLL, lel ne double /nknStartiing
93, 30, tctal revenue
193.An\nMes An
cost to bake these items
didn't les: 54 divided by 3 is 18, That seems right JatDuupcaceCep 20 Limes 2 40, That's 's coc rect. (es.snnnneeeeee
my
lona
any
That's ateaichtfosward, Ja'nIotai zevenue: is 58, plus 35 15 93. That adda up. innnooaa- cost i5 given as $15, 50 subtracting that from 93 gives 78. Hmm, total coat? Tae probles tal cost, net PEE stem. S9, yea, subtracting 515 from tctal revenue is cosrect. Ja)nao, profit 976. injawait, but let think again: 54 cookies
seems righ, but
cest per item cost them 515 bake thess itema, 30, thatfs the to:
mo think, (nnnkk.n,,n
for 31 is 18 dollars. 15. 20 supcakes at each is 10. 35 brownies at each is 35, 8o, 18 40 18 58, plus 35 is 93, So, tasal revenue rect, www.://nnnttttrattt.oo.o serse. Ir but the probleg em says It cost then to Lotal cost, not liem. 30, don' need calculate cost
is 80, maybe I con calculate the aost per item and 5ee 15 that makes
yes,
seens cor-
these Items. 8o, that's s the item. So, that's can Girectly subsract the total cest from total revenue, \ninso, I think my calculation corrent. |p)ntherafors, the profit is 378. (nlasainni AnsMer /arhe math club's profit (boxed78 dollars. wow.ars..aeaaeeee............ hwt/chink>lalaa determine the profit for the math club from their bake tatal revenue
sale, ие nsed to calculate and ther subtract the total cost of baking these items. /n/nl. +Callulla "evenue from cookleairr
selling cockies, cupcskes, and brownies,
cookles sold at 3 For $1. Revenue Trom cookles: \( Crac544 18 dollars. \n5n2. veCaloulate each. in Revenue from cupsakes: VI 20 \zines 2 40 }) dollars. Jn)nd. sstalculate rey- from brown 10
From cupcskes:**
20 capcekes seld at 82
brownies sold
from brownies:
Stimes 1 35 dollars. . \n/z4, e*Calculate total revenueiva Total revenue: ( 18 93 dollors. 1n/a5. v*Caloslate profit:*e Jotal coat
the itams: $15.1n in Profit: 93 15 78 dollars. AnynThua, the profit for the math club was ([boxee78 dollass.
b Turn 2:
sell cookles. They sold 54 pookies for $1. So, the number of sets sold is 54 dividad by 3, which equala 18. Therefore, the revenue from cookles
caleilate
dol lara. VaNoNeNt, 1'1 ca calevlate tha revance from eupcakes. They sold 20 eupcakes at 52 eaca. Mult Multiplying 20 supcakes. AnlaTaen, 1'll desermine the zevenue Ezom brownies. They sold 35 brownies at from brownies is 35 dollers. www.eessseelaaae.... AelnAdding sll the dollars (cookies) = 40 dol lars (aupeakes) 35 dollars (brownies) equals a total rev- prof
gives revenue of 40 dol From
51 each. the
doll Tarlaar.iii. Antns. inalll. 2F 15 dollars from the total revanse. TEis gives profit of 93 minus 15, which equale the math club's & proFiE from the bake aa 16 (Bboxed788
Find
78 lars. www.hrrraaathtttaaooo lars. /ps///ssks//aaiiie (nA determine the profis for the math club from their bake sale, need to calcuiate the tetal revenue from selling cookies, cupsekes, and brownies, and then subtract the
\n)ml, vACaleslate fessAs)r - 54 cookies es sold d at 3 for (51. Revenue fron cookless \ \frao543 a 18 V dollara. Ja/nZ, --Caloulate zevenue Tron
baking these
From cook-
eacs, Reverue From cupcaken: V7 20 Stimes 2 40 V collors. collars.Anyrs. +*Ca (5alle late revenue from 35 brownles sold at {31 each. In Revenue Trom brown \ Lime Totel revenue:
brownles
35 dollere Aalrs..eaa. AxCaleulate Lotel revenuerxaja
18 10 35
33 V) dollars. /nlars..a/aa... vaCaleulate profit:ar
ictal cost to bake the items: 1315. dollara. (niaiannn the profit the math club
Profit: ((
78
dol
Responses of DSL-8B with [AM(EAGLE-3] under Multi-turn Thinking (Target, Draft) Turn
Oxay, 80 need to figure out the profit for the math olub from their bake aale. Frm, profit bsually caleplated
tctal revenue minus total costa, right? 7 8o, should Saleul ing how much money they made from sel 3elling the bookles, es,, oup- cakes, and brownies, and Lhen subiract the cost of baking them, NolaPizst, let me note Jown all the given informatiers len: \n)a=
probably start
three for $1 each. sold 20 cupcskes at 52 eaca, \n- They seld 35 brownies st 51 each, \a- The total cest to find the total
sold 54 cookies
They
$15. An) MlAnArrigh right, subtracl the Lotal coat Lo get the peocil, (coCit,Ni AnVoLet me break K it down Lzer: by [Lem, AnnnnCars 31. 3o, 1E chree cookies cost $1, then one cookle would coat 1/3 which is appzoximetely 80,333, 33,, 3ut maybe It's better to calculate 15 in whole dollers first.
these
and then
Ing wish the cookless 54 bookles sold booklea7\nlnBaa
much did they dollar,
First.Jelnso, SelnSo, 54 cockies divided cookies per dellar is 18 dollars. So, they made
28

---

Published as a conference paper at ICLR 2026
$18 from cookies. nnait, me verify that: cockies for 91, so divided by indeed 18, 50, cook1 contri $188 to the revenue AnNett, aupcakees cupeakes sold au 52 each. 5o, Limea 52 $40, That seema Bliaigal Lorward. Aninao, cupcakes hring 940. \nhnn brownles: browniea sold
$35. So, brownlea add $35 to the revenue, nNNow Iet me add up all the Soveenes:\yn ies: SiEVa\nCupsakess
18 40 35. AnVaLet compute that: 16 40 is 58, and 58 + 35 93. So, total revenue is 593. An the total cest to thase itoma MOS $15. 50, profit sotal revanue minus botal Best. ccst.)nnnag \n)no, profit 93 - 15, \n5n93 minua 15 i8 78, 80, the profit 1a 578, Aninnaat, 1et me double-chees calculations Lo make aure dida't make mistake... ValnStart aascaaaiii iiiigg with cookies: 54 divided That's
2t0.n\rBBoww.ees 833%plalotel
by
Anw.at.......rr.........r. oRoowlesss 35 Limes is 35, Thai's straigal Corward Lorward.Anyaaolaa AnAaTolal revenuer 18 1 40 is 58, plus 35 is Yea,
That
Anpooppaksss 20
up. ap.'nnnnotal cost given 915, 20 subtracting 72. smm, that 5zems like 1ct, but let me think if did everything cesrect ly. (n)nwast, 1a the cest per item
gives
total bost 6152 bake these items, 30, Shat's the tocal cost, From 533 is cocrest. (aynSo, profit 1s 878, inlnneit, but let me think again. is theze en pez item and then subtracting
prohlem saya cost them 315 item. 80,
subtractiog 515
other wey compute this? Maybe by calculating the From the revenue PEE item?)a)aiet me try that asproach to cross verley. AnYaPer cockies: So, each cookie costs 1/3
dollar, which approxi- mately 50.333. So, total cost for cookiea would be 54 timas (1/3) which i5 18 dallars. 8o, same as before. (nl.orr..nynCuooon hnppopaaeees which is $40. \aynBromnies Is 835. An)notal cost: 13 40 35 15 93, Kaiz, but the problem says the tctal cost maybe
times 2, brownies at 9- each, 80, Lotal cost is 35 Limes which
cupсakea at 52 each. 80, tocal 0ost
$13. That doesn't t make misread the problem. Let caeck again. agein.\n\a" math club is having bake sale Fundraiser so raise money for upconing trip. They sell cookies three
hold
that can't
(n/ndait, hold
and 20 eupcakes at 62 each, and 35 brownias at $1 each, T+ it coat the math club $15 to hake these items, what was their procit? " Wnlaaalt, 80 the total cost bake these Ttems was 515, the Izems they sold. Sc, the total cost is a fixed 815, not per item. Ectal stem.)n/ns... \nSo in that case, i5 915, profit
iten. 8o, that neans the total cost $15, regardless of how
the 478.12)......
coloulated before, /alawait, but 1E the total cost 18 $15, that seems reslly low For baking 54
393,
15 a $78. 2ookies, supcakes, total. Arlaait, the problem eaya, "1S 0085 the math elub $15 bake these 1tems. 80, Chat's the Lotal cost, not pez ILem, So, that's correct, Sel.LL........... AnVaSo, the :0tal coat 1s 815, profit total is 378. knlaait,,, but let me think again, If the total cost is 615, but the total rev- 1a 593, that sidering they sold a 1ot oF items. ininwais, 54 zookies at thres for doller is 18 dol- Tara, 20 cupcakes
brownies. Moybe should chock the coat per 1tem
total
which is 93 15 78. the profit
1i sigrrt ffeent profit.
retlistio
maybe,
and 35 brountes 0s at each 35. 30, 93. 3o, thaz's correot VnlnTotal 0082 15, profit that'ss cocrect. www.ect...nnAee..... (aw.Aiteras..i... ively, maybe the problem 18 expect ing Re 20 calculate the cost per item and then subtract from the proach. (n) 1et mo try that AninIf calculate the cost per cookio, supeake, and the
6361 15
35 - 15 78. An I think
item,
that would be different ap-
prownle, Shat might not be the right approzch. \rlaWait, perhaps the problem that the Ia 515, 50 post 1a per Item, but the problem says cost the math club 815 to bake these itema. ilem. Vahnso, in that profit is tetal revenue minus botal cost. An) 53 15 is 78. fa.laaaieeeeeore
then
per Item,
8ays
Se, Lhal's zhe Lotal
the Lo:81 cost 1s 815, so the profit per item and
but let think again. Maybe should calculase the then see 1f the total cost ta 15. (nMait, if the total cost i8 $15, then the coat per would be
is
cookles + 35 wwwwnins....... \n)n8o, 54 + 20 15 74, blus 35 l5 209 itemg 37 t058 20551 An)n8o, So, Ectal cost is 315, s0 0082 per Item 1a 15 139, which approx wwwtelllllyyy...mmmmmmo...o..yye $0. 133...11 An that aeems too lew per item, DE Stem. nelie, the problen. Joesn's specify whether the 315 is the total cost oF per type.
by Lotal Liens, www.s.AAATao AeNaTolal 1tems:
cupoakes
maybe that's the
maybe the
is pez type
thot'a the totel An)n.,0) In that case, the cotal cost 18 $15, 80 profie 18 zotal a. revenue minus total An5So,
says "it
math club 615 bake these
the profit 1s 376. 576. Answer+* Anche math club's profit was (boxed76 collars... wtt/ whrnkkkkkknaaa No)noo de- Lermine the profit for the math elub Irom their bake sale, wе need to calculate the total from selling cockies, capcekes, and brownies, and then subtract the total baking these itams. \n|zi, staleviate the zevenue from cookies: (a n 54 cockies sold as cookles: \I \frac543 imea
cost, which
that's correct, www.ee.ennaeeeeeooooo
of
18 V dol lars. +*Ca Iate the zevenue from cupcakes 24 20 oupcakes sold at 52 each \n Revenue from cup- [times
for 31
Cakes:
dollars. AnAn3. **Celculete 35 brownies sold at 31 each. Revenue from brownies: \( 35 times 1
zevenue Erom browniestwely
35 dol-
lars./ulad, AsCalculate the Lotal zevenue:4<4p
Total revenue: ) 13 40 35 - 93 V dcllars. JalnE. vccelculate the profitivela Total to bake the itens $15.
Profit: {( 93 33- = 78 dollars. (n/athus, the profit for the math club wea (8
Turn
First, I nned to caleulate the revenue from selling cookies. They sold 54 cookias at for $1, So, the number of sets sold is 54 divided by 3, which equals 18, Therefore, the revenue from cookies times $1, totaling $18, (alneext, I'll calculate the
29

---

Published as a conference paper at ICLR 2026
from cupeaken 1. 20 mul- Eiplied by 52, which equals 340. AnlnThe, I'1l salculate the revenue from brownies. They 35 brownies st 91 eack. S0, the revenue From bresnies is 35 meltiplied by 61, totel- ing 935, (olnkdding up all the revenbes: $18 (cookles) $40 (cupeakes)
from supeakes. They sold 23 cupeakes 92 occh. Sc, the
935 (brownles)
equals a 19.41 revenue of 593, A93. inally, Lo Find the prorit, 1'1l aubtract the Lotal of baking the items, which is 815, Erom the tetal revenue, Se, 993 minus 515 equals profit of $78, 178.1alaihen www.aarreeooooe the math clab's 5 profit From the bake sale i5 (bxxedd88 the profit for the moth cluo from their bake sale, need to calculate the tctal revenue From selling cookies, cupsakes, and brownies, and then subtract the total cost of baking these 1toms. st.ms./nkrl. *4čalculate tha revenue from 34 cookles sold at 3 for \ A0 Revenue from cookie8: V< Trrao543
lara. hww/thinnnnnnnnnnn
3ooklea:<>
20 cupcakes 20 \times 2 40 V dollars. \nta3. v+Cal- aulata the revenue from browniss: 1 - 35 brownies sold at 161 each, \a Ravenua from dol lara. Anhnn.
[Limes 1 ------1-00 18 V dollass. dollars.\uhn2. VACalculate the seveaue 1:om capcakessxr
scld a: 192 each. \n Revenue From cupcakes:
brownles: (
the total revanses Eal revenus: ( 18 + 40 + 35 = 33 V dollard. \nins. Anins. A+Calculate the profit:*el
To-
Fer tal cost to bake the Thoms \315. Profies VI 93.93 15 18 V dol lars, Aninthus tha profit lor the math olub was (8oxed dollane.
A.4 POTENTIAL EXTENSION OTHER REASONING FRAMEWORKS
This work primarily investigates two mainstream test-time scaling frameworks BoN and multi- round thinking. We argue that other complex reasoning frameworks, such ToT and MCTS-based search algorithms, could also benefit from speculative decoding. Conceptually, these frameworks can be regarded localized, step-level test-time scaling methods. Instead of generating multiple complete responses as in BoN or multi-round thinking, these frameworks expand and evaluate several reasoning paths at each intermediate step. This localized exploration inevitably introduces substantial overlap and redundancy among candidate steps, a phenomenon closely related to the computational inefficiency examined in this benchmark. However, leveraging speculative decoding to address this redundancy in search-base frameworks presents unique technical challenges. A key distinction makes the direct application of existing speculative techniques to ToT or MCTS-based reasoning non-trivial. Unlike the relatively linear generation process in BoN multi-round thinking, these search-based frameworks involve dynamic operations such as state evaluation, pruning, and backtracking, Existing speculative decoding methods, whether N-gram-bases or advanced variants using suffix automata (e.g., SAM) or adjacency matrices (e.g., Recycling), are designed for monotonic forward generation with single contiguous prefix. Training-baser methods such as EAGLE-3 share the same limitation. Therefore, adapting them to search-base frameworks would require modifying their underlying write and lookup logic, for example by maintaining separate speculative modules (such as the individual SAM) for each search step, or extending data structures such as suffix automata and adjacency matrices to handle deletion operations, since certain steps may be pruned during the search. In summary, adapting speculative decoding to complex reasoning frameworks is a promising direction for exploiting redundancy, but it requires non-trivial framework-specifi modifications that fall beyond the scope of this benchmark. We leave this adaptation for future work.
A.5 DISCUSSION ON HYBRID STRATEGIES
The simple hybrid strategy benchmarked in this paper demonstrates remarkable effectiveness. If such a straightforward combination already yields a substantial performance gain, it strongly indicates the presence of extensive, underutilized redundancy within current architectures. We certainly agree that developing more dynamic hybrid strategies is vital future direction. While a comprehensive exploration extends somewhat beyond the scope of this work, we suggest several promising directions, including: (i) Context-Aware Switching, where the strategy adapts dynamically based on the semantic context of the generation task, such as prioritizing N-gram methods for structured outputs like code while employing training-based draft models for open-ended natural language generation; (ii) Phase-Specific Optimization, where the strategy is tailored to different stages of the generation process, for example by utilizing N-gram speculation during summarization phase where the text is highly likely to reiterate content from the preceding reasoning steps; and (iii Uncertainty-Baut -Based Toggling, where the strategy is adjusted on-the-fly according to the model's predictive uncertainty,
30

---

Published as a conference paper at ICLR 2026
specifically by leveraging N- N-gram speculation when low uncertainty suggests formulaic structures, while switching to training- based methods for open-ended generation.
A.6 THE USE OF LLMS
the preparation of this manuscript, LLMs are used solely for language polishing purposes. Specifi- cally, the LLM assisted in improving the clarity, grammar, and fluency of the text during the final editing stage. They do not contribute to the design of the research presented in this paper. All idcas, analyses, and conclusions are the sole work of the authors.
31


Reviews:
[{"review_id": "qBSTUCrFhD", "reviewer": "Reviewer_7EPL", "summary": "This is a benchmark paper that targets on accelerating test time scaling method by speculative decoding. It compares model-based, training-based, and N-gram approaches.\nThe key finding is that simple N-gram-based methods (like SAM) are highly effective at capturing this redundancy. A hybrid method, SAM[EAGLE-3], achieved the best overall speedup by combining N-gram's repetition capture with a training-based method's semantic prediction. N-gram methods also show \"progressive acceleration,\" getting faster across multiple reasoning turns , though they are sensitive to temperature", "questions": "See weakness.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The problem this paper targets seems relevant, and the method is intuitive and motivated.\n\n- There are many interesting findings and the experiments seem to be solid. For example, the most interesting finding is that simple N-gram-based methods, particularly SAM, are highly effective at capturing the redundancy in Best of N or Multi Turn answers. The findings are comprehensive.", "weaknesses": "- The one concern I have right now is the novelty of the method. Maybe this is because this is my first time reviewing on the dataset and benchmark field.\n\n- Whether other complex reasoning frameworks such as Tree-of-thought or MCTS-based search algorithms could also benefits from speculative decoding?\n\n- Also, would the approach generalized to other field such as code generation and summarization?\n\n- The paper correctly identifies that hybrid methods (like SAM[EAGLE-3]) are a promising direction and achieve the best performance. However, it also admits that \"current hybrid strategies remain heuristic\" and that the potential of N-gram matching is \"underexploited\" due to \"suboptimal integration strategies\". The paper benchmarks one such simple strategy but does not propose or experiment with more dynamic or refined strategies to truly unlock this potential.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DDyJ2Ufx0T", "reviewer": "Reviewer_mU4f", "summary": "The paper discusses different methods of speculative decoding and tests them across reasoning benchmarks to measure MAT/wall clock speed up. \nThe paper is a little limited in scope and could have been a bit more thorough. If the authors wouldn't mind running this on some larger benchmarks with different scaled models, I believe this would make a very nice paper. I'm still content with the paper, so I recommend acceptance.", "questions": "N/A", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. It's a nice paper with useful results. \n2. The Takeaways are a pretty good analysis of what's going on with each speculative decoding method. \n3. Presentation is pretty nice. \n4. Practical results.", "weaknesses": "1. It's not really a benchmark. It's just running the different methods on common benchmarks. Not a huge deal since it's still good science. \n2. WOuld have been nice to scale these across difficult model scales. The results may have been different. \n3. Datasets are tiny, like 120 total samples. Could have done this on more deata. \n4. Error bars could have been nice if you could rerun this multiple times to see what happens.\n5. Varying batch size would have been nice.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qfABNl91aK", "reviewer": "Reviewer_eh5y", "summary": "The paper presents the first benchmark for evaluating speculative decoding methods in test-time scaling of LLMs, offering standardized protocols across paradigms like Best-of-N sampling and multi-round thinking. Experiments show that simple N-gram-based methods efficiently handle repetitive reasoning patterns, suggesting their integration with model- or training-based approaches can make test-time scaling both faster and more effective.", "questions": "Would the Best-of-N speedups be different if the reasoning traces were samples in a batch? (e.g. The amount of overlap would be much smaller for N-gram based models)", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper tackes an important problem of evaluating speculative decoding approaches in test-time scaling scenarios, and provide a formal benchmark.\n2. The authors perform extensive evaluations ranging in nine different speculative decoding methods, four datasets, and two LLMs.\n3. The authors provide a comprehensive analysis of the experiment results, along with clear takeaways for the readers.", "weaknesses": "1. As far as the reviewer understands, the reasoning traces were sampled sequentially in best-of-N scenarios, using batch size 1 (L264). However in realistic scenarios, these traces would be sampled in parallel (i.e. using batch decoding). This suggests a critical gap between the benchmark performance and real-world benefits.\n\n2. The benchmark does not consider test-time tree search (e.g. beam search or DVTS [1]), which is another major test-time scaling strategy. (Note: This weakness is not critical, as the reviewer understands that it might add too much complexity to the benchmark.)\n\n[1] Beeching et. al., Scaling Test Time Compute with Open Models", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "NRp2MbNniu", "reviewer": "Reviewer_Hr8t", "summary": "The paper presents comprehensive benchmark experiments evaluating three types of speculative decoding methods -- model-based, training-based, and N-gram-based -- in the context of LLM test-time scaling. The empirical results reveal several notable findings: (a) training-based methods exhibit robustness to sampling temperature, (b) N-gram-based methods are effective at identifying repeated patterns at the token level, and (c) hybrid approaches can leverage the strengths of both. These findings offer insights into the behavior of different approaches for reasoning models and may inspire further research on more effective methods for test-time scaling.", "questions": "Q. Have the authors experimented with models of other scales, other types of reasoning benchmarks, etc.?\n\nQ. How large is the variance for the performance numbers, e.g., reported in Tables 3 and 4.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Empirically evaluates an important setting for the use of speculative decoding, i.e., test-time scaling for reasoning models. This setting can particularly benefit from more efficient inference, and understanding what works and why is highly valuable.\n\n2. Evaluates a wide range of speculative decoding methods including model-based, training-based, N-gram-based, and hybrid methods.\n\n3. Reports several notable findings, suggesting that hybrid approaches may be particularly promising.", "weaknesses": "1. The datasets used for the benchmark experiments are mostly focused on math reasoning. Evaluating how the findings extend to other types of data, e.g., coding, general reasoning, social reasoning, would provide more valuable insights into the methods.\n\n2. Limited model scales. Results for models of scales other than 8B would strengthen the findings.\n\n3. While the paper reports multiple notable findings regarding how different types of methods perform on the benchmarks, technical novelty (e.g., proposing a better working hybrid method) is rather limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)