Guided Speculative Inference for Efficient Test-Time Alignment of LLMs
Authors: Jonathan Geuter
Youssef Mroueh
David Alvarez-Melis
Venue: ICLR 2026 Poster (2026)
Keywords: Test-Time Scaling
LLMs
Large Language Models
Speculative Decoding
Inference
Inference-Time Scaling
Best-of-n
Soft Best-of-n
PRM
Reward Models
Reward Guidance
KL Regularization
GSI
Primary area: foundation or frontier models, including LLMs
TL;DR: We describe a novel algorithm for test-time scaling that combines ideas from speculative decoding and best-of-n sampling and has provable guarantees.

Abstract: We propose Guided Speculative Inference (GSI), a novel algorithm for efficient reward-guided decoding in large language models.
GSI combines soft best-of-$n$ test-time scaling with a reward model $r(x,y)$ and speculative samples from a small auxiliary model $\pi_S(y\mid x)$. We provably approximate both the optimal tilted policy
$\pi_{\beta,B}(y\mid x) \propto \pi_B(y\mid x)\exp(\beta\,r(x,y))$ of soft best-of-$n$ under the base model $\pi_B$, as well as the expected reward under the optimal policy. In experiments on reasoning benchmarks (MATH500, OlympiadBench, Minerva Math, MMLU-STEM, GSM8K) and across different model families, our method achieves higher accuracy than standard soft best-of-$n$ with $\pi_S$ and reward-guided speculative decoding (Liao et al., 2025), and in certain settings even outperforms soft best-of-$n$ with $\pi_B$, while reducing end-to-end latency by up to 28%.

Published as a conference paper at ICLR 2026
GUIDED SPECULATIVE INFERENCE FOR EFFICIENT TEST-TIME ALIGNMENT OF LLMs
Jonathan Geuter
Youssef Mroueh IBM Research
David Alvarez-Melis
Harvard SEAS
Harvard SEAS
Kempner Institute
Kempner Institute
jonathan. geuterê de
ABSTRACT
We propose Guided Speculative Inference (GSI), a novel algorithm for efficient reward-guided decoding in large language models. GSI combines soft best-of- n test-time scaling with a reward model r(x, y) and speculative samples from a small auxiliary model It (y x). We provably approximate both the optimal tilted policy T3,8(y ) z) X Tn x)) exp(3 r(z. y)) of soft best-of-n under the base model nT well as the expected reward under the optimal policy. In experiments on reasoning benchmarks (MATH500, OlympiadBench, Minerva Math, MMIJU STEM, GSM8K) and across different model families, our method achieves higher accuracy than standard soft best-of-n with and reward-guice speculative de- coding (Liao al., 2025), and in certain settings even outperforms soft best-of-n with TB, while reducing end-to-end latency by up to 28%.
INTRODUCTION
Large language models (LLMs) have demonstrated remarkable performance across diverse gener- ation tasks, with scaling model and data size being the predominant way to reliably enhance their capabilities (Kaplan et al., 2020; Gemini Team, 2024; OpenAl et al., 2024). However, such scaling incurs ever-increasing computational and economic costs, and there is growing evidence that seal- ing training compute yields diminishing returns (Hernandez et al., 2022; Muennighoff et al., 2023), prompting the need for efficient alternatives.
Test-time scaling (Snell et al., 2025; Muennighoff et al., 2025; Zhang et al., 2025) has emerged as a promising direction, which focuses on scaling inference-time rather than training time compute. Various test-time scaling methods, such as best-of-n sampling (Gao et al., 2023; Mrouch & Nitsure, 2025; Beirami et al., 2025) and soft best-of-n sampling (Verdun et al., 2025), have been proposed, all of which achieve improved downstream performance through increasing inference FLOPs. How- ever, users can have constraints on inference compute and latency, and test-time scaling can quickly become prohibitively expensive. This has led to the development of latency-efficient test-time scal- ing methods such as speculative decoding (Leviathan et al., 2023; Sun et al., 2025), where a small draft model iT accelerates inference from a larger target model nT.
Moreover, the goal is oftentimes not only to achieve better downstream performance, but to do so in way that maximizes the rewards of given reward function r(æ, y) quantifying the quality of a response y given a prompt æ. Several frameworks for aligning model outputs to a reward model have been proposed, both for training as well at test-time (Yang & Klein, 2021; Ouyang al., 2022; Touvron et al., 2023; Mudgal et al., 2024; Huang et al., 2025). Recent work on reward-guided speculative decoding (RSD) (Liao al., 2025) combines model alignment with speculative decoding from a draft model, though it lacks theoretical guarantees on distributional fidelity.
Contributions. In this paper, we introduce novel test-time algorithm, Guided Speculative Infer- ence (GSI), which leverages samples from draft model Rş to (approximately) sample from the
We will interchangeably call Rs the draft small model, and F the bose or target model.

---

Published as a conference paper at ICLR 2026
Accept
(log wrfir 1);
F. 2
??? my 88
(x(x))))
Fs = (r log In Fs
(vF²)
S BoN(
(log ms (oix""")) })
if Pi. <
vf" ~ 5
Draft model
Target model me
S- S-BoN (prr())))
Reward model
Soft best-of-n (S-BON)
V
Reject
Guided Speculative Inference workflow for one reasoning step. A sample v generated from the draft model my selected with soft best-of-n (S-BoN) with parameter from the tilted rewards P4. If its reward lies above threshold u it is accepted. Otherwise, it is rejected, which triggers resampling from the target model TB with soft best-of-n.
Figure 1:
base distribution nn aligned to reward model r, namely the tilted distribution (Section 4):
To x) TB æ) exp r(x,)))
Zz.n(v)
Importantly, by rilting (i.e., adjusting) the rewards r according to the log-likelihoods under both w and TTy (Figure 1), GSI provably approximates this tilted distribution, making the first speculative test-time scaling method with distributional guarantees to the optimal tilted distribution, to the best of our knowledge. We summarize our contributions as follows:
We propose novel test-time scaling algorithm, Guided Speculative Inference (GSI), which uses draft model iT to accelerate inference from target model TT while aligning re- sponses to a given reward model (Section 4)
We prove that GSI enjoys strong theoretical guarantees and provably approximates the optimal tilted distribution (Theorem 1), as well as the expected reward (Theorem 2)
In extensive experiments on reasoning benchmarks (MATH500, OlympiadBench. Minerva Math, MMLU-STEM, GSM8K) and across model families (Qwen-2.5-Math, Qwen-3) and sizes, we demonstrate that GSI outperforms both reward-guided speculative decoding (Liao al., 2025) and soft best-of-n sampling with the draft model, and sometimes even soft best-of-n sampling with the target model (Section 5.1)
2 RELATED WORK
Test-Time Scaling. Inference time compute can be scaled along different axes. Broadly, such methods can be divided into parallel and sequential approaches. In sequential approaches, the model spends more time on a single response and aims to improve it, for example by appending think tokens (Muennighoff al., 2025) via self-correction (Qu et al., 2024). While sequential approaches can often generate high-quality responses, they don't scale well. Parallel approaches instead scale test-time by parallelizing computations, which typically involves generating multiple responses reasoning steps at time. Common parallel approaches include majority voting (Wang et al., 2023) and best-of-n sampling (Mrouch Nitsure, 2025; Beirami et al., 2025) (see Section 3).
Speculative Decoding. Speculative decoding (SD) (Leviathan al., 2023) accelerates sampling from лy by first drawing proposals from ny and then accepting or rejecting them based on a criterion derived from the ratio TB/ On rejection, one falls back to direct sampling from Tg. SD provably samples from the distributions и The core idea is that tokens can be sampled from πs autoregressively, but verified by æa in parallel, thus generating up to l + tokens from ag with
2

---

Published as a conference paper at ICLR 2026
single forward pass of Tg. Variants of SD include block verification (Sun et al., 2025) where sequences of draft tokens are verified jointly instead of token-by-token, and SpecTr (Sun et al,, 2023) which allows for verification of multiple draft sequences in parallel by framing SD as an optimal transport problem. SI has also been combined with early-exiting (Liu et al., 2024), and Bhendawade al. (2024) propose using n-gram predictions of a as drafts, which alleviates the need for an auxiliary model.
Reward-Cuided Speculation. recent work proposes RSD (reward-guided speculative decoding) (Liao et al., 2025), where samples are generated from Ts, and a threshold on the reward of the samples from T determines whether one should accept the sample or resample from TB. While this approach shares similarities with GSI, it only provides a guarantee on the expected reward: under the assumption that Erg [r(y x)] 2 ET [r(y x)) RSD satisfies E (rrsso((y) x)] 2 Exy Eng|r(y x)}. which in the worst case does not yield any improvement over the small model πTs, and also does not guarantee anything about the policy TRSD itself. As we will see in Section 4, GSI provides guarantees on the induced policy directly. In concurrent work, Cemri et al. (2025) propose SPECS, an algorithm that pairs draft-generated samples with a cascading routine, which determines which model - draft or target- to use in subsequent iterations. Similar to Theorem 1, they also derive a KL bound with respect to the target distribution. However, their bound requires assuming that the block size (i.e., the length of reasoning steps) tends to infinity, and that the number of samples n. and the rejection threshold v are random variables, all of which are approximations that do not hold in practice. Our KL bound in Theorem does not require any such assumptions. Moreover, GSI seems to significantly outperform SPECS on downstream tasks (e.g. up to 11.5% improved accuracy on MATH500). RSD, SPECS, and GSI all have in common that they operate on reasoning steps of reasoning models, where each iteration of the algorithm produces a subsequent reasoning step.
3 BACKGROUND
Let V denote a (finite) vocabulary. Let X = UnEN IT: V be the (countable) space of inputs, consisting of finite sequences over the vocabulary (in practice these will be prompts and already generated reasoning steps), and y = [ V the (countable) space of reasoning steps. Note that mathematically, these two spaces are identical, but we define both X' and y for notational convenience. By A(Y), we denote the set of probability measures over V. For x E X, let BB (y r) A()) and (y iz) E A()") be the base and small language model distributions over y E ) given x. Note that we define the distributions over reasoning steps instead of single tokens. When we write B.( x. v), it denotes the distribution of n over y given prompt and (partial) response When y consists of a sequence of reasoning steps y", we will denote them with superscripts y - (v- ....,'""" y",,, ") y² Y. Note that by definition of V. we can view y itself an element of y again. Subscripts yr denote different samples generated by the same model.
Reward Models. Reward models for LLMs predict how good a generated response y is for a given prompt x. They can broadly be split into two classes: Outcome reward models (ORMs) assign a reward r(z. y) a complete response y (i.e. generated until EOS) for a prompt E X. Process reward models (PRMs) (Lightman et al., 2024) instead assign a reward r(z,, (y (v':..,y'') to every partial sequence of reasoning steps (y' "u."""",""))) t = 1, ..., T. In the following, we assume we are given a PRM X x ) + 10, Rt] for some R < We assume that r approximates a golden reward (Gao et al., 2023) r" X' y + Rzo. which can be thought of as the "true" reward function.
Divergences. Recall that the Kullback-Leible divergence between two distributions P,Q E A()) with P < Q is defined as
KL(PIIQ) = Eynp [og
p(y) Q(y.
and the chi-square divergence as
x((111)=
11) dQ
dP -1 aQ
-1
/ dP2
1.
dQ
KL Regularized Reward Alignment. A standard formulation for maximizing the reward r(x, y)
given x. X, while constraining how far the policy can move from the hase policy nn( x)) is to add a KL. regularzeer, and find Th. maximizing
max E [r(, - KT. (x) T lx)),
TEA())
3

---

Published as a conference paper at ICLR 2026
where B > 0 trades off maximizing the reward versus fidelity to Tg. It is well known (e.g. (Korbak et al., 2022)) that the optimal policy has the closed form
T3 x) T x) exp 3r(2,
(1)
Zo,
( Note that sampling from this distribution becomes in- where (x) By ((112)
tractable when decoding more than one token in y at time.
Best-of-n Sampling. Best-of-n (BoN) (Beirami al., 2025) is a common inference-time method for scaling LLMs. Best-of-n draws y1,.... 1/ BB x), and selects
y arg max (1r, IE (l.....)
Since BoN greedily selects the response that maximizes the reward, is also sometimes referred to as hard best-of-n. When the reward model is suboptimal, best-of-n is known to be prone to reward hacking (Skalse et al., 2022), which can be mitigated by sampling via soft best-of-n.
Soft Best-of-n Sampling, Soft best-of-n (S-BoN) (Verdun et al., 2025) weighs each drawn sample by a temperature-scaled softmax w, x exp(3 rs, ys)) (where samples response Y: with probability w;/ E; Wi-- We denote the soft best-of-n distribution over by ( z). Note that both soft and hard BoN can be applied to one-shot generation (where the complete response it generated in one step) reasoning tasks, where the y; correspond to reasoning steps, and the BoN procedure is repeatedly applied. In this work, we focus on reasoning tasks. By moving from hard to soft best-of-n, the distribution T. distribution J (Verdun al., 2025):
an inverse temperature), then
T, x) enjoys KL bound the tilted
KT & log 1+ Var (0
n( (
In other words, the tilted distribution TE.8 can be approximated by soft best-of-n sampling by letting n + oc. Aminian al. (2025) provide thorough theoretical analysis of soft best-of-n compared
to regular best-of-n and show it can mitigate reward hacking.
4 GUIDED SPECULATIVE INFERENCE
Our goal is to (approximately) sample from the distribution T3,B. As we have seen, while one cannot sample from the distribution directly, can be approximated arbitrarily well by soft best-of- sampling with the target model πB, cmp. equation 2, which is linked to the closed-form solution of π as a reward-tilted version of mB (I). However, this requires autoregressively generating n responses from the target model, which can get prohibitively expensive. We would like to utilize small draft model Ty to accelerate inference, resemblant of speculative decoding. However, the tilted distribution (1) is distributions T, not T.. The trick is to note that write
Ts | x)) exp (Br(z, v) + log ( ))
T ())
Zz,n(x)
i.c. can rewrite it as distribution over iTS (exponentially) tilted by the tilted rewards
1
TB x))
F(I, = r(x, v) log
Ts(y xx)
with the convention log(0) = -c. This allows us to do soft best-of-n sampling over samples from Ts with the tilted rewards instead of r to approximately sample from Jg
Neward-Likelihood Tilted S-BoN. For X, the (one-step) reward-tilter tilted S-BoN is defined as:
1. sample yı,..., U/ ~ Ts( x)
2. compute F; r(x. vs) + log (m(x))) *
1
3
3. sample y: exp(
4

---

Published as a conference paper at ICLR 2026
Algorithm Guided Speculative Inference
Require: base model TB. small model ss, PRM 3 0, threshold E R. n E N, prompt x e X #empty response
1: y +
2: t 0,, ....
until EOS do
3: Sample ( ~ T 2,,) #reasoning steps, generated up to An
4: i; t r(z, (v. v() (log B x. y) log us(y) x,,)), i= 1.,,,,
5: Sample index i" ~
6:
if P.- 2 u then
7:
+ (v.
Happend step y"t_ (i**)
8:
else
9:
Sample {uf]}=1 ~ TB( y, ry r(z, (y, v)), 1,....
10:
11: Sample index j softmax(3r,,... Brn)
12: y t- (v. (f)
13:
end if
14:
end for
We will denote the distribution generated by this sampling algorithm by #asi( x). Crucially, log n (y x)) can be efficiently computed parallel by summing over logprobabilities obtained from single forward pass, as opposed autoregressive generation soft best-of-n with лт. Of course, we can only hope that Rasil- x) is close to T x)) if the support of π is sufficiently covered by TT, which we make precise with the following uniform coverage assumption, following prior work (Huang et al., 2025). This assumption is reasonable in practice, as any response has non- zero probability when sampling with positive temperature, hence the supremum Assumption finite if restricting ) to responses of some maximal length.
Assumption (Coverage Assumption). Throughout, will assume that
T z) ( x)
Coo(z) sup v
00.
Under Assumption 1, reward-likelihood tilted S-BoN with Ts indeed approximates the tilted distri- bution T in the sense of the following theorem.
Theorem 1. Let X. Assume that the coverage assumption (Assumption 1) holds. Let u E R. be an acceptance threshold (emp. Algorithm 1), and c > 0 be an arbitrary accuracy. Assume that
(x2 (TT( x))
Ts(- x)) + 1) 2811||l -1
Then,
(T( (>)) Rasi( xx))
For discussion of Theorem and practical implications, please see Appendix C.5. addition to sampling from the reward-likelihood tilted S-BoN, we also add a rejection sampling-like threshold on the tilted reward, which triggers resampling from the base model Tg in case the tilted reward falls below it. While this is not required for the distributional guarantee from Thcorem 1, it improves performance empirically, cmp. Section 5. The complete GSI method can be seen in Algorithm 1. We denote the distribution induced by Algorithm 1 (including the rejection step) as Tesi (i.e. Tasi equal to #est when the sample accepted, and equal to the soft best-of-n distribution T otherwise).
Note that in principle, it is possible choose different for the draft and target models. We leave exploring this for future research. While GSI is, in theory, applicable to one-shot generation tasks, consider y in Algorithm to be reasoning step, i.e. in each iteration t of the algorithm, drafts are generated until the end the reasoning step, which is attained when a double line break \,)) i is generated. The algorithm generates reasoning steps until an and-of-seqeenee (EOS) token is created.
In addition to the distributional guarantee from Theorem 1, we can also guarantee that the expected difference in (golden) reward goes to increases.

---

Published as a conference paper at ICLR 2026
Theorem 2 (informal). Let E X. Assume that By-xess((1]] (rr y)]
and By Byy-a at al- (x, (x.v)) v)] < co. Furthermore, assume the coverage assumption (Assumption 1) holds. Under mild assumptions on T,, (Assumption Appendix A), we have
E By (x. By--asst(aa) (2. v) n 0
at rate 0(1/vn)
Both proof's can be found in Appendix A, where we also provide formal version of Theorem 2 and an explicit bound in terms of
5 EXPERIMENTS
Models. We evaluate GSI on two model families with draft and target models of different sizes, to emphasize that GSI leads to consistent latency gains across families and sizes. For the Qwen2.5- Math family (Qwen Team, 2024), we choose Qwen2.5-Math, 1.t.5---Ma-------nsse 1.0-1...--nsstucct as the draft model TTs and Been2.5-Math-BB-Beee
as target model я. On Qwen3 (Qwen Team, 2025), we choose Qwen3-1.7B as the draft and Qwen3-14B as the target model, and disable thinking mode. We select Qwen2.5-Math-Pa Beatt--PRM-BBE 7B as the PRM r throughout. The rewards lie in [0, 1].
Implementation. We implement all models with vLLM (Kwon et al., 2023). The log-likelihoods for as are computed without any additional computational overhead within the forward pass of TT. The log-likelihoods for TB can be computed with minimal computational overhead, they only require a single forward pass through म T.. We note that for improved latency gains, verification of draft steps with the PRM and the computation of log-like ligllhoossoods of draft steps under a could be parallelized; however, for simplicity we have not implemented this in our current implementation. Each model is hosted its own GPU; we evaluated on NVIDIA A100, H100, and H200 GPUs. Our implementation is available at https: ///tttttbbba com j- www.geeeeeeegeee
Datasets. We evaluate on the following reasoning benchmarks: MATH500 (Lightman et al., 2024), OlympiadBench (He et al., 2024) (the TE.tOomaths.oo en COMP split which is text-only math prob- lems in English), Minerva Math (Lewkowyez al., 2022), GSM8K (Cobbe et al., 2021), and MMLU-STEM (Hendrycks et al., 2021) (which spans topics such as physics, chemistry. biology, math, astronomy, computer science, and electrical engineering). We decode stepwise with chain- of-thought, where "An \n" tokens denote the end of a reasoning step; rewards are computed on each reasoning step. Following common practice (Zhang et al., 2024; Cemri et al., 2025; Qiu al., 2025), we evaluate randomly selected subsets of the datasets to make evaluation feasible. We select 500 samples per datasct (note that MATH500 contains 500 and Minerva Math 272 samples, hence we the full datasets). We report 95% confidence intervals all datasets, computed from evaluations over three different random seeds with N = 500 samples each.
Methods. We compare GSI against our implementation of RSD (Liao et al., 2025) using the same hyperparameters as in the paper, S-BoN with иs. and S-BoN with яи- As SPECS (Cemri et al., 2025) does not have publicly available implementation, we do not compare it in our experiments. However, we compare to the results reported in their paper in Section 5.
Note that we do not compare to vanilla speculative decoding with the draft and target model and step- wise s-BoN sampling (i.e., where n reasoning steps are generated in parallel with vanilla speculative decoding, and then verified with the PRM), since speculative decoding is known scale very poorly with batch size. Even modern frameworks like EAGLE-2 (Li et al., 2024) have been shown to have a token throughput of up to 50% less than that of the target model larger batch sizes (Yan et al., 2025). In particular, even sophisticated frameworks like EAG, that require targeted finetuning of the draft model, have not been evaluated beyond n = 64 and do not achieve strictly better throughput than the target model alone (Li et al., 2025). GSI circumvents this issue altogether, as generation both from the draft, as well as the target model remains fully parallelizable.
Hyperparameters. We use 3 = 20 (see Appendix C.3 for an ablation), ically amongst a range of values based on accuracy vs. latency trade-off; see Appendix C.4 for an 0.7, and top p 1.0. We set the threshold in RSD to 0.7, which is the same as in the RSD paper (Liao et al., 2025). Further hyperparameter and implementation details can be found in Appendix B.
0.5 (selected empir-
ablation),

---

Published as a conference paper at ICLR 2026
MATH500
Minerva Math
MMLU-STEM
85
60
35
Accuracy (%)
80
30
50
76
25
70
40
256
16
OlynpiadBench
GSMSK
Average
Accuracy (%)
95
35
85
256
256
16
11
11
GSI
GSI w/o rej.
RSD
S-BoN (small)
S-BoN (base)
MATH500
Minerva Math
MMLU-STEM
Accuracy (%)
40
80
80
70
70
60
256
16
OlynpiadBench
GSMSK
Average
30
95
70
Accuracy (%)
60
85
80
30
50
256
16
11
I1
GSI
GSI w/o rej.
RSD
BoN (small)
BoN (base)
Figure 2: Qwen2.5-Math (top) / Qwen3 (bottom): GSI outperforms RSD (Liao et al., 2025), soft best-of-n with the draft model, and approaches the performance of soft best-of-n with the base model. We also compare against GSI without rejection step. The plots contain 95% confidence intervals over three random seeds.

---

Published as a conference paper at ICLR 2026
5.1 PERFORMANCE ON REASONING BENCHMARKS
Figure 2 compares GSI without the rejection sampling step (i.e., without lines 6 to 11 in Algorithm 1) regular GSI, S-BoN with тB and TTs and RSD. GSI significantly outperforms both soft best-of-n with the draft model, as well as RSD. We see that GSI also clearly outperforms GSI without rejection step; however, this difference becomes less significant as n increases, hinting at the fact that with larger n, the samples from the small model reach better coverage of the support of TB. Furthermore, on some datasets the accuracy GSI with and without rejection step approaches or even surpasses the accuracy of T. which empirically verifies Theorem 1. We leave investigating this behaviour beyond n = 256 for future research. An interesting observation is that amongst all methods, GSI without rejection step seems benefit most from increasing n and the only method that does not plateau around n = 256. Comparing GSI to SPECS (Cemri et al., 2025) with the accuracies reported in their paper (as there does not exist public code repository) for the same Qwen2.5-Math models, we see that while SPECS slightly outperforms GSI on OlympiadBench (n 4: +1.6%, n = 16: +3.2%). GSI significantly stronger on MATH500 (n = 4: +11.5% n = 16: +2.9%). Note that we evaluate subsets of N 500 samples, while SPECS reports accuracies on random subsets of = 100 samples, hence accuracies might not be directly comparable.
In Table 1, report the inference time per sample (in seconds) across methods (averaged over datasets), as well as the average percentage of samples accepted by GSI and RSD. RSD generally tends accept almost all samples, which explains why its performance is comparable to S-BoN with the small model (compare Figure 2) while being slightly worse terms of inference speed. GSI accepts less samples, thus is slower than RSD, while still outperforming S-BoN the base model in terms of inference speed. For example, on Qwen3 with n 16, GSI achieves a 51% increased throughput in terms of steps per second, with only 3% in relative performance degradation (emp. Table 3). While GSI tends to generate slightly more steps per problem, this still translates to up to 28% reduced end-to-end latency compared to the target model. An extended version of Table can be found in Appendix C.6. Note that inference times rely many factors and can be unreliable. For instance, we found that sometimes, vLLMM faster large batches are artificially fed in sequential chunks instead of one batch. All times reported in Table are for full batches of size Figure 4 shows how much each of the methods spends on each the three models, averaged across datasets. We note that our PRM consumes significant amount of end-to-end runtime, and more efficient (i.e., smaller) PRM would likely lead to more pronounced latency gains achieved by GSI.
Table Lateney of Qwen2.5 on H100, Qwen3 on A 1100 Inference time (in seconds) per reasoning step, number of reasoning steps per sample, acceptance rate, and steps per second (averaged across all datasets, with 95% confidence intervals over three random seeds). GSI is significantly faster than S-BoN on the base model, with up to 51% more steps generated per second.
Model Family
Method
s/step (L)
# steps
% accept
steps /s (t)
GSI (ours)
0.43 0.03
10.6 0.3
76.7 0.1
2.33 + 0.15
RSD
0.34 0.01
9.7
94.9 +.90000 0.0
2.94 = 0.08
4
S-BoN (small)
0.32 1 0.01
9.6
3.12 1 0.09
Qwen2.5-Math (H100, 1B/7B)
S-BoN (base)
0.57 0.01
1.3
1.75 + 0.03
GSI (ours)
0.72 0.05
1.2
82.0 0.1
1.39 + 0.09
RSD
0.61 0.01
10.3
+7.30000
1.64 + 0.03
16
S- S-BoN (small)
0.52 0.03
0.1
1.92 0.10
S-BoN (base)
0.94 0.03
10.5
1.06 + 0.03
GSI (ours)
0.56 0.04
26.7 00.7100 0.3
88.0 1.0000 0.1
1.79 + 0.12
RSD
0.40 0.01
2.1
97.2
2.50 0.06
S-BoN (s)
0.38 1 0.01
24.8
2.63 - 0.07
Qwen3
S-BoN (h)
0.83 0.02
2..9
1.20 + 0.03
(A100, 1.7B/14B)
GSI (ours)
1.21 0.05
27.5
91.5
0.83 = 0.03
RSD
0.96 + 0.01
29.0
98.2 0.1
1.04 + 0.01 0.01
16
S- S-BoN (s)
1.01 + 0.00
2.2
0.99 = 0.00
S-BoN (b)
1.82 1 0.06
24.8 0.3
0.55 1 0.02
8

---

Published as a conference paper at ICLR 2026
GSI: MATH500, Example (Qwen2.5-Math)
Prompt: In the soquence 0, I. 6. 9. the first term Subsequent ferms produced by alternately adding and multiplying by each successive inleger beginning with 1. For instance, the second term produced by adding lirst term; I: the fourth term produced adding the third and
the third produced multiplying the secund on What is the value of the first term that greater than 125?
Step
Generated Reasoning Steps Draft
Target
P(a 1/((w
To deternine the value of the lins tern in the sequence that is greater than 125, need to fullow the rule alternately by
abding and mulliplynng each sucoessive integer starinn from Let's
0.719
The fest temm ea D. 2, The second form ueg +-1
D
1. The third tgrm
x I an 1. The fourth term
2 - 3. The fitth term
sight tenn ag 07
0g
497.
1. The first temm given as 0. 2. The sacond term obtained by adking to the first temm
3. The third temn obsained muktiplying second term by
as
temn is obtained by muhiplying the fourth term by 2: as obtained by adding to the fifth tem:: as
x
6. The sinth term is 3 9. 7. The seventh serm as obtained by
2
x
+ 3
0.979
mulliplzing the sinch by
x
The eighth obsained adding obtaiee muttiplying eighth The tenth term obtained by adding to the ninc term:
27.
ar +
+
temm by ag as 4
124.
10.
124 129.
We see that the senth temm 129, is the fins uemm in the sequence that greater tham 125. Therefore, the value
3
of the first term that greater than 125 is 129
1.087
Correct Answer found: True
GSI: MATH500, Example (Qwen2.5 (owen2.5-Math M-Mthh
regular pentugon rotated countercluckwwse center. rotated until it coincides with its original position?
Prompl:
degrees
Step
Generated Reasoning Steps Draft
Target K
F(u (.
To determine the minimam number of degrzes regalar pentagen must be rotited cosaterclackowie about its cester coincides original position, need understand symmetry pentagon. regular pentagon symmetry, meaning looks some after being roeated by contain angles around its center.
regular pentagan las fiee ring can rotited to five different positions thas look the sume: Ligrefore, the angle cotation for each symunetry is 360 divided by
The the angles point
A regular pentagon has live equal sides and live equal angles When we notuie regular pentagen, it will coincide with original pasition that muitiples calculltedd by dividing
(360 degrees) by subtended sides pentagon,
vertiees
augle
angle
aGp
3
72
0.968
5
Therefore, the minisum number of degrees the pentagon must be ritatal comnterlloskwise until it coineides with its
4
original position 72
0.961
Correct Anewer found. Trie
Figure 3: Reasoning traces generated by GSI two MATH500 samples. Top: GSI correctly
identifies that the second step generated by the draft model is wrong (crossed out means rejected) and resamples from the base model. Bottom: Sometimes, GSI rejects steps that correct if the base model tends to word them very differently from the draft model.

---

Published as a conference paper at ICLR 2026
BoN (base)
GST
RSD
S-RoN (small)
10
20
30
40
Seconds per sample (rotal runtime)
w
PRM
Figure 4: Qwen3 on A100 runtime breakdown across ms (I.7B), TB (14B), and the PRM (7B).
5.2 GSI REASONING TRACES
Figure shows two sample reasoning traces generated by GSI with the Qwen2.5-Math models for 4 on MATH500. The last column contains the tilted rewards F for samples from ws, and regular rewards r for samples from тB, aligning with the GSI algorithm. correctly identifies incorrect step generated by Ts in the second step by its small tilted reward, and resamples a correct step from n, whereas the second example shows that tilted rewards F can also sometimes be misleading. We provide additional samples in Appendix C.7, including comparisons to the reasoning traces generated by RSD, and examples that highlight the advantage of using tilted rewards instead of raw rewards.
the first example, GSI
5.3 ABLATIONS
Appendix C contains additional experiments, including more detailed accuracy results, a more de- tailed comparison of the acceptance rates of GSI and RSD, a discussion of Theorem and its prac- tical implications, and ablations with Qwen2.5-Math on MATH500 over 3 and over u, which show that our choice of 3 = 20 strikes balance between weighing and the log ratio log( B /////// Ts), and that the threshold u = 0.5 is optimal for smaller values of n. Note that while the ablations show that 8 = 20 and periments confirm that these can be used out-of-the- across datasets and model families without further hyperparameter search. However, the performance of GSI can likely be improved with more fine aiee-graind threshold schedule {u, }n, depending on n, which we leave for future research.
0.5 are sensible choices for MATH500 with Qwen2.5-Mait our ex-
6 DISCUSSION
Developing compute-efficient algorithms remains critical challenge in test-time scaling of lan- guage models. In this work, we introduce Guided Speculative Inference (GSI), novel inference- time algorithm for efficient reward-guided decoding from target language model. GSI leverages speculative samples from a small draft model to approximate the optimal tilted policy of the target model with respect to given reward function. We show that unlike previous approaches. GSI prov- ably approaches the optimal policy as the number of samples n generated at each step increases, and can provably achieve expected rewards arbitrarily close the optimum. Empirical results on vari- ous reasoning benchmarks (MATH500, OlympiadBench, Minerva Math, MMLU-STEM, GSM8K), model families (Qwen2.5-Man and Qwen3) and sizes ranging from 1B to 14B parameters show that GSI consistently and significantly outperforms existing approaches, such as reward-guided specula- tive decoding (Liao et al., 2025), SPECS (Cemri et al., 2025), soft best-of-n with the draft model, and, perhaps surprisingly, even surpasses soft best-of-n with the target model in some cases. Re- sults on inference time show that GSI can efficiently trade off inference time compute for significant performance gains, making it practical framework for efficient L..M deployment.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY
The complete code needed reproduce our results available at https://github. com/ GSS..
ACCNOWLEDGMENTS
We would like to thank Nick Hill and Guangxuan (GX) Xu from Red Hat for their help with vLL.M JG and DAM acknowledge support from the Aramont Fellowship Fund and the FAS Dean's Com- petitive Fund for Promising Scholarship. JG is supported by graduate fellowship the Kempner Institute for the Study of Natural and Artificial Intelligence at Harvard University.
REFERENCES
Gholamali Aminian, Idan Shenfeld, Amir Asadi, Ahmad Beirami, and Youssef Mroueh. Best- of-n through the smoothing lens: KL divergence and regret analysis. In ES-FoMo IlI: 3rd Work- shop on Efficient Systems for Foundation Models, 2025. URL https: / /ectt///ppeee /pppeeeveeeeeeeeeeeeeeeeeee net/ forum? www.wk/wwWX........
Ahmad Beirami, Alekh Agarwal, Jonathan Berant, Alexander Nicholas D' Amour, Jacob Eisenstein, Chirag Nagpal, and Ananda Theertha Suresh. Theoretical guarantees the best-of-n alignment policy. In Forty-second International Conference on Machine Learning, 2025. URL https: //openzeview.
Nikhil Bhendawade, Irina Belousova, Qichen Fu, Henry Mason, Mohammad Rastegari, and Mahyar Najibi. Speculative Streaming: Fast LLM Inference without Auxiliary Models, 2024. URL
https: http://xxxvvvvr/ee abs /ww.////222. 11131.
Mert Cemri, Nived Rajaraman, Rishabh Tiwari, Xiaoxuan Liu, Kurt Keutzer, Ion Stoica, Kannan Ramchandran, Ahmad Beirami, and Ziteng Sun. SPECS: Faster test-time scaling through spec- ulative drafts. In ES-FoMo III: 3rd Workshop on Efficient Systems for Foundation Models, 2025.
URL wwt.tttps:// /openrev ew net. forum? i d==wRti FTM5b.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training Verifiers to Solve Math Word Problems, 2021. URL https://arxk
oxg/abs/2110. 14166.
Leo Gao, John Schulman, and Jacob Hilton. Scaling Laws for Reward Model Overoptimization. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference Machine Learning, volume 202 of Proceedings of Machine Learning Research, pp. 10835-10866. PMLR, 23-29 Jul 2023. URL https: http///ooceeeeggeeeeeeeeeeeoeoce mlr wwwpsppsssss...p.....e 2tress/v202/gao22222 .html..
Gemini Team. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of con- text, 2024. URL htwps:tpps.. http:://aaivvvvoee wrg. /ttg//b/////r/ 2403 05530.
Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, Jie Liu, Lei Qi, Zhiyuan Liu, and Maosong Sun. OlympiadBench: A challenging benchmark for promoting AGI with olympiad-level bilingual multimodal scien- tific problems. In Lun-War Ku, Andre Martins, and Vivek Srikumar (eds.). Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pa- pers), pp. 3828-3850, Bangkok, Thailand, August 2024. Association for Computational Linguis- tics. doi: 10. 1/t.3//11/22..llllbbaeo acl-lone 211/.
URL https: /awlallloo wellanthhlogyyy org/ 2024.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song. and Jacob Steinhardt. Measuring Massive Multitask Language Understanding. In International Confer- ence on Learning Representations, 2021. URL https:/ /ttp:////p///eeee//////
/ net fortt/froumid
d7KB.jmI3Gmg
11

---

Published as a conference paper at ICLR 2026
Danny Hernandez, Tom Brown, Tom Conerly, Nova DasSarma, Dawn Drain, Sheer El-Showk, Nel- son Elhage, Zac Hatfield-Dodds, Tom Henighan, Tristan Hume, Scott Johnston, Ben Mann, Chris Olah, Catherine Olsson, Dario Amodei, Nicholas Joseph, Jared Kaplan, and Sam Mc- Candlish. Scaling Laws and Interpretability of Learning from Repeated Data, 2022. URL httpss://wxvvv..rg/ooo. 22220555 10487.
Audrey Huang, Adam Block, Qinghua Liu, Nan Jiang, Akshay Krishnamurthy, and Dylan J Fos- ter. Is Best-of-N the Best of Them? Coverage. Scaling, and Optimality in Inference-Time Alignment. In Forty-second International Conference on Machine Learning, 2025. URL
net./ wwt.:/eeeeee/eeeeeeeeeeeee.m. fkhrrrr fkhrbYK.
forum?id=Qn
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling Laws for Neural Language Models, 2020. URL https: / aaxivvvv brg/abs/// 2001. 08361.
Tomasz Korbak, Ethan Perez, and Christopher Buckley. RL with KL penalties is better viewed as Bayesian inference. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.). Find- ings of the Association for Computational Linguistics: EMNLP 2022, pp. 1083-1091, Abu Dhabi, United Arab Emirates, December 2022. Association for Computational Linguisties. doi: 10118653/1/2022 htw.://........nneemmmmmm.. findings-emmlp.77. URL https: / https///ccaanthooggoo org//022 2022.. findings-emn p. 77//
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and lon Stoica. Efficient Memory Management for Large Language Model Serving with PagedAttention, 2023. URL https: http//aavvvvvv///////ee////aa org/ abs 2309 60..0
Yaniv Leviathan, Matan Kalman, and Yossi Matias. Fast inference from transformers via speculative decoding. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org, 2023.
Aitor Lewkowycz, Anders Johan Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Venkatesh Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Son -Sloo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. Solving Quantitative Rea- soning Problems with Language Models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https: / wpw.review.eeeeeeeeeeeeee..eewwe net/ id=
Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. EAGLE-2: Faster inference of lan- guage models with dynamic draft trees. In Empirical Methods in Natural Language Processing, 2024.
Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. Eagle-3: Scaling up inference accel- eration of large language models via training-time test, 2025. URL httpt:tt///// /atpaaaxvvvoor.... abs/2503. 01 140.
Baohao Liao, Yuhui Xu, Hanze Dong, Junnan Li, Christof Monz, Silvio Savarese, Doyen Sahoo, and Caiming Xiong. Reward-guided speculative decoding for efficient LLM reasoning. In Forty- second International Conference on Machine Learning. 2025. URL net beet/fose wtt////rummm/// weriiiivAVeseeeeeeee
Hunter Lightman, Vincet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2024. URL https: httpt://///pgg /ttttts://pperreee /ttp:///eee/e.e.ee//
Jiahao Liu, Qifan Wang, Jingang Wang, and Xunliang Cai. Speculative decoding via early-exiting for faster LLM inference with Thompson sampling control mechanism. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics: ACL 2024, pp. 3027-3043, Bangkok, Thailand, August 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024 fin.0244..niing-al... 179. URL https: httpps::///aaaaa/aee /accanthooogyyy.....oooo .org/ 2024. fi Findings-enn .79/
12

---

Published as a conference paper at ICLR 2026
Youssef Mroueh and Apoorva Nitsure. Information Theoretic Guarantees For Policy Alignment In Large Language Models. Transactions on Machine Learning Research, 2025. ISSN 2835-8856. URL btttps://cpereevieww net
Sidharth Mudgal, Jong Lee, Harish Ganapathy, YaGuang Li, Tao Wang, Yanping Huang, Zhifeng Chen, Heng-Tze Tze Cheng, Michael Collins, Trevor Strohman, Jilin Chen, Alex Beutel, and Ah- mad Beirami. Controlled Decoding from Language Models. In JCML, 2024. URL https: // //openreview
Niklas Muennighoff, Alexander M Rush, Boaz Barak, Teven Le Scao, Nouamane Tazi, Alcksandra Piktus. Sampo Pyysalo, Thomas Wolf, and Colin Raffel. Scaling Data- Ceta-Consttriiedd Language Models. In Thiirt--eventth Conference on Neural Information Processing Systems, 2023. URL
pettps:/cpeereeee
net/ wwweeeeetoouteeeee forum? dd= J5BTTH33
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hanmanch Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling, 2025. URL https: http:ss////xvvvvvo.v/.e abs
OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad. Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji. Valerie Balcom, Paul Baltescu, Haiming Bao, Moham- mad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Sernadett-Shaping Christopher Berner, Lenny Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-Luisa Brakman, Greg Brock- man, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittany Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang, Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho. Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix, Simón Posada Fishman, Juston Forte. Isabella Fulford, Leo Gao, Elie Georges, Christian Gib- son, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Gontijo-Lopes, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufei Guo, Chris Hal- lacy, Jesse Han, Jeff Harris, Yuchen He, Mike Heaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey. Peter Hoeschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang, Haozhun Jin, Denny Jin, Shino Jomoto, Billie Jonn, Heewoo Jun, Tomer Kaftan, Lukasz Kaiser, Ali Ka- mali, Ingmar Kanitscheider, Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim, Jan Hendrík Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, Łukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis, Kyle Kosic, Gretchen Krueger, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy Lee, Jan Leike, Jade Leung, Daniel Levy, Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz Litwin, Theresa Lopez, Ryan Lowe, Patricia Lue, Anna Makanju, Kim Malfacini, Sam Manning, Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinney, Christine McLeavey, Paul McMillan, Jake McNeil, David Medina, Aalok Mchta, Jacob Menick, Luke Metz, Andrey Mishchenko, Pamela Mishkin, Vinnie Monaco, Evan Morikawa, Daniel Mossing, Tong Mu, Mira Murati, Oleg Murk, David Mély, Ashvin Nair, Reiichiro Nakano, Ra- jeev Nayak, Arvind Neelakantan, Richard Ngo. Hyeonwoo Noh, Long Ouyang, Cullen O' O'Keefe, Jakub Pachocki, Alex Paino, Joe Palermo, Ashley Pantuliano. Giambattista Parascandolo, Joel Parish, Emy Parparita, Alex Passos, Mikhail Pavlov, Andrew Peng, Adam Perelman, Filipe de Avila Belbute Peres, Michael Petrov, Henrique Ponde de Oliveira Pinto, Michael, Pokorny, Michelle Pokrass. Vitchyr H. Pong, Tolly Powell, Alethea Power, Boris Power, Elizabeth Prochl, Raul Puri, Alec Radford, Jack Rae, Aditya Ramesh, Cameron Raymond, Francis Real, Kendra Rimbach, Carl Ross, Bob Rotsted, Henri Roussez, Nick Ryder, Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry, Heather Schmidt, David Schnurr, John Schulman, Daniel Sel- sam, Kyla Sheppard, Toki Sherbakov, Jessica Shich, Sarah Shoker, Pranav Shyam. Szymon Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song, Natalie Staudacher, Felipe Petroski Such, Natalie Summers, Ilya Sutskever, Jie Tang, Nikolas Tezak, Madeleine B. Thompson, Phil Tillet, Amin Tootoonchian, Elizabeth Tseng, Pre- ston Tuggle, Nick Turley, Jemry Tworek, Juan Felipe Cerón Uribe, Andrea Vallone, Arun Vi- jayvergiya, Chelsea Voss, Carroll Wainwright, Justin Jay Wang, Alvin Wang, Ben Wang, Jonathan
13

---

Published as a conference paper at ICLR 2026
Ward, Jason Wei, CJ Weinmann, Akila Welihinda, Peter Welinder, Jiayi Weng, Lilian Weng, Matt Wiethoff, Dave Willner, Clemens Winter, Samuel Wolrich, Hannah Wong, Lauren Work- man, Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan, Wojciech Zaremba, Rowan Zellers, Chong Zhang, Marvin Zhang, Shengjia Zhao, Tianhao Zheng, Juntang Zhuang, William Zhuk, and Barret Zoph. GPT-4 Technical Report, 2024. URL https://arxiv: http://awwxxi.ogaao//o////oo wttp://///aaoooooooot// 08774.
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang. Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kel- ton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. Ad- vances in Neural Information Processing Systems, 2022. URL /tttp:://arxii org//bs////////// 2203. 02155.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, Alban Desmaison, Andreas Köpf, Ed- ward Yang, Zach Vito,o Martin Raison, Alykhan Tejani, Sasank Chilamkurthy, Benoit Steiner, Lu Fang, Junjie Bai, and Soumith Chintala. PyTorch: An Imperative Style, High-Peeformance Deep Learning Library, 2019. URL https:////// /tte://arxxiis www.iiiw.vvaaaa.ooooooooooo
017031
Jiahao Qiu, Yifu Lu, Yifan Zeng, Jiacheng Guo, Jiayi Geng, Chenhao Zhu, Xinzhe Juan, Ling Yang. Huazheng Wang, Kaixuan Huang, Yue Wu, and Mengdi Wang. TreeBoN: Enhancing Inference-Time Alignment with Speculative Tree-Search and Best-of-N Sampling, 2025. URL https: //tpss///xxivvv
arxivv. ats//b//24111 16033.
Yuxiao Qu, Tianjun Zhang, Naman Garg, and Aviral Kumar. Recursive Introspection: Teaching Language Model Agents How to Self-Improve. In The Thirty-eighth Annual Conference on Neu- ral Information Processing Systems, 2024. URL https: /ttp:s:////pp/eeeeeeeee net htt//foummmico DRC9p2WBwR.
Qwen Team. Qwen2.5: A Party of Foundation Models, September 2024. URL https:// qwenlm. github. httblo/////////o/oeo /wenn////eee223mmmm 5/.
Qwen Team. Qwen3 Technical Report, 2025. URL
09388.
Joar Max Viktor Skalse, Nikolaus H. R. Howe, Dmitrii Krasheninnikov, and David Krueger. Defin- ing and Characterizing Reward Gaming. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), Advances in Neural Information Processing Systems, 2022. URL https: /ttpr//eeeeeeeeeeeeeeeeeeee net. / forum? 1d=yb3H00031X2.
Charlic Victor Snell, Jachoon Lec, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. In The Thirteenth Interna- tional Conference on Learning Representations, 2025. URL http::::///peeereiicc net/
Gorum?id=4FWAw2td02.
Ziteng Sun, Ananda Theertha Suresh, Jae Hun Ro, Ahmad Beirami, Himanshu Jain, and Felix Yu. SpecTTr: Fast Speculative Decoding via Optimal Transport. In Thirty Conference on Neural Information Processing Systems, 2023. URL https: /ttps/////peeeeeeeeeeeeeee n forum?
id=SdYHLTCC5J.
Ziteng Sun, Uri Mendlovic, Yaniv Leviathan, Asaf Aharoni, Jae Hun Ro, Ahmad Beirami, and Ananda Theertha Suresh. Block Verification Accelerates Speculative Decoding. In The Thirteenth International Conference on Learning Representations, 2025. URL htttss///pperevieee net / ftrr//orummm id= frsg32u0r0.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshom, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anna Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra,
14

---

Published as a conference paper at ICLR 2026
Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, lliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models, 2023. URL https::/// httLl:ttt///aaattt/ http:://a.vv..oo........ /200s/237...2 022.202077.0......
Claudio Mayrink Verdun, Alex Oesterling, Himabindu Lakkaraju, and Flavio P. Calmon. Soft Best-of-n Sampling for Model Alignment, 2025. URL
03156.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe V Le, Ed H. Chi. Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-Consistency Improves Chain of Thought Reasoning in Lan- guage Models. In The Eleventh International Conference Learning Representations, 2023. URL htw.htttss//opeeevvieeee
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowiez, Joe Davison, Sam Shleifer, Patrick Von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. Transformers: State-of-the-art natural language processing. In Qun Liu and David Schlangen (eds.), Proceedings of the 2020 Confer- ence on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38- 45, Online, October 2020. Association for Computational Linguistics. doi: 10. (0.18653/v120200 emnlp-demos.6. URL https:://// /atpaaaaannnnoooooaoaaaoa org 22200 emn.p-demos.........mommo. 6/.
Siyuan Yan, Mo Zhu, Guo qing Jiang. Jianfei Wang, Jiaxing Chen, Wentai Zhang, Xiang Liao, Xiao Cui, Chen Zhang, Zhuoran Song, and Ran Zhu. Scaling laws for speculative decoding, 2025. URL https:tt///.. / /tt./aaxvv..../ org/ abs/ 2505. 07858.
Kevin Yang and Dan Klein. FUDGE: Controlled Text Generation With Future Discriminators. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Lin- guistics, 2021. doi: 10.18653/v1/2022 ma..cl-mann.27.. URL http: /xxx org/10. 18653/v1/20211 www.nnaacaaaaa.a 276.
Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Wenyue Hua, Haolun Wu, Zhihan Guo, Yufei Wang, Niklas Muennighoff, Irwin King, Xue Liu, and Chen Ma. A Survey on Test- Time Scaling in Large Language Models: What, How, Where, and How Well?, 2025. URL https: / arxivv. org/ abs// 2503.20002 250122223.........
Ruiqi Zhang, Momin Haider, Ming Yin, Jiahao Qiu, Mengdi Wang, Peter Bartlett, and An- drea Zanette. Accelerating Best-of-N via Speculative Rejection. In 2nd Workshop on Ad- vancing Neural Network Training: Computational Efficiency, Scalability, and Resource Op- timization (WANT@ ICML 2024), 2024. URL https:/ /ttps/////pereeew../o net forum?id= dRp8tAIPhj.
15

---

Published as a conference paper at ICLR 2026
CONTENTS
Introduction
Related Work
3 Background
4
Guided Speculative Inference
Experiments
5.1
Performance on Reasoning Benchmarks
5.2
GSI Reasoning Traces
10
5.3
Ablations
10
Discussion
10
Proofs
17
A.1 Proof of Theorem
17
A.2 Proof of Theorem
18
B Implementation Details
21
B.1 System Prompts
B.2 Generation Details
21
Additional Experiments
C.I Extended Accuracy Results
21
C.2 Acceptance Ratios
C.3 Ablation over 8
21
C,4 Ablation over 0
22
C.5 Discussion of Theorem
22
C.6 Runtime Comparison
25
C.7 Reasoning Traces
25
Assets
27
D.I Hardware
27
D.2 Libraries
27
D.3 Code Repository
28
Use Large Language Models
28
16

---

Published as a conference paper at ICLR 2026
A PROOFS
A.1 PROOF OF THROREM
Theorem 1. Let X. Assume that the coverage assumption (Assumption 1) holds. Let E R be an acceptance threshold (emp. Algorithm 1), and e > 0 be an arbitrary accuracy. Assume that
z (T)(TT()))
x)) TT( x)) + 1) 2281|rll=>
e
Then,
KT (T [r) # ()) S c.
Proof. By Lemma 1 in (Verdun et al., 2025) (which equally holds for countable spaces V), we have T x)) exp Br(x, u) + log TB x T
TosI(y xx) 2
exp Br(s, v) + log (=(y)=) [s(y(=)
Te n-1 B
[r(aw))
Tn a)) (((
1 TB σ)
e ((m))) + n-1 E Eg'NTa(( ( [can(as()]
Ts x))
Hence
T x) log
KT. (T,TT,,llAGsS)
Σ y
(y 1x)
#csi (y x))
B x) (r(z.a) + "- " ( (r
TB(U x) (r(
Tn x) (r(.))
TT( x))
log
Ey ( [fr(z.)r)]
Ey By'nnol (11x)///8eee (e TBU x) or(anw)
an x) Br(z,))
( | x))
e Br(ay)
log
n-1 E ()) ( [eDr(av)))
Ey [e
T5( 2))
7
TB x²2
2(,,2)))
< log
+ n-1
(Jensen's 's inequality)
IT (y x)
(Ey (Byr=an(t))lesree../)
29 Ir (TB(( x) || Ts(: x)) +1
< log
n-1
(Bynro((=))e le ()) r(x,,/11))
(x2(Fn(-1)) ns(-lx)) +1) 2011 n-1
log
using the fact that
Ey Be ar(xz00)) 1 21
since r(z, v') 2 0. Now for O, have
(x (1( 20110
log
+ n-1 5 c
(x2 (T)())))) m5((12) 2001
1 1 S e
2011-1100
1 +
< E
(l 200111000
$ 6" -1 1,
(x²(TT( x) IT(s x)) + 1) 201 -
5
17

---

Published as a conference paper at ICLR 2026
A.2 PROOF OF THEOREM 2
Assumption 2. Let Yz = {v F(y) 2 wJ. Assume that
1
Jss ('()dm3,,n00) 2 J r' ("())dr3, mly))
(Y,)
words: 73,n has higher average golden rewards r² on the set Yz than on the entire set Y. Furthermore, assume that
T (Y2) #Ys)(Y2),
words: Tg.n is more likely to generate samples with high tilted rewards than fres--
Theorem 2. Let x E X. Assume that Exas [rr] < oo and [r < oc (here we implicitly assume that distributions and rewards are conditioned on I, which we omit for ease of notation). Furthermore, assume that Assumptions I and 2 hold. Denote by p(u) the acceptance probability of GSI. Then
||r* Vn
E "rrl-Errstt [r*] <
((uu) (x²(TBITTS) +1) + p(()) (CV( +2 {
Vary' (oy--aptooooolttaa)
where CV(ePr):= := V (Ryrngt (edr))no'1))) In particular, we have Eross [r-]- - [r]]] at rate O(I/VT).
10000
Proof. Denote by Y2 C Y the set where f(y) 2 u, and Y = Y Y2. Letx E N. We note that for y E 22, we have
Ty) #Css (V) + #asi #Gsi((Y))))))))))) (Y<)=ī,B(y))
since any y E Ys can be generated either from RGst directly (in which case the sample is accepted by the algorithm), or the sample from Rast is rejected (which happens with probability TTGi(Y=)) in which case y can be generated with S, Similarly, for y E Ye.
(v) #esi (Y²) (Y<)=š,a(v)
(3)
as y E Y² can only be generated by Tast if sample from #ast is first rejected and then y is generated by zs. Thus,
ET [rr) Eresi = Eng,o [1yrr'] E#osi r" 1-1 #asi (rasi(Y²2B) (Y2) g. [y.r r*1 + Exa [1x-r"] #Gsi (Y= (Y)) [1x
E.gg rrrr Fiast [1 r²]++ E.g, [1. r")
Bag n [1r r*+++++
#asi Y |B 3. [Ix #GSI(Y B 3. [Lyrr"]
E [1y2r"]] Eiest [Iy2r"]
+ + Emg.a [1 [1 Baz [1 + Tast (Yz E a.8 [1 r"] Trst (Yz) Mg
R.8. [1yr"] Tost (Y< )Bg [1y, r'] [I] #ost (Y2) [1ys
Eng,o [1y, r"1 Eose [1ygr] r"] + Emg, [1y r"1 - E [1y r'1 + #Gsi (Yz En Bn [r*] - E Em 1yg r*]
(0)
Step 1: Bounding (a). We have by Sahwhy-Schwarz:
(a) By (yy (v) r (Ir, v)] - Ey- [Iys (v) r (x,
< |||"" 1mx (fIva Irg (1) ducsi(y xx) (s(
TT. x)) Fasi (x) l
2 Rcsi (dy [x)
#oSi( x))
=||r"|be (Fosi(Yz xz))² (x²(Fe)(() 1x) #osi(- [z)) 1/2
(4)
18

---

Published as a conference paper at ICLR 2026
By Lemma from (Verdun al., 2025) we have
x (T, x))
T x))
(5)
T3 x)2
dy 1
Fo x)
(TB(U x)) a r(x.)))) (e
dy
masily x))
(B( x) ) 2 (By [0 (()())))))))
1 ma(yle) (r((a.))
n-1 By ay (e
Lemmal
*s(N)z)
dy 1
TB 2) (r(()
TB x)2 TT x)
23r dy n n-
-1
(By (pyneett 201111l
(x (T [)][ x)) + +1)
11 (1yv-
20 (x²(TB(- (((TT(-|2)]aa x1][s(( x)) + +1)
(6)
Plugging equation into equation yields
||o (Yusi(yz x)) (" 2011 (xB(TT) +1) 31
(a)
|r'' al (x2( + +1)
p(u)
vn
Step 2: Bounding (b). Similar to the bound for (a), we get
(b) / Iy. (w)r" FB | x)) - 73 x))
(dy x)
T.,(" 2x)
$ ((Irr(W)000 10) 1 (1
TT. x2) ( xx)
T/ B (dy xx)
Th TB, x))
(TRAOll I x)) (1
TB.oly x))
$ < ||
T Td xx)
TB x)
< (1 - p(u)) || (TT
(8)
by Cauchy-Schwars In the last step, we used the second part of Assumption 2, which implies Th (Y²) #as)(Y²), hence
T TB (Y=) < (t (Y2)
by equation
19

---

Published as a conference paper at ICLR 2026
Again, using Lemma I from (Verdun et al., 2025) we get
T3,0(y x)²
B. x) dy
Lemma
/
(BY x)) ( 2
(( + n-1 Eyyoasl 2) (eBr(an)))
n
dy 1
E y - [e E (22((22)0))
TB 2) e8lzz.))
11
n By B (r())))))))
yy ( 28
n1 (By w (11z) ( Var Varg Valg
(Ey (1)) (e +
Plugging equation into equation yields
(b) ||r vn
Var, Very-na (eOr(aw))
p(u))
(1 +1)
(10)
Step 3: Bounding (c). We have by Assumption 2:
(c) Tasi (Y> ) Mm [r'] - Ewg [1rg r*]
#asi (Yz)
E Fx ry] Bay [4y r"]
3 (Yz)
0
(11)
where we use the first part Assumption 2 the first inequality, and the second part the second inequality. Combining equations (7), (10), and (11) gives
E (ra. [r*]
||r 1loc
(p(u) (x²(TT)ITT) + +1) +11 (1- p(u)) (CV( + +1)
Vn
as desired.
Remark 3. Asymptotically, also get
Brast [r*] Ems. [r*] n=000
without assuming the second part in Assumption since by Theorem combined with Lemma from Verdun et al. (2025), we get
KL (iGss)
n 20+00
and (I n-000 0,
which implies
TGST (Y>)
1,
(Yz)
assuming that (Y>) 0. This means that term (c) in the proof of Theorem converges to 0 (not necessarily from below). Furthermore, one could simply bound the term T. ( appearing in our derivation of the bound on term (b) by 1, which avoids relying on the second part of Assumption 2 for term (b) and guarantees the asymptotic behaviour above.
20

---

Published as a conference paper at ICLR 2026
B IMPLEMENTATION DETAILS
This section contains additional implementation details.
B.1 SYSTEM PROMPTS
We slightly adapt system prompts based on the dataset. Our base system prompt is:
"Please reason step by step, and put your final answer within [[boxed()."
On Minerva, append it by
Do not include units in your final answer. For example, if the answer is 15 m/s', write "[[boxed(5)'""
On MMLU, we instead use
"Please reason step by step, and select the answer from the given choices 1, 2, 3, or 4. Respond only with the number of the correct answer, from 1 to 4, not with the answer itself. Put the index of the correct answer within |]boxed[]."
B.2 GENERATION DETAILS
We max_new_tokenn in vLLM 512 (this is the maximum number of tokens per reasoning step). If the rewards of all draft steps lie below 0.1, we stop generation for that sample and count it as "solved incorrectly", as we have observed that such generations lead to incorrect solutions. On Qwen2.5-Mar models, we had used maximum number of reasoning steps of 45 (after 45 steps without finding solution, the sample counts as "solved incorrectly"). However, as Qwen3 models tend to generate significantly larger numbers of reasoning steps, we increased this limit to 100 on Qwen3, We use a maximum context window of 8192 for all three models (if response exceeds this context size, it counts as "solved incorrectly").
C ADDITIONAL EXPERIMENTS
C.I EXTENDED ACCURACY RESULTS
In Tables and 3, we report the average accuracies of GSI, RSD, S-BoN with тS, and S-BoN with n. for both model families. While GSI outperforms RSD and S-BoN with सş both model families, this difference is more significant with the Qwen3 models, since the Qwen3 draft and target model exhibit larger performance difference than our Qwen2.5-Math models.
C.2 ACCEPTANCE RATIOS
In Figure 5, plot the average acceptance ratio of GSI and RSD for both Qwen2.5-Math and Qwen3. The acceptance ratio GSI increases from an average 70% (Qwen2.5-Math) and 80% (Qwen3) around 90% at 256. That of RSD significantly higher, increasing from 90% (Qwen2.5-Math) resp. 95% (Qwen3) to almost 100% (n 256), suggesting that as n increases, RSD collapses soft best-of-n with Rs, least without more careful hyperparameter tuning. We note that more intricate acceptance threshold schedules could stabilize acceptance rates across n, compare Section C.4. We leave exploring such approaches for future research.
C.3 ABLATION OVER
GSI relies temperature-scaled soft best-of-n sampling with parameter & (corresponding to an inverse temperature) both for samples from ns, as well as for samples from TØ in case the draft sample gets rejected. Increasing & leads convergence greedy best-of-n, while reducing it verges to random choice. To better understand the behaviour of GSI in terms 3, we evaluate GSI with Qwen2.5-Mo Math MATH500 for different values of f3, ranging from B 0 (i.e. ignoring r,
21

---

Published as a conference paper at ICLR 2026
Table Qwen2.5-Math: Accuracy on reasoning benchmarks (95% confidence intervals over three random seeds). GSI consistently outperforms RSD (Liao et al., 2025) and soft best-of-n (S (S-BoN) with the small model. S-BoN with the base model represents the target distribution. On average, GSI surpasses all baselines and closely approaches the performance of the base-model S-BoN. As grows, performance saturates.
n
Method
MATH500
OlympiadBench
Minerva
MMLU
GSM8K
Average
GSI (ours)
75.3 0.5
35.6 1.2
27.8 + 2.1
42.8 + 1.6
90.3 + 0.3
54.4 1.1
RSD
74.0 1.2
34.5 + 0.6
24.3 + 1.
46.1 1.7
89.0 1.2
53.6 1.3
S-BoN (s)
71.1 + 1.9
34.1 1.5
22.1 + 0.1
39.2 0.0
85,5 = 1.4
50.4 = 1.0
S-BoN (b)
80.0 3.1
32.9 = 1.7
31.0 1.6
1.3
1 0.9
58.2 1.7
GSI (ours)
0.9
39.7 1.1
27.5
0.6
94.9
59.5 0.7
RSD
79.5 1.0
40.1 +0.1110 1.0
24.5
1.5
92.3 0.3
57.8 +
S-BoN (s)
77.4
39.6
23.6 0.3
47.7 1.9
89.4
55.5 1.2
S-BoN (b)
83.1 1.6
38.1
57.1 32.3
95.9 0.5
61.3 0.8
GSI (ours)
82.2 0.6
40.8 1.3
28.2 :: 1.8
52.3 + 1.5
96.3 0.1
60.0 1 1.1
RSD
80.0 + 0.9
41.5 1.6
26.4 2.1
52.3 + 1.5
93.3 + 0.7
58.7 1.4
S-BoN (s)
79.7 1.3
41.3 0.2
25.2 0.5
50.9 + 1.2
92.0 0.7
57.8 0.8
S-BoN (b)
83.6 1.3
41.1 1.8
33.6221
56.1 0.9
96.3 = 0.6
62.1 = 1.3
GSI (ours)
83.4 + 0.5
42.0 - 1.6
29.6 - 0.6
52.5 0.6
96.0 = 0.6
60.7 = 0,8
80.0
4.9
24.6 1: 0.9
+
93.7
58.6 1.1
64
S-BoN (s)
80.9 + 1.3
42.0 4..0
24.3 1 0.8
50.8 + 2.4
92.7 + 0.5
58.1 + 1.1
S-BoN (b)
83.4 1 1.0
41.9 1
33.0 + 1.4
57.6 + 1.6
95.9 0.7
62.4 1.1
GSI (ours)
84.1 0.4
41.5 1 0.2
30.8 1: 1.6
50.7 + 1.4
96.2 + 0.2
60.7 1 0.8
RSD
80.4 1.0
41.7 0.8
24.3 1 1.8
51.8
2.0
94.5 0.6
58.5 :: 1.2
256
S-BoN (s)
80.8 + 0.2
42,5 0.3
25.9 + 0.6
51.9
+ 1.9
92.7+
58.7 + 0.9
B-BoN (b)
84.1 0.3
41.2 1.7
33.6 0.9
57.3
+ 1.5
96.5 0.9
62.5 1.1
and sampling from softmax( the average acceptance ratio of GSI on MATH500 for different values of 3. A sharp phase transition between 3 = and 20 can be observed. In Figure 7 we plot the average accuracy for different values of 3 MATH500, in terms of both n and seconds per reasoning step. While 3 20 is not uniformly better than other values, achieves best accuracy overall. These figures demonstrate that 20 strikes balance in weighing the reward and the log ratio log(#z/m s/s), leading to acceptance ratios that are neither too low nor too high and good accuracies overall.
())))) to 1000. Figure depicts
C.4 ABLATION OVER 2
crucial hyperparameter in GSI is the acceptance threshold u, compare Algorithm To better understand the behaviour of GSI with respect to we plot the average acceptance ratios of GSI with Qwen2.5-Math on MATH500 for different values of Figure 9, and the average accuracy (over and over seconds per reasoning step) in Figure 10. As is to be expected, higher thresholds tend to have lower acceptance rates and higher accuracies, as they sample from the target model лв more frequently. Hence, it is important choose in such way that it strikes balance between accuracy and latency. In Figure we show an empirical Pareto frontier of u as a function of n. This suggests that the optimal depends n, and adaptive threshold schedule (u,), could improve GSI in terms of accuracy-vs-latene trade-off. For simplicity, we pick a constant value = 0.5 and leave exploring more intricate choices for future research.
C.5 DISCUSSION OF THEOREM 1
We provide short discussion of Theorem 1 and its practical implications. Note that while Assump- tion is necessary in order for all objects in the proof Theorem 1 to be well-defined, it does not directly impact the bound appearing Theorem 1. The important quantity here the chi-squared
22

---

Published as a conference paper at ICLR 2026
Table Qwen3: Accuracies on reasoning benchmarks with 95% confidence intervals. GSI outper- forms RSD (Liao et al., 2025) and S-BoN with the small model much more significantly than on Qwen-2.5-Math. As n grows, performance saturates.
n
Method
MATH500
OlympiadBench
Minerva
MMLU
GSM8K
Average
GSI (ours)
77.7 + 2.3
37.6 - 0.4
32.6 + 1.7
71.4 + 2.0
88.3 2.5
61.5 1.8
RSD
74.3 1.5
34.5 0.3
26.8 0.9
67.7 2.3
83.5
57.4 4 1.3
S-BoN (s)
66.9 0.8
31.0
22.0 + 0.2
58.6 4.3
79.3 0.5
51.5 1.3
S-BoN (b)
82.4 0.8
46.4 0.8
38.8 + 0.9
76.8
93.3 0.6
67.5 - 1.1
GSI (ours)
80.5 0.6
42.0 0.4
34.0 0.2
0.5
65.2 0,4
RSD
0.8
38.2 + 0.5
29.8 4 1.4
69.1 3.6
89.5 + 1.1
60.8 + 1.5
S-BoN (s)
73.9
35.9 +
25.4
1.3
S-BoN (b)
0.9
48.0 + 1.2
39.5 + 1.3
78.8 + 0.6
94.7 0.3
68.9 1 0.9
GSI (ours)
84.3 + 0.6
42.7 + 0.6
36.5 + 0.8
78.3 + 0.3
92.8 0.8
66.9 + 0.6
RSD
78.5 1 1.6
37.3 4 2.6
28.5 + 1.0
69.1 4.0
91.0 - 0.6
60.9 + 2.0
S-BoN (s)
77.1 0.5
37.5 0.7
26.3 + 1.3
63.3 3.9
88.1 1.2
58.5 +
S-BoN (b)
85.3 1 0.8
46.2 0.7
40.0 + 0.8
81.0 =
94.5 0.6
0..4
GSI (ours)
84.0 + 1.2
42.6 + 1.6
36.9 + 1.1
78,0 + 1.6
93.5 - 0.6
67.0 = 1.2
0.4
38.3 0.2
30.3 00
0.4
0.6
61.5 1.0
64
S-BoN (S)
77.7 +
38.5 0.4
26.2 4.2.:.0... 0.8
66.7 0.6
89.6 0.8
59.7 :.9
B-BoN (b)
85.1
46.5 0.7
39.3 0.3
81.4 0.6
95.3
69.6 0.5
GSI (ours)
84.8 1.8::00 0.2
42,9 + 0.7
36.0 + 0.0
78.1 + 2.2
94.3 1.0
67.2 0.8
RSD
79.9
38.7 0.2
28.5 + 1.0
66.8 + 1.2
91.6
61.1 + 1.1
256
S-BoN (s)
0.8
39.5 19.5::20 2.7
28.7 1 0.0
66.3 +: 1.8 1.8
89.9 0.2
60.6 1.1
S-BoN (b)
85.6 0.4
47.7 +
0.3
81.7
95.6 0.4
70.0 0.9
divergence (T |x) [. x)) between x. 12) and T [x), where æ corresponds to either the prompt, the prompt concatenated with all reasoning steps generated up to certain point. In Table 4, we show that the chi- chi-squared divergence generally well-behaved in practice, with mean values of between 1.48 and 3.91, depending the model family. These values Monte Carlo estimates over 50 samples from MATH500, where we average both over samples, as well as over reasoning steps in the generation. In each step I, generate N 64 subsequent reasoning steps yi...., V64 (- (-1 (z, (..,,..1))), and estimate the x² for that step as
N
TB(Ul (x, y v..,,(--1) T (z, y¹ ...., y'.,)"-11) -1
N
(xpp (log TD (x, , v"-1)) ms(u! ,.,,",,'-'1)) 1)2
from the logprobabilities computed under both models.
Table 4: Empirical estimates of (- |
=)|[=(( x)). Averaged over 50 samples from
MATH500 and reasoning steps. Monte Carlo estimates with samples in each step.
Model Family
mean x²
max x2
Qwen-2.5-Math (1.5B/ 7B)
1.48 + 2.20
109.20
Qwen-3 (1.7B/ / 14B
3.91 + 12.76
155.21
While we do not recommend using the bound in Theorem 1 as a practical guidance for choosing hyperparameters, the theorem is not necessarily tight. it can yield practical values in practice. If, for example, the x is equal to 2, and we set 3 = 1, the bound would guarantee that, if choosing (3e² 11/ 0..1 R 201, the KI. between т and fest is bounded by = 0.1.
23

---

Published as a conference paper at ICLR 2026
GSI: Qwen2.5-Math (1.5B / 7B)
RSD: Sowen2.5-Man (1.5B / 7B)
100
100
Acceptance ratio (%)
90
90
80
60
50
50
40
40
256
GSI: Qwen3 (1.7B) 14B)
RSD: Qwen3 (1.7B 14B)
100
Acceptance ratio (%)
90
80
80
70
70
60
50
40
40
4
16
64
256
4
16
64
256
MATH500
Minerva Math
MMLU
OlynpiadBench
GSM8K
Average
Figure 5: Acceptance ratios for GSI and RSD across datasets and models, with 95% confidence intervals. As n increases, the acceptance ratio of GSI approaches 90%. The acceptance ratio of RSD is much higher and converges to almost 100% n increases, which means RSD effectively collapses to soft best-of-n with nS.
8 acceptance ratio
B acceptance ratio
Acceptance ratio
0.8
Acoeptaace ratio
0.8
0.6
0.6
5
8
20
20
0
100
100
0.2
1000
0.2
1000
0.0
0.0
16
61
256
61
256
Figure 6: Acceptance ratio of GSI for different values of on MATH500. Left: 5wen2,5-Math;
right: Qwen3. A sharp phase transition between B = 8 and | = 20 can be observed.
24

---

Published as a conference paper at ICLR 2026
G accuracy VS
B accuracy vs scc/step
82
Aceuray (%)
Accuracy (%)
80
80
78
8
0 6
76
8
0 m 8
8 180
0
74
8 100
74
0 100
1000
1000
16
61
256
Tt
sec/steg
Figure 7: Qwen2.5-Math: Accuracy of GSI over (left) and over seconds per step (right) for dif-
ferent values of 8, on MATH500. In the right plot, each curve corresponds to n = 1. 4, 16, 64, 256 for fixed value of B (where each dot on the curve corresponds to value n). Our value 3 = 20 performs best overall, but as n varies, different can have an edge. Runtimes reported on H200 GPUs.
3 sccuracy VS
B accuracy vs sec step
Avcuracy (%)
Accuracy (%)
75
0 8
70
70
100
1000
1000
16
61 256
10-1
sec/step
Figure 8: Qwen3: Accuracy of GSI over n (left) and seconds per step (right) for different
values of B. on MATH500. In the right plot, each curve corresponds to = 1, 4, 16, 64, 256 for fixed value of 3 (where each dot on the curve corresponds one value n).. Our value 8 20 performs best overall, but n varies, different can have edge. Runtimes reported on H200 GPUs
C.6 RUNTIME COMPARISON
In Tables 5 and 6, we provide extended versions of Table with runtime values across n on H100 GPUs for Qwen2.5-Math, and A100 GPUs for Qwen3.?
C.7 REASONING TRACES
We provide several examples from MATH500 and MMLU-STEM and the reasoning traces generated by GSI and RSD with our Qwen2.5-Main models, in addition to the two examples in the main text. The following boxes contain samples, alongside the reasoning steps selected by the two algorithms (including rejected steps, which are marked by being crossed out) for n = 4. For GSI, the last column contains the tilted reward (for samples from TT) resp. the normal reward (for samples from TB). For RSD it always contains the normal reward. We picked samples where reasoning traces were not too long in order to fit them one page; note that on average, reasoning traces are much longer (emp. Table 5).
For computational reasons, for the Qwen3 experiements ran 256 on H100 and H200 GPUs, hence do not report them in the table for consistency.
25

---

Published as a conference paper at ICLR 2026
Threshold u acceptance ratio vs m
1.0
Acceptance ratio
0.8
0.6
0.4
0.2
16
64
256
12 - 0.05
0.25
0.5
- 12 0.15
0.3
0.75
1.5
Figure 9: Larger thresholds lead to lower acceptance rates in GSL. We show acceptance ratios of GSI for different acceptance thresholds u on MATH500 for the Qwen2.5-Math models.
Threshold u accuracy vs
Threshold u accuracy vs sec/step
Accuracy (%)
82
Accurac (%)
0.05
== 0.05
80
80
== 0.15
0.25
== 0:25
78
0.3
78
= 0.3
0.5
x 0.5
0.75
0.75
1.5
1.5
74
16
64 256
10a
72
8ec stc/step
Figure 10: Left: Larger acceptance thresholds u lead to higher accuracy in GSL This is to be expected, as larger thresholds mean higher probability of sampling with TB. Right: When plotting accuracy function of seconds per step, single threshold performs best. Each line in this plot corresponds to the values n 1, 4, 16, 64, 256, for a fixed threshold u. All plots averaged over MATH500 using the Qwen2.5-Man -athhhh models.
MATH500, Example 3. For this difficult question, GSI repeatedly resamples from the base model to find the right answer. RSD accepts all draft samples and arrives at a wrong answer.
MATH500, Example 4. In the fourth example, we see that GSI can sometimes also reject correct steps generated by the small model, if the tilted reward is too small. GSI still arrives at the correct answer the end. In this example, RSD does not produce any final answer.
MATH500, Example 5. This example highlights an interesting phenomenon: without any interven- tion, GSI and RSD generate almost the exact same reasoning trace. crucial step, कş incorrectly rounds 233/43 43 to 5.5, which GSI corrects by resampling from тв and correctly rounding to four decimals, 5.4186, while RSD accepts the sample from T and arrives at a wrong answer. This exam- ple also highlights why including the log ratio in the reward can be crucial: The incorrect step under Ts receives an (almost) perfect reward of = 0.999 in RSD. The almost identical step GSI has an (almost) perfect reward of r 0.998 (not depicted in the box), while its tilted reward is only 0.148.
MATH500, Example 6. We show that it can happen that GSI does not solve a problem that RSD manages solve. However, this only occurred three times in the entire dataset of 500 samples.
26

---

Published as a conference paper at ICLR 2026
Threshold u accuracy vs sec stte
86
.00
1.50
1= 0.75
84
1030000
0.50
Accuracy (%
82
n=0.50
n-0.30
80
n
78
n
π 16
n 64
76
-
0.30
n 256
Best per
100
10
sec/step
Figure 11: Empirical Pareto frontier of optimal thresholds u for different values of n. The optimal u is a concave function of For each value of average accuracy as a function of seconds per reasoning step for u 0.3, 0.5, 0.75, 1.0, 1.5 with 95% confidence intervals three random seeds. For the Pareto frontier, select value for each n. Averaged over MATH500 using the Qwen2.5-Man models.
4, 16, 64, 256, show the
MMLU-STEM, Example 1. The draft model seems to be generally quite weak on MMLU-STEM and often produces nonsense, including random artifacts such as Chinese and Korean characters. This example shows that GSI can help in mitigating the weaknesses of the small model to some degree. However, several nonsensical steps from the draft model still slip through. Nonetheless, GSI manages to find the correct response, whereas RSD does not.
MMLU-STEM, Example 2. As in the previous example, the draft model struggles to produce co- herent responses. GSI catches some of its errors, but both GSI and RSD answer this question wrong.
D ASSETS
D.1 HARDWARE
Most of the experiments were run on NVIDIA H100 GPUs. Each model was hosted on its own GPU and implemented with vLLM (Kwon et al., 2023). Some experiments were also run on NVIDIA 100 GPUs and NVIDIA H200 GPUs with the same setup.
D.2 LIBRARIES
We heavily relied the following open-source python libraries: PyTorch (Paszke al., 2019) (license: BSD), transformers by HuggingFace (Wolf et al., 2020) (license: Apache-2.00 2.0), and vLLM (Kwon al., 2023) (license: Apache-2.0).
27

---

Published as a conference paper at ICLR 2026
Table 5: Qwen2.5 on H100: Inference time (in seconds) per reasoning step, number of reasoning steps per sample, and percentage of steps accepted (averaged across all datasets, with 95% confi- dence intervals over three random seeds), for n. 1, 4, 16, 64, 256 (extension of Table 1).
n
Method
$ s/step ())
# steps
% accept
steps /s (f)
GSI (ours)
0.33 0.02
8.1
65.4 0.1
3.03 + 0.17
RSD
0.24 = 0.01
8.8
90.1 + 0.0
4.17 = 0.17
S-BoN (small)
0.20 = 0.00
8.1
5.00 = 0.00
S-BoN (base)
0.39 0.03
9.3 9.3 0.2
2.56 = 0.18
GSI (ours)
0.43 0.03
10.6
76.7 0.1
2.33 = 0.15
RSD
0.04
9.1
94.9 0.0
2.94 0.08
S-BoN (small)
0.32 0.01
9.0
3.12 + 0.09
S-BoN (base)
0.57 + 0.01
1.3
1.75 + 0.03
GSI (ours)
0.72 = 0.05
0.1 1.2
1.39 = 0.09
RSD
0.61 + 0.01
0.3
0.0
1.64 0.03
16
S-BoN (small)
0.52 + 0.03
0.1
1.92 + 0.10
S-BoN (base)
0.94 + 0.03
10.5
1.06 + 0.03
GSI (ours)
1.78 0.12
12.0 + 0.4
84.3 3.1
0.56 + 0.04
RSD
1.60 0.03
10.9 0.3
98.2 0.1
0.62 + 0.01
64
S-BoN (small)
1.50
10.7
0.67 + 0.01
S-BoN (base)
1.99 +
10.8 + 0.2
0.50 + 0.02
GSI (ours)
5.80 - 0.23
13.0 0.3
0.17 0.01 0.01
RSD
5.52 - 0.42
11.3 1.0
99.1 0.0
256
S-BoN (small)
5.46 + 0.10
11.3
0.18 0.00
S-BoN (base)
5.88 - 0.11
11.1 0.3
0.17
D.3
CODE REPOSITORY
We used the RewardHub library by Red Hat AI Innovation Team, and grading functions from Ope-
nAl's prm800k repository to extract and grade answers from LLM-ggnnrated responses.
E USE OF LARGE LANGUAGE MODELS
We utilized generative AI tools for code generation and debugging. The authors carried out all of the substantive research contributions, experiments, and proofs.
28

---

Published as a conference paper at ICLR 2026
Inference time (in seconds) per reasoning step, number of reasoning steps per sample, and percentage of steps accepted (averaged across all datasets, with 95% confidence intervals over three random seeds), for 1, 4, 16, 64 (extension of Table 1).
Table Qwen3 A100:
Method
s/step ({)
steps
% accept
steps /s (1)
GSI (ours)
0.35 + 0.02
24.1 0.0
80.9 0.1
2.85 + 0.15
RSD
0.24 # 0.01
25.2 0.1
95.3 = 0.1
4.17 = 0.17
S-BoN (s)
0
23.3
5.00 + 0.04
S-BoN (b)
0.59 + 0.01
23.3
1.69 + 0.03
GSI (ours)
0.56 + 0.04
26.7 0.3
88.0 0.1
1.79 0.12
RSD
0.01
28.1
97.2 0.1 _
2.50 0.06
S-BoN (8)
0.38 + 0.01
2..8
2.63 0.07
S-BoN (b)
0.83 0.02
0.1
1.20 0.03
GSI (ours)
1.21 0.05
9.1 2.4
0.83 + 0.03
RSD
0.96 + 0.01
0.2
9..2
1.04 + 0.01
16
S-BoN (s)
1.01 0.00
26.1 0.2
0.99 + 0.00
S-BoN (b)
1.82 + 0.06
2.38
0.55 + 0.02
GSI (ours)
3.26 + 0.11
28.8
93.1 + 0.0
0.31 0.01
RSD
2.7 2.93 0.05
98.7 0.2
0.34 = 0.01
64
S-BoN (s)
3.01 +
26.4
0.33 0.01
S-BoN (b)
4.75 + 0.06
2.3
0.21 0.00
29

---

Published as a conference paper at ICLR 2026
GSI: MATH500, Example (Qwen2.5-Math)
Prompt:
The wheel shown number on the wheel being equally likely). The two numbers determined in this way are recorded. The lirst number divided determining
twice. that the numbers indicated by the pointer are randomly determined (with cach
of the remainders 1,2.3 marking colurns of the checkerhuuud shown. The second number divided by 5, determining one of the remainders 1,2,3,4 marking the rows of the checkerboard. Finally, a checker placed on square where this column and lasyl unitsizel lem]:
What the probability that the chocker is placed shaded square the eheckerbourd? WWWCCCrrlo(UU))2...neeedl
(1aw(((.7.))-1111. (11.))-(111-111..--. 1i-1).ineeidthh0.7
draw (1.7, 177-11+(- label(""""
1...).inewiithhO.
(-0.(""9.-0000...5. draw((0,2)-(0-22) label("I" (...0..........) label("2 (0..-
(.0.8.0.5),,B))
label("3",(-0...00,
Iabel(""".007.0000
draw((2.8.0)--22 (.1.22808)0--.10000000. A.r.o..
"oiitt""
288,0.W:: (Il((3,0)) (3,1) (4.0) (4.))-ycclle..aaa0000. grayl...
(), (1,,11)),44111))
(4,1))-4422) (4,-1))))
(I)((4)))-4.2)
(5,2) (501))yyyyylll.graa00..
All((4111) (4.00)(55.0)-5555
(5,0))(5.-1)))) cycll.ggyy000..)).
6II(5.0) (5.1)))))511--))-))))) (6.1))
n (---))066---------lllllee-------
draw((3-11)-0
1)linwwidbb0077) (00.13000-6.....iewwi000000.
draw(331)
draw((S,23 (5.2.)).iieeww
labe(("",33.522..
label(" 1bbe5("",,445..2...
label (3.-0.5). W); label("""" (3.0.). label (3".....)) [/asy]
Step
Generated Reasoning Steps
Draft
Target K
F(v 1/m(x
To determine the probability that the checker placed on shadled square of cheekenbuurd, need to amilyre outeomms the two spins and the resalting positions on the checkerhoad.
0.940
2
idemiity possible ouicomes span. finst spin can in-hee mumbers sccond spin can resalt in the mimbers 4,4,3
0.047
Fisst, let's identily the possible remainders when the numbers en the wheel are divided by and S:
0.990
For division by 4, the possible remainders are 1, and 4, For divisiun by the possible remainders are 2, and 5.
0.422
Hlowever the prublem unly asks about the remainders 1, and the fins spin odisided hy and and the second sprin olivided by 5). This memns ane only considering the firs Three remainders for the lirst spin and all four remainders the second sprin.
-0.706
chockerbeund
squrees, arringed and
many
these squares are
0.980
From observe that the shaded squres
(lrb. 12.21 (.3..
-0.128
From checkerbour. observe columns are shuded in fint and chird rows, and the squares in the second and fourth columns are shaded in second and fourth rows, This gives the fallowing shaded squares:
squares pastern squares
0.803
Shaded Unshaded Shaded
Unshaded Shaded Unshnded
Shaded Unshaded Shaded Unshaded
Unshaded Shaddd Unshaded Shaded
Shaded
Unshaded Shaded Unshaded
0.543
Unshaded
Shaded
7
Couming the shaded sguares, we find there are shuded spures out of the 20 aguanes on the chockerbeond.
1.132
equally likely, each remainder when by is equaity likely Lieretore,
each the possible outcomes-is is equally likety.
0.402
The probubility that the checker is placed shaded square the ratio the number of shaded squares to the total
number of
Number shaded squames
Probahility
Total number of squares
20
1.000
probability that placed square
Correct Answer found:
True
30

---

Published as a conference paper at ICLR 2026
RSD: MATH500, Example (Qwen2.5-Math)
Prompt:
The wheel shown number on the wheel being equally likely). The two numbers determined in this way are recorded. The lirst number divided determining
twice, that the numbers indicated by the pointer are randomly determined (with cach
of the remainders 1,2.3 marking colurns of the checkerhuurd shown. The second number divided by 5, determining one of the remainders 1,2,3,4 marking the rows of the checkerboard. Finally, a checker placed on square where this column and Jasyl unitsizel lem]:
What the probability that the chocker is placed shaded square the ehockerbourd? WWw.Crrco((UU))2...neeelll
draw (1.7.-1 y-( (0.8-..)) label("""" (.).8.0.5),,B)) draw((0,2)-(0-22)) label("I" (0... label("2
(122((1..1))-11122 1......ineeddthh....
1...).inewiithhO.
(0..(("9.--0..0003.5 label("3".(-0..00,,
"abei(""".0.7.01..
draw((2.8800)- A...o.) label("Pointer", (-
288,0.W:: (II((3,0)) (4.1) (4.0)-ycclle..gaaa0000 grayl.gr
(3, (3,1))(44411)))) (4,-1)))) (4)
(l)((4.))-4,2)
(5,2) (5y...ylcyyll...aa00..., All(((111)
6II(5.0) (5.1)-(6.1)
(6.0)-oycll.gray0077))
dmmw((3-1)6666
1)linewidb(0077) (0w.(/3.)-(6.01..ieewwwdl
draw((S,23 (52)).)..inew labe(("",33.522..)
label("2" 1bbe5("",,445..21..
label (3.00.5). W); label("""" (3.0.). label( (3.15...).. [/asy]
Step
Generated Reasoning Steps
Draft
Target K
r(v 3/m(x
To detemmnne the probability that the checken placed on shuded squire of checkerbuurd, need to amilyre srrectre of the chockerheadd and the possible outcomes of the spin.
0.996
First, let's identify the shaded squares on the checkerboad. The checkerhoard is 3x4 grid, with the following shaded
2
squares: (1.1) (1.3)- (2.2)-(2.)
0.784
This means there are shaded squares out of total of aquares.
Next, we need determine the probabality of landing on shaded square. The wheel is spun twice, so there are possible for (1. 3, 4,5, 6). When first number divided by pussible remainders Similarly,
divided possible
Therefore,
12 possible outcomes in total.
squre. From checksbourd, that: The fins spin landing on (yemainder 1) and the second spin landing on (remainder 1) results in the shaded (1.1), The fins spin landing (remainder and the srcend spin landing (remiinder resulis first spin lunding fremainder the shaded square (2.2). The first spin landing on 2 fremainder 2) and the secund spin landing on tremaindrr 4) results in the shaded square (2.4).
Now, count mumber outcomees
squar
spin
So, there are outcomes that result a shuded square. The probabiiity the number of Enveaale outcomes divided by the total namber of outcomes, which is x
Therefore, the prohubility that the checker placed on staded
1.000
Correct Answer foundd
False
31

---

Published as a conference paper at ICLR 2026
GSI: MATH500, Example (Qwen2.5-Math)
Prompt: Let be complex oumber such that
1 vB
2
85
Step
Generated Reasoning Steps
Draft ms
Target x
F(u ((n())/((ff))
oquation + Therefore, we can write
1/ rocognize complex number meaning |=|
0.330
for some real number 0.
express
60
2 2008..
1
2
0.412
cos
Solving for we pal
1 v5
cos
The equation rewritien
10 e- 2 cos
know
0.922
v5
angle
equation
can / 8n/5.
find 96
.im Let's consider separately.
II
18/5 them:
(c/r/5) 88
1857/5
((16747) - 4 -1. 117
0.122
So.
233
2. If
/0 then:
(6-1n/0)85 85 = 1858/8 - -17e (((60++) -1-
0.998
.85
-2
In both the resuli. Therefome, the value of
285
-2
0.998
Correct Answer found. True
32

---

Published as a conference paper at ICLR 2026
RSD: MATH500, Example (Qwen2.5-Math)
Prompt: Let be complex oumber such that
1 vB
2
85
Step
Generated Reasoning Steps
Draft ms
Targel 27.1
r(u 3/m(x
_855
2an given equation
v
by socogaizing that
golden
1+v8
con O.
Given that 2 +
1 we have:
1+ v5
2
Thos,
vB
cos
Next, we need to find 80
Using the exponestial forn, we have:
0.993
Therefore,
6 -1500 roe(()))).
To find cos(850)), we use the angle adkitiion formula for cosine:
cas cos(840 0) cos(810) cos(O) sin(849) sin(
We need to detemmin cow (846) and sin (846). Notice that:
(oe(2 2 coc (H) 1.
Substirating
--V get:
coe2( (0)
1
(1 + vo12
1 2/5
2V3
3
vB
0.959
16
16
16
Thus,
2
255
2/5
/B
cne(26) = 2
steps, recognize pattem in the powers of cos and sim that simplidies our work. We use the fact that cos (20) related to
(4o(40), (80)
becomes increusingly comples,
0.821
cos () and sini (), and we can use the properties of the golden rutio.
found
False
33

---

Published as a conference paper at ICLR 2026
GSI: MATH500, Example (Qwen2.5-Math)
Prompt: Juan's Old Stamping Grounds
Juan organizes the stamps his collection by country and by the decade which they were issued. The prices paid for them Brazil and France. cents each, Peru 4 cents each, and Spain cents each. (Bruzil and Peru are South American countries and France and Spain are in Earope.) [asy]/ AMC8 2002 #8, 9, 10 Problem */ size(3inch, 1.5inch;; for intt
stamp
0. [= 5+++))
draw(0.y)((18.)))
(0.)((0.)))(05))) draw((6.00-(66.))) (6.55) draw((9.0) (9.5); draw)((1,00)
(1,00(15,0)))))) (,0)-(15,55):
draw(((8,0)-11888)
labeltscallt0.88 (7.5,4.5):: labelfscalo(088)""""", (7.5.3.5)) labellicclko(088"""""""" (7.5,2.5)): label(scule(0.8)
(7.5,1.5)): label(scale(0.8S)"" (7.5.0.5)):
labeliscale(088" (10.5,4.5)): labellscnle(.88)"77",, (10.5.3:5): lahel(scale(0,8)"""7 (10.5,2.5)); (ahelsstale(088)""""
(1555,1.5) Iabel(scalle(O 8y"" (10.5.0.5));
labeliscale(0.8) (13.5.4.5)): label[scalo(0.8)""11,, (13.5.3.5) labelisealee0.8)"112"" (13.5,2.5)). 1abellscalc(0.8)""6""""
(1355,1.5)); label(scale(0.8)) 13"",,""" (13.5,0.5));
Iabeliscalee0.8) 80""0 80s". (16.5,4.5) labclfscalee(.88"" (16.5,3.5) Labelfscals((.8)"111""" (16.5.2.5)): labeliscale(O... (aaasalloOO.8"11110 "00".
(16.5,1.5) labelfscaler0.8) (16.5.0.5)):
(3.4.5)): Labrl(scaleeO.s)"
(3,3.5);
label(scale(0.8) "Tumme"" (3.2.5)): la-
bel(scale(1.8) "Penn" (3.1_ (3)1.5)); lahel(scale(0.8) "Sans (3.0.5));
label(scale(0.9) 9)""Jan'ss Stamp Collection". (9.0), S); label(scale(0.9) 9) "Nmbber of Stamps by Docade", (9,5), Ny; [asy] What
average price, cents,
stamps? your the teuth
Step
Generated Reasoning Steps
Draft as
Target J
F(u 3/rls()))
the avenaee price Juan's stamps, need stee.
0.899
**Identify stamps country
stamps
stamps
2
stumps Spain: 13 stamps
0,802
**Calculate the stampe the 70'
12 13
0.636
*DDttammin total of stamps from each country: stampp stamp
FFrnnee: 12 stamps cents per slamp oents Peru: stamps 13 stamps 5 cants stamp 65 cants
cents per stamp 24 centss Spain:
0.399
**Calculare the rotal pne of all the 70's stampe:
2333
**Find the average price per stamp:**
Total
233 cents
Average
Total number of stamps
43 stamps
**Perform the division and round to the nearest tench of cent:*
6:185
43
Roundang 5-4th the ncnrest tenth gives us
6. **Perromm the division:**
5.4186
0.996
43
**Round to nearest tenth
G.4186 Rs 5.4
1.027
Therefore, the average price of Juan's 70's stamps is
5.4 cents.
1011
Correct Answer found: True
34

---

Published as a conference paper at ICLR 2026
RSD: MATH500, Example (Qwen2.5-Math)
Prompt: Juan's Old Stamping Grounds
Juan organizes the stamps his collection by country and by the decade which they were issued. The prices paid for them Brazil and France. cents each, Peru 4 cents each, and Spain cents each. (Bruzil and Peru are South American countries and France and Spain are in Earope.) [asy]/ AMC8 2002 #8, 9, 10 Problem */ size(3inch, 1.5inch;; for ( intt
stamp
0. [= ++++))
draw(0.y)((18.)))
(0.)((0.)))(055)) draw((6.00-(66.))) (6.55) draw((9.0)-00 (9.5); draw)((1,00)
(1,00(15,0))))) (,0)-(15,5)::
draw(((8,0))(18..)
labeltscalot0.88 (7.5,4.5):: labelfscalo(088)""""",, (7.5.3.5)) labellicclko(088"""""""" (7.5,2.5)): label(scule(0.8)
(7.5,1.5)): label(scale(0.8))"" (7.5.0.5)):
labeliscale(088" (10.5,4.5)): labellscnle(.88)""7",, (10.5.3:5): lahel(scale(0,8)"""" (10.5,2.5)); (ahelsstale(088)*"""
(1555,1.5) Iabel(scalle(O 8y"" (10.5.0.5));
labeliscale(0.8) (13.5.4.5)): labelfscalc(0.8)""11,, (13.5.3.5) labelisealec0.8)"112"" (13.5,2.5)). 1abellscalc(0.8)""6""""
(1355,1.5)); label(scale(0.8)) 13""""1l000"""13"",,"_" (13.5,0.5));
Iabeliscalee0.8) 80s"",". (16.5,4.5) labclfscalee(.88"" (16.5,3.5) Labelfscals(0.8)"111""" (16.5.2.5)): labeliscale(O... (a "00"
(16.5,1.5) jabelfscaler0.8)=>4 (16.5.0.5)):
(3.4.5)): Labrl(scaleeO,sy"
(3,3.5);
label(scale(O.8)" "Tummm"" (3.2.5)): la-
bel(scale(1.8) "Pnn"" (3.1m (3)1.5)); lahel(scale(0.8) "Sain" (3.0.5));
label(scale(0.99 9)""Jan'ss Stamp Collection". (9.0), S); label(scale(0.9) 9)) "Nmbber of Stamps by Docade", (9,5), Ny; [asy] What
average price, cents,
stamps? your the tenth
Step Generated Reasoning Steps
Draft as
Target J
r 1/m(sst)
avenge price of 70s stamps, we
stege:
0.986
2
stamps Spain: stamps
**Idenily stamps councy
stamps
stamps
0.993
**Calculate the 70k stamps:**
12 13
0.999
**Dttemmnne total stamps country:
stamps conts/atamp
France: stamps x cents/stump 72 cents Peru: stamps centshitmp 24 cents Spain: 13 stamps X centalttarmp 65 cants.
0.999
**Calculare the total pnet of all the 70s stampe**
2333
**Find the average price per stamp:**
Total
233 cents
Average
Total number of stamps
43 stamps
**Perfonas the division to find the trice:**
Ro 5.68872
0.999
**Round
6.4888772 Se 5.5 cents
0.999
Thux, the average prive Jun's 70s stamps
5.5 centx.
1,000
Correct Answer found.
False
35

---

Published as a conference paper at ICLR 2026
GSI: MATH500, Example (Qwen2.5-Math)
Prompt: Evaluate
sin(arcsin 0.4 + arcsin 0.5) sincaresin 0.5 aresin 0.4).
Generated Reasoning Steps
Draft
Target 8
F(y 1/m(s
To evaluste (arcsin 0 arcsin 0.5) (arcein aresin 0.4), letiing aresin (B)-iiin ox).
Firsi, the angle adlition fommulas
2
sin(a + 5) sin ear " cos a sin
0231
win(8 sin cos coe ein
Using
formulas,
sin(a B} = sin coe 9 + cos sin
sin( a)
Multiplying these two expressions together, we get:
9) sin(3 (sin cos + (sin
sin P)
This is difference of squares, simplifles to:
sin(a + ()) sin( aa) (sin cos 002 (cos sin 22)²
We know that cos" @ = sin" for angle O. Therefore, we can substiuue cos" a sin" and
0.982
cos?
equation:
sin(a B) sin(d Substicuting sin
(sin 21 sin" 2) (sin 8)² sin" a))
0.4 and sin U.5 into the equation, we
sin(a + (3 sin ca) (u.4)" (u.5)") (a.5) (1 (0.4)²)
Simpllifying
parentieeses
sin(ox (8) s/n ox (0.16)(1 0.25) (0.25)(1 0.16) (0.16)(0.75) (0.25)(0.84)
Calculating products, get:
sin(er ()) (1 - .x) 0.11 0.21 0.08
Therefore,
0.09
Correct Answer Bound. False
36

---

Published as a conference paper at ICLR 2026
RSD: MATH500, Example (Qwen2.5-Math)
Prompt: Evaluate
sin(arcsin 0.4 + arcsin 0.5) sin(aresin 0.5 aresin 0.4).
Generated Reasoning Steps
Draft
Target 8
r(u 1/m()
To evaluate expression sinu (aresin aresin 0.5) sin(arcsin 0.5 arcsin 0.4). Therefore, we have sin 0.4 and sin U.S.
peoperties
by areein
0.997
and
areain
sin
2
0.47
v0.8d
(21
v21
0.996
Similarly. since
aresin D.5, we have ons &
v1 win
0.52
v
0.25
v0.73
0.999
Now, we use angle aklition and subtmctioon formulas sime:
sin(a B)
1000
sin(B a) = sin cos a cos @ sis
Substicuting the values we found:
v
VII
0.4v3
0.5.
2/3
V2
2v +
V2
2
10
10
V3
0.5v2I
0.4v3
V2I
2v3
VTI
2v3
sin(S - 0)) 0.5
0.4
Now. the product two
sin(rx + 9) sin(f o) = 223 v2I
VII
2
1.000
This product of the furm (a ba(a b) a² - 62, where
VIT
and b 2v/3.
10
10
VII
10
10
10
100
100
100
Thus, the value of expression
Correct Answer Bound. Trme
37

---

Published as a conference paper at ICLR 2026
GSI: MMLU-STEM, Example (Qwen2.5-Math)
Prompt:
The seventh cranial nerve supplics
Choices:
Answer I:
taste buds the posterior third the longue.
Answer 2: muscles the soft palate.
Answer 3:
museles of the lower lip.
Answer 4: the parotid salivary gland.
Slep
Generated Reasoning Sleps
Draft
Target
F(U 1/r(())))))
To determine the correct anewer. nood whicheddetwwhhhhhhhfhhht options desemibes the seventa Ei.
-0.571
To desermine which seventh cranialner supplies, understand the functions the cranial conesponding structares. analyze opeion step step:
#T Logrunge's sear": This not standard term the contest of cranial scmestov, and-irdoeunn aireedly relane
-1.422
tongue** posterior third of the tongue. Therofore, the soventh comialner Ron does not supply taste buds on the posterior thind of the longue
Posterice
supplics
0.970
*MMuscles of the Seft Problem"* The seventh cranialser Bon supplies themm centroid of the soft problem. The problem crnnilnner supplies, centroid problem.
centroid lip":: seventh cranialner Ben supplies centroud lower lip. Ben supplies.
description structuee
*Parotid Salivary oone**. The seventh cmmislner Ren supplies the parotid saiivary come The-goneehhaaiivrr seventh craniclser supplies, process.
for the structure
"Thhe Parotid Salivary Gland": The sixth cramialner Ben supplies the parotid salivary gland. Therefore, the seventh conniinnr Ren does the parotid valivary gland.
0.993
Based un the analysis, the connect answer is:
1002
0.99
Correct Answer found: True
38

---

Published as a conference paper at ICLR 2026
RSD: MMLU-STEM, Example (Qwen2.5-Math)
Prompt:
The seventh cranial nerve supplics
Choices:
Answer I: taste bads the posterior third of the longue.
Answer 2: muscles the soft palate.
Answer 3:
museles of the lower lip.
Answer 4: the parotid salivary gland.
Generated Reasoning Sleps Draft
Target
r 1/r(()))))))
To daternine the conrect answer, let's analyze each optioo step.
0.999
2 **Tase gnans on theenoe the**; This not correct desetiption of aoveehh cranial vurialle. The Ieseel, which is the seventh veristle, is trauually lotated en the lateral thind of the, not theerior third.
0.644
"*Tase Buds Postetior Thind the lungue:*e.... The seventh cramial does indeed supply tasle buds longue.
anstomy.
3 *MMssless of the Softsampie:* The seventh cranial use supplies themu of the sampl, not the sofi divdends.
0.577
museles of the Soft Palote: The seventh emanial nerve does supply the museles the soft palate. This
function performed by
**Muscles the Ch Betty:* The seventh cranial does supply thema of lower This performed by eighth
0.987
supply salisury gland
performed by the temth cranial note.
0.21
anolysis, cvenect anewer
0.73
1000
False
39

---

Published as a conference paper at ICLR 2026
GSI: MMLU-STEM, Example (Qwen2.5-Math)
The members bird species capable of nesting troctops the ledges of ncarby clifs make their homes only in the treetops due to the established presence of a more aggressive bird species on the clift ledges. With respect to habitat, the treelop- nestinn hirds utilizing Choices:
Prompt:
I: fundamental niche only
Answer 2: their realistic niche only
Answer 3: ccological niche only
Answer 4: aeither their fundamental niche their realistic niche
Step
Generated Reasoning Steps
Draft i
Target 77.3
To determine the conrect answer, let's analyze the situttion step by step:
0.368
*Unddasttndinng Situtionn*, problems lndges neachyfiihes species on beflihhe..
species mestiug-in tmesmps only Sapeddeeton suablissed presenee mare aggregate
2
-3.779
1. *UUndessaadinn the Fundamental Niche The fundamental niche of species refers the set of emironmen- conditions under species exist isolation. includes abiotic (won-lving) temperatur, minfall, soil type, light.
2. Unndmttnding the Realistic Niche*s, The realit' of species refers to the set of em-ronmental conditions under which the spacies can survive and replace liseff includes all the ahintic and hivii (living) factors such as food aniilablliy, competition, prodation.
2/67
*UUddestandia og thelCooogical Niche*>; ecologicalo conditions species with other species and the commanity environment.
and pootit particular community.
inseractions
0.18
*nnalyzing the Problems* The prublem states that the U88 birds are using only isevoce which meons they are using the isetops as their fundamenian ii. This is because they and the tne- places they can lind food and water, and other sesops available other specier occupy. Therefore, fundamentalaa St suisfied setops.
66.19
4. ** Amalyzing the Given situation*:; The members of the bird species capable of nested in wre tops or on the ledges of nearby chifls make their homes only the plunged due to the established presence of more aggressivve bird species on the cliff ledges. This mouns that the tres top-necing hinds have chowen specific the community being outcompeted eaten by more aggressive
0.985
they can
lodges.
"DDiermmning the Niche**, Since the tres iop-nestling bird have choses specific Chocommunity Where they sested chosen ecologicalawode. They are using Tundumemmal realsticnights, which takesiine aeceent the interactions with other spacies and the community cnvinoment.
outzomp Et & by the
clil" lodges, they coologizzlnwide
From this information, we can see that the tress top nesting birds are unlizinng their ecological niche only because they without affected
specifie part community ledges.
aggressiive
Therefore, conrect unswer
40

---

Published as a conference paper at ICLR 2026
RSD: MMLU-STEM, Example 2 (Qwen2.5-Math)
The members of bird species capable of nesting in troctops the ledges of ncarby clifs make their homes only in the treetops due to the established presence of a more aggressive bird species on the clift ledges. With respect to habitat, the treelop- nestinn hirds utilizing Choices:
Prompt:
I: fundamental niche only
Answer 2: their realistic niche only
Answer 3: ccological niche only
Answer 4: aeither their fundamental niche their realistic niche
Step
Generated Reasoning Steps
Draft m
Target 27.3
r(u 3/rls)
To determine the uppcupriate answer erswer, let's anilyze the sinuation step by step.
0.999
**lcemify the environment and the binds involved=*e. The birds are cupable of nesting in peanuts on the ledges of Wearby plates The peinuts are home to more Aggressive bird species on the use plates.
2
0.904
**Undentand the forme-. as Fandamentall:**. The ff is the most busic esseniall function practical
function purpuse something **ECollgicaln+** The coologicaln the relationhip between all living things (bints) their cummon ervironment.
purpose something. **Realitien:**
**nalyyee the siustion a-. The binds using their establish presence
their home for their nunping. schicl libilt based impples
Apgressive hind spocies soppiigg natural medium the binds to comcetiratted-eeii sapping and ure likely "realistic" 'n place where they can depents on the presenge of che--mon "ggressive bird species.
*mmlyy the birdk' habioa=* The tree-nescing binds chuuse nested only the This specilic
ledgos.
options**
fundamentaltracker only
hirds' tisenes only:ee
cavironment. Amswer This would imply that the bands tress-nees nested are the practical real look function their environment. answer 3: their coological mess only ** This would imply that the binds" trass-nrsted are the ecological funerion of their esvironment. most
ness-nested most essential
0.922
neither fundamentala their realistic fuaction enviranment.
would that
or practical
""Detemmine most appcopriate uuswer:** bocause they chouse to nessed only in the uree mest due to the established presence of more Agable bird spectes on the use plates.
tress nest-aesed coologicals
0.874
7
Therefore, the correct choice 3
0.99%
Currect Answer found: False
41


Reviews:
[{"review_id": "NTM17lo5Vk", "reviewer": "Reviewer_8re2", "summary": "- The paper presents an interesting approach for efficient reasoning by combining speculative decoding with reward guidance.\n- Demonstrates improved accuracy over state-of-the-art baselines, though at the cost of higher compute.\n- The paper is well-written, and the proofs are well-justified.", "questions": "1. The draft model appears to perform strongly based on Table 1, particularly as $n$ increases. Could the authors provide an analysis of the proposed method’s effectiveness when using weaker draft models?\n2. For draft models with higher $n$, performance seems competitive. Could the authors report the accuracy for the s-BON draft configuration with $n=16$?\n3. How is the activation point for the PRM model determined? or how many tokens are generated before calling the PRM. What heuristic is this based on? For reference in speculative decoding, the optimal draft length is downstream task dependent. \n4. The new reward formulation incorporates the logarithm of the acceptance criterion. Could the authors elaborate on any theoretical connections between this formulation and standard speculative decoding?\n5. The experiments utilize Qwen2.5, while newer and stronger models such as Qwen3 are available. Could the authors clarify why Qwen3 was not considered?\n6. The PRM model is reported to be the same size as the base model. Were any ablation studies conducted with smaller PRM models to assess trade-offs between performance and computational cost?\n7. In Theorem 2, expression (b), is there a missing term involving $\\tilde{\\pi}_{\\text{GSI}}(Y\\geq)$​ ?\n8. In Figure 6, each curve appears to correspond to a different value of β\\betaβ. Do the circular points on the curves represent specific β\\betaβ values? If so, please clarify.\n9. The paper mentions that Table 2 compares GSI with SPECS, yet there is no row for SPECS in the table. Could the authors explain this discrepancy?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Novel integration of speculation and reward guidance for reasoning tasks.\n- Strong empirical results showing accuracy improvements.\n- Theoretical analysis and proofs are clear and rigorous.\n- Writing quality is high, making the paper easy to follow.", "weaknesses": "- The scaling factor $\\beta$ (to balance reward and log-likelihood ratio) requires exhaustive search, which reduces practical efficiency.\n- Higher computational cost compared to standard RSD methods.\n- Limited discussion on scalability and efficiency trade-offs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sjq1Oudyjv", "reviewer": "Reviewer_KZqA", "summary": "This paper proposes Guided Speculative Inference (GSI), a test-time algorithm that mixes (i) soft best of n (S-BoN), ii) a process reward model PRM $r$ and iii) speculative drafts from a small model $\\pi_s$ to approximate the reward tilted distibution of a larger base/target model $\\pi_B$. The method computes the tilted rewards as a linear combination of the rewards of generated seequences and loglikelihood ratios of those sequences between the target (big) model and the draft (small) model. The paper claims theoretical guarantees (KL bound to $\\pi_{\\beta,B}$ which is their approximation (GSI) algorithm, and empirical gains on math-reasoning benchmarks (MATH500, OlympiadBench, Minerva, MMLU-STEM, GSM8K) against RSD and S-BoN (with small model) and states that rarely GSI even outperforms S-BoN with big model.", "questions": "* Report end-to-end metrics: GPU-seconds, FLOPs, tokens/sec, peak VRAM, and number of served models per method. \n* Include curves of accuracy vs end-to-end latency under a fixed hardware budget; compare to S-BoN($\\pi_B$) and best-of-n baselines.\n* Can you quantify PRM calibration and correlation with ground truth correctness? \n* Please provide the time-breakdown of different components in the algorithm: how much time is spent on big model and how much time is spent on small model generation. \n* Compare all the frameworks here against just doing speculative decoding with the same Big and Small models and doing S-BoN. \n* Please provide empirical estimates or upper bounds for $\\chi^2(\\pi_B||\\pi_S)$ (or $C_\\infty$) on your tasks; discuss practicality at $\\beta=20$.If the bound is loose, what guidance does it give practitioners?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 1, "contribution": 1, "strengths": "* The algorithmic description with pseudocode in Algorithm 1 is clear.\n* Reasoning-trace visualizations (Fig. 3) aid intuition about reject/accept decisions.", "weaknesses": "* The biggest problem is that the paper claims efficiency but the presented algorithm is not efficient at all on hardware utilization. Paper serves three models on separate H100s; no FLOPs/GPU-seconds/tokens-per-second or memory footprint comparisons vs S-BoN($\\pi_B$) is provided. It seems that GSI uses more memory footprint compared to  S-BoN($\\pi_B$)  (almost 30-50% more given that for reasoning traces the KV-Cache dominates model parameters hence small model and large model usage are comparable). Nonetheless, GSI fails to outperform S-BoN($\\pi_B$) on most benchmarks, thus there is no or little incentive to use GSI.\n\n* Aside from higher burden on the hardware, GSI could claim they are efficient because of being faster. However there is no clear demonstration of GSI scaling better with the time spent for answering problems against the accuracy in those problems. The x-axis in Figure 1, Figure 4 and Figure 5 have $n$, but this is expected and do not convey much information. Nonetheless, the question arises: why would it be desirable to run GSI when someone can run S-BoN($\\pi_B$) with fewer resources and get better results?\n* Theorem 1 bound is practically vacuous at $\\beta=20$ with $r \\in [0,1]$. No empirical check on $\\chi^2(\\pi_B||\\pi_S)$ or the coverage constant $C_\\infty$. The guarantee does not meaningfully inform realistic $n$.\n\nOverall, I do not think this paper will be practically relevant or desirable to implement, and theory seems even further away from being useful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "P3UCojaucz", "reviewer": "Reviewer_j8aM", "summary": "The paper introduces Guided Speculative Inference (GSI), an efficient decoding algorithm that integrates reward-guided generation with speculative sampling from a smaller auxiliary model. GSI provably approximates the optimal soft best-of-n policy and achieves higher accuracy on reasoning benchmarks than both standard soft best-of-n and prior reward-guided speculative decoding methods.", "questions": "Could you provide some more results on diverse setup, including different LLMs, PRMs, and non-mathematic benchmarks?", "limitations": "", "rating": 4, "confidence": 2, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well written and easy to follow.\n2. The proposed approach is theoretically grounded, and the theoretic analysis holds under more realistic conditions compared to SPECS.\n3. GSI empirically achieves better performance than other reward-guided speculation approaches, when evaluated on a wide range of math benchmarks.", "weaknesses": "The authors evaluate on only a single pair of LLMs and a single PRM. This makes the practical generalizability of their findings questionable. Also, the evaluation is heavily focused on math benchmarks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)