ATTS: Asynchronous Test-Time Scaling via Conformal Prediction
Authors: Jing Xiong
Qiujiang Chen
Fanghua Ye
Zhongwei Wan
Chuanyang Zheng
Chenyang Zhao
Hui Shen
Alexander Hanbo Li
Chaofan Tao
Haochen Tan
Haoli Bai
Lifeng Shang
Lingpeng Kong
Ngai Wong
Venue: ICLR 2026 Poster (2026)
Keywords: Conformal Prediction
Test-Time Scaling
Speculative Decoding
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Large language models (LLMs) benefit from test-time scaling but are often hampered by high inference latency. Speculative decoding is a natural way to accelerate the scaling process; however, scaling along both the parallel and sequential dimensions poses significant challenges, including substantial memory-bound execution and synchronization overhead. We introduce ATTS (Asynchronous Test-Time Scaling), a statistically guaranteed adaptive scaling framework that follows the hypothesis testing process to address these challenges. By revisiting arithmetic intensity, ATTS identifies synchronization as the primary bottleneck. It enables asynchronous inference through online calibration and proposes an ordinal classification algorithm that supports a three-stage rejection sampling pipeline, scaling along both the sequential and parallel axes. Across experiments on the MATH, AMC23, AIME24, and AIME25 datasets and across multiple draft–target model families, we show that ATTS delivers up to 56.7x speedup in test-time scaling and a 4.14x throughput improvement, while maintaining accurate control of the rejection rate, reducing latency and memory overhead, and incurring no accuracy loss. By scaling both in parallel and sequential dimensions, we enable the 1.5B/70B draft/target model combination to achieve the performance of the state-of-the-art reasoning model o3-mini (high) on the AIME dataset.

Published as a conference paper at ICLR 2026
ATTS: ASYNCHRONOUS TEST-TIME SCALING VIA CONFORMAL PREDICTION
Jing Xiong'; Qiujiang Chen': : Fanghua Ye', Zhongwei Wan", Chuanyang Zheng? Hui Shen', Chenyang Zhao?. Hanbo Li Chaofan Tao", Haochen Tan?
Haoli Bai Lifeng Shang'. Lingpeng Kong', Ngai Wong'
'The University of Hong Kong 4 University College London
Independent Researcher Huawei Technologies Co., Ltd
The Chinese University of Hong Kong
ABSTRACT
Large language models (LLMs) benefit from test-time scaling but are often ham- pered by high inference latency. Speculative decoding is natural way to accel- erate the scaling process; however, scaling along both the parallel and sequential dimensions poses significant challenges, including substantial memory- cennory-bound ex- ecution and synchronization overhead. We introduce ATTS (Asynchronous Test- Time Scaling), a statistically guaranteed adaptive scaling framework that follows the hypothesis testing process to address these challenges. By revisiting arithmetic intensity, ATTS identifies synchronization as the primary bottleneck. It enables asynchronous inference through online calibration and proposes ordinal classi- fication algorithm that supports a three-stage rejection sampling pipeline, scaling along both the sequential and parallel axes. Across experiments the MATH, AMC23, AIME24, and AIME25 datasets and across multiple traft-target model families, we show that ATTS delivers up 56. 7x speedup in test-time scaling and a 4. 14x throughput improvement, while maintaining accurate control of the rejec- tion rate, reducing latency and memory overhead, and incurring no accuracy loss. By scaling both in parallel and sequential dimensions, we enable the 1.5B/70B draft/tageet model combination achieve the performance of the state-of-the-art reasoning model 03-mini (high) on the AIME dataset.
INTRODUCTION
With the rapid advances in large language models (LLMs), attention is increasingly turning to reasoning models (Guo et al.) 2025 Muennighoff et al. 2025 McCoy et al.) 2024 Shao et al.)
2024) systems that transcend next-token prediction in order to emulate human-like like reasoning be- haviors. These models excel at leveraging complex reasoning chains, especially in test-time scaling settings (Snell et al. 2024 Li al.
2025
Muennighoff et al.
2025
Zeng al. 2025). and have
shown strong potential in mathematical reasoning (Xiong et al.
2022
2023b a).
Guo al.) 2025 constitutes new paradigm that enhances the model's reasoning capabilities by allocating additional computational resources during the inference stage. Typically, test-time scaling can be categorized into two ap- proaches: sequential scaling (Muennighoff et al.] [2025) Guo et al. (2025) and parallel scaling (Chen al., 2025). However, despite its potential, the challenge of efficiently managing increasing sam- pling size or complexity during inference remains a critical limitation, hindering the achievement of high-pes geg-peeforrance deployment.
Test-time scaling (Chen al. 2023 Muennighoff et al. 2025
Benefiting from the shared- shared-prefix mechanism of the inference engines (Kwon et al. 2023 Zheng et al. 2024), parallel scaling (Chen et al.] 2025) increases the number of samples concurrently, thereby partially mitigating the inference-time latency and memory footprint introduced by scaling the per-trajectory token budget (i.e., longer reasoning paths), while simultaneously improving token- sampling throughput.
Although some methods (Huang et al. 2025 Wan et al. 2024 that adopt confidence-bart early
stopping of reasoning chains improve parallel sampling efficiency, problems still remain in memory
"Contact Email: junexiong@comeecthuu Equal contribution.

---

Published as a conference paper at ICLR 2026
efficiency and high inference latency. Another potential issue is that early stopping prines away potentially correct reasoning paths and reduces the diversity of the output space.
Speculative decoding (Li et al. 2024a Leviathan cct al.
Moawey (DB)
2023 Kim et al. 2023] Pan et al. 2025b
75
Seak Prozs
Peenac Mon
Yang et al., 2025) represents a promising approach for accelerating decoding. In this framework, a lightweight draft model generates tokens, which are subsequently validated and refined by a target model. This dual- dual-phase approach not only speeds up inference by offloading most of the generation process to the draft model, but also ensures that the final outputs retain high fidelity, thereby achieving favorable balance between efficiency and accuracy.
20
Figure 1: Memory Overhead vs. Sampling Sizes (QwQ 32B, Token Budget 500)
However, when speculative decoding (Pan et al.) 2025b, Yang et al., 2025) meets test-time scaling, the decoding process faces two key challenges. The first is the memory bottleneck of the target model during the prefill phase. As shown in Figure [ as the number of sampling increases, the memory overhead of the target model tends to grow due to KV cache accumulation. This effect becomes more pronounced target models when attempting to scale the number of requests from the draft model. During real-world deployment concurrency sampling, especially when simultaneously validating multiple long reasoning chains, can lead memory peaks that easily exceed the GPU's maximum capacity, causing the server to crash. Therefore, is crucial to constrain the request budget from the draft model to the target model within a manageable range.
the SGLang server (Zheng et al., (024), high-
Synchmneus
10halt Model free
Specalative
Diaft Model beoode
Decoding
Rest Model Freltl
Sare Model Dacade
Asynchnonous
Speculative
Decoding
Figure 2: Comparison of naive and asynchronous speculative decoding.
In speculative decoding, one stage involves rejection sampling: prior to acceptance, the target model either ranks draft- sentt-generated candidates computes divergence between the draft and target dis- tributions, introducing an additional synchronization overhead bottleneck. As illustrated in Fig. 2 during the multi-turn sampling process (with sampling quantity of 3 each turn), if the target model aims to reject the sampling with the lowest two confidences, ranking operation must be per- formed each turn. Given the limited computational and memory resources, the target model needs to prioritize processing the most important requests. Especially in test-time scaling, when combin- ing sequential and parallel scaling, the synchronization overhead from precise budget control and the pursuit of globally optimal ranking is amplified. Although this issue has been widely discussed in the context of tool calls (Gim et al. 2024 Ginart et al. 2024), it has not been formally proposed in the context of test-time scaling.
To analyze the synchronization bottleneck and address the two challenges mentioned above, we first introduce a novel variant of arithmetic intensity called asynchromous arithmetic intensity to analyze the system bottleneck, and then explore conformal prediction (Vovk et al.) 2005 [003] Romano et al. 2020; Lei et al.) 2018) for ranking predictions to design the asynchronous algorithm. In formulation. conformal prediction defines a prediction set Ca: sampling in C. are rejected, while sampling outside accepted. This yields distribution-fii free guarantee that the right sampling is retained (i.e., lies outside C.)) with high probability, enabling asynchronous test-time scaling. This paper presents the following contributions:
We propose asynchronous arithmetic intensity, a performance metric designed to character- ize and quantify throughput/latency bottlenecks that emerge in test-time scaling scenarios.
2

---

Published as a conference paper at ICLR 2026
We introduce conformal prediction tackle prediction ranking, and lever-ging the result- ing ranking construct stable prediction sets that mitigate GPU-memory bottleneck risks.
We propose ATTS, training-free, lossless acceleration method that achieves 56.7x speedup in test-time scaling and a 4.14x throughput improvement in both sequential and parallel settings.
2 PRELIMINARY
We first introduce how to build the prediction set in the classical setup, and then present our setup.
Classic Setup. Formally,
Deal ((X, Yi).................... (X,, Yn))
denote the calibration dataset. Each pair (X,, Yi) for i = 1, ..... ... is data point, consisting of an X; (the input question for the i-th example) and a ground truth denoted as Y, The symbol n denotes the size of the calibration dataset. For each input X;, we draw candidate sampling
where m is the number of samples per input (the sampling budget), and Y denotes the k-th can- didate sample. In adaptive prediction-set construction (Romano al. 2020 Angelopoulos al.) 2020 Huang et al. 2023), the conformity score for each sample is computed via a softmax function:
exp(- e(x;, ((a))
Σ exp (( Y?'))
where -í denotes the negative fog-likelihood loss. Next, the global conformity threshold T is ob- tained by computing the p-quantile over all candidate scores:
Qr (f0b pi ,.... n,;j, 1....,m?}.
(4)
[(n + 1)(1 aa) p
(5)
n
and a E (0, 1) is the user-specified miscoverage rate. To satisfy conditional coverage, The predic- tion set for input X; is then defined as
C. (X,), = 5>
T, 1,..,,
which ensures that the resulting set includes the ground truth with probability at least a. Al- though conformal prediction in the classical setting can provide guaranteed conditional coverage, the conformal scores assigned candidate outputs are typically required be normalized, shown in 3 and this normalization inherently introduces bottleneck to parallelization. Therefore, in the subsequent Ordinal Classification, we transform the problem into a hypothesis testing framework via p-values to avoid normalization, with a proof of its coverage guarantee provided in Appendix A.8
Problem Setup. In the asynchronous test-time scaling setup, we leverage a draft model for fast sampling and delegate verification to slower target model. Unlike classical rejection sam- pling (Chen et al. 2023). which approximates target distribution with draft distribution, we focus on accurately predicting the rejection rafe, thereby reducing VRAM out-of-memory risk and the synchronization overhead caused by global ranking or softmax function. Given predefined a, we estimate a confidence level such that the ground truth y falls within the prediction set C. (Y) with probability at least 1 a:
Ply E C. 2-
(7)
where is conventionally interpreted as the significance level (e.g.. 0.05 corresponding to 95% confidence). this work, however, we reinterpret as the rejection rate of the target model.
3

---

Published as a conference paper at ICLR 2026
Ordinal Classification. In typical inference engines (Zheng al. 2024 K Kwon al. 2023). par-
ticularly those with asynchronous scheduling, obtaining the normalized scores for all sampling in different batches can be challenging. To avoid normalization and global ranking operations.
formulate the task of constructing prediction sets as an ordinal classification (Dey et al. 2023 Xu
al. 2023), meaning that predict the ranks of all samples. Formally, aim to ensure:
P( E C. 2 a, Vi (I....., m)),
where P(j' E (Y)) denotes the probability that the i-th candidate step ÿ' lies within the pre- diction set Ca. and m represents the number of sampled steps. This procedure provides marginal coverage, meaning that the coverage guarantee holds on average over the distribution of test inputs. The stronger notion of conditional coverage aims ensure
P( E CY) |X 2) Vi (l...,,., Vr.
That is, it provides probabilistic guarantee for the sampled outputs corresponding each input instance. To achieve this, our setup focuses on developing asynchronous algorithms for ranked prediction. where the construction of the prediction set ensures that its size matches the predefined budget while maintaining both marginal and conditional coverage. This approach avoids the need for normalization while addressing the challenges posed by asynchronous scheduling.
100
200
Ferfomannc
Performance
10
700
1000
Execution Time
300
400
200
200
100
Sample
(a) Sampling latency.
(b) End-to-end latency per turn.
Figure Execution cost comparison between synchronous and asynchronous test-time scaling.
CHALLENGE AND DESIGN
To identify performance bottlenecks in the classic setup, we introduce arithmetic intensity (Spector
& Re 2023 which measures the utilization of arithmetic units. It is defined as:
(10)
where f is the number of floating-pon point operations (FLOPs) and is the number of bytes accessed.
3.1 Q1: WHAT ARE THE EMERGING PERFORMANCE BOTTLENECKS?
Speculative decoding (Leviathan et al. 2023) accelerates inference by overlapping computation with memory accesses, enabling multiple draft tokens to be validated in parallel. Its main bottleneck is parallel score computation (Yin et al. 2024), making the process computation-bound.
Building upon this perspective, asynchronous scaling can be seen an even more aggressive paral- lelization strategy. The target model validates far more tokens in parallel than speculative decod- ing, which intensifies the prefill bottleneck and results in total computation time far exceeding total memory access time, as illustrated by the comparison between the green and yellow lines Fig. 4a
As shown in Fig. 3b synchronization overhead grows exponentially with the number of sampling turns. the parallel scaling setting (Fig. Bap, this overhead increases linearly with the number of concurrent samples. To this end, observe Fig, 4althat increasing the sampling size naturally raises arithmetic intensity (with memory access time being negligible). To account for synchronization costs within arithmetic intensity, we define asynchronous arithmetic intensity

---

Published as a conference paper at ICLR 2026
AgyCc
340 120-3
ane Time Syndharcc
-O- Hamory Times
320
236
00
Radta
60
200
16
(a) Arithmetic intensity vs. sampling size.
(b) Asynchronous arithmetic intensity comparison.
Figure 4: Analysis of arithmetic intensity.
te x Te Tm + T. tm x b T T, T.
Te
r
(11)
where T. is computation time, Tm is memory access time, te and 6m are the per-unit costs of com- putation and memory access, respectively. It can be observed from Fig. 4bjthat under classic setups, r decreases the sampling size increases which indicates that synchronization overhead emerges as the primary bottleneck.
3.2 Q2: HOW THE PREDICTION SET CONSTRUCTED?
Online Calibration. Conformal prediction typically relies held-out calibration set deter- mine the threshold т. However, in the test-time scaling setup, held-out examples are generally unavailable. To address this limitation, we propose an online calibration strategy. Specifically, m outputs are pre-sampled for each imput in the test set, yielding (Y......, Y:mi). Previous efforts (Ding et al. 2023; Romano et al. 2020) impose strict sum-to-one constraint on the conformal scores under the classification setting (where events are mutually exclusive). In contrast, we compute conformal p-values (Bates et al.) 2023] Jin & Candès, 2023 Wang et al., (024) under ordinal classification setup, where the events are not mutually exclusive and the ordinal relationships are preserved. In this setup, we relax the strict requirement that conformity scores sum to one, and instead directly define:
2e = -
(12)
This formulation is used to estimate conformal p-values for rejection sampling:
E m 1 s/) +1
P
(13)
In this formulation, o denotes the conformity score of the test-time candidate Y which represents the k-th sample of the Ç-th input on the test set, and se are the scores from the calibration set (X,, Y?'). The indicator function 1(-) returns 1 when the condition is satisfied. The p-value based on formula [3] guarantees marginal coverage at level 1 - a, which can be intuitively explained as calculating one's rank by comparing the conformity score with the score from the p entire calibration set. Conditional coverage can be achieved by adjusting the comparison with the p- m calibration from the current input sample.
in the
A detailed proof of these two approaches is provided in Appendix A.8 The conformal Sps-value governs rejection sampling: candidate is accepted P > a, ensuring that only high-confidence outputs retained, thereby achieving precise budget control.
5

---

Published as a conference paper at ICLR 2026
R
Uver Racpert
febrence
CoT
Intrence
CeT Stap
Cut
Kinwee
COT Stup
Step 1: Draft Model Sampling
Stop 2: Verification
Step 3: Target Model Sampling
Nodt
SSL
Ordinal
Accest
Roject
Raqucs
Casster
feccier with reporrs..
Cassdis
Culbeiton Set
Taight Model
Torpet Mroee
Target Model Scoring Ranking
Ressmpling Hhelecass
Ressmraiig /horlcctio
Ortinal Bort-
Accept
Figure 5: Asynchronous test-time scaling pipeline. The green box illustrates parallel scaling and follows the rejection sampling procedure, while the blue box illustrates sequential scaling.
Budget Prediction. Let B denote the predefined budget (i.e., the number of candidates reject). Given test-time input Xg, sample m candidate CoTs (Yg)...,, y.m)) in each turn and then compute their corresponding p-values P
Importantly, this sampling and evaluation process conducted asynchronously; each candidate is generated independently and evaluated for its p-value without requiring synchronization with other candidates. As a result, the outputs implicitly exhibit a descending order:
PE 2..2..
(14)
The ordered set be partitioned using threshold construct the prediction set, by directly comparing each candidate's p-value with the miscoverage threshold a. Specifically, the prediction set includes all candidates whose p-values satisfy:
C. (Yg) = (ve k (1. (1...., m.....m..pp a).
(15)
This formulation ensures that the selected candidates meet the coverage rate. Equivalently, this can be interpreted rejecting the top-B candidate sampling.
3.3 How TO PERFORM REJECTION SAMPLING VIA CONFORMAL PREDICTION?
We adopt three-stage sampling pipeline, illustrated in Fig. 5 to realize rejection sampling with target rejection rate a.
Draft Model Sampling. Given input tokens æ1.N-1, the draft model proposes m candidate con- + K.-1- by sampling from the draft model qa: tinuations of length K₂ in each turn, denoted i
UWIN+Ka-1 ~ qa(- I1:N-1), j = m.
(16)
Verification. For each candidate sampling i UN.N+K2-1 we score it under the target model q. by computing the logits qe ( K x1.N-1) and converting to a conformity score. Using the calibration set, we compute p-value for each candidate and reject those inside C.: otherwise
accept.
Target Model Sampling. Although classical rejection sampling discards the rejected samples from the draft model and resamples entirely from the target model, to the token budget proceed as follows. In each turn, the per-turn target-side token budget is K,: for each candidate in Co. we let the target model qe continue generation using that candidate from ga prefix for up

---

Published as a conference paper at ICLR 2026
Table 1: Comparison when draft and larget models are from different families under marginal (Mar Cov.) and conditional (Con Cov.) coverage. SMar- Scan end-to-end speedup (x) un- der marginal/conditinall coverage (larger faster); Values red indicate lossless acceleration. Gray rows indicate draft models (non-reasoning).
Datasct
Draft Model (DM)
Mar Cov. Con Cov. DM Baseline TM Bascline SMa (x) Scoa (x)
QwQ-32B (RL-tuned reasoning model) as Target Model
87.0
94.0
83.0
96.0
1.8
1.2
Qween:.5-7B-Instrucctin
86.0
96.0
84.0
96.0
7.2
5.4
MATH100
SeesSeek-RI-istillLL-iee
96.0
96.0
85.0
96.0
1.4
1.4
Llama-3.1 Instruct
87.0
95.0
75.0
96.0
2.1
2.2
DeepSeek-R 1ee--R--Diin
86.7
66.7
60.0
86.7
4.0
2.6
Owen2.5-77-IIssruccton
46.7
33.3
33.3
86.7
5.7
10.1
AIME24
DeepSeek Llama-3.1-88
80.0
80.0
80.0
86.7
2.3
Llama-3.1-8B- Instruct
33.3
40.0
13.3
86.7
4.5
2.8
DeepSeek-R 1roSSeck-R1-iisilll---o -i-istiill-Qwen--
53.3
46.7
40.0
73.3
2.0
1.2
Owen2.5-7B-Instruccii
33.3
40.0
26.7
73.3
14.5
12.8
AIME25
DeepSeek 1--tmaa-,.--B--ssstse
66.7
60.0
46.7
73.3
2.1
1.8
-lama-3..-8BB-nsiteee
26.7
33.3
20.0
73.3
6.7
2.3
DeepSeek-R 1-Distill----eeeeeeee -welll--weenn.S
88.0
90.0
74.0
94.0
1.0
1.2
Qwen2.55-7B-Istrucctin
76.0
72.0
68.0
94.0
10.4
8.2
AMC23
DeepSeek Ilama-3.1180 1-B--...--B--nstruucc
92.0
80.0
94.0
1.5
1.1
Llama 3.1 11----Imm Instruct
68.0
68.0
44.0
94.0
3.6
1.7
sl.1-328 (SFT-tuned reasoning model) as Target Model
DeepSeek-R 1--bbeck--1-Ditilll---- 1-Distill--wwen-1151
86.0
95.0
83.0
96.0
0.7
0.9
(wen2.55-7-IInsrrucee
89.0
84.0
96.0
2.9
2.6
MATH100
DeepSeck Lllmm-3.11-80 1-8--33.11BB-nntrree
88.0
95.0
85.0
96.0
0.8
0.8
Llama-3.1-8B-Insse
79.0
85.0
75.0
96.0
1.3
1.6
DeepSeck-R1-oe 1-iist--ill---- Swwllll--een-1..mm 1..B
73.3
80.0
60.0
86.7
4.4
2.2
(wen2.5-7B-Instrueeee
40.0
33.3
33.3
86.7
22.2
13.5
AIME24
DeepSeek 1--aama.1----nntttarr
73.3
73.3
80.0
86.7
2.4
3.1
Llama-3.18BB-Instuu.
26.7
40.0
13.3
86.7
5.6
3.4
DeepSeek-FR -i-is--------------- Owen-1.5p
60.0
60.0
40.0
66.7
2.9
2.0
Qwen2.557B-Instrruchh
33.3
40.0
26.7
66.7
18.5
11.9
AIME25
DeepSeek 1--ma-3.1-8B--mme
66.7
60.0
16.7
66.7
2.3
1.7
1lama-3.1-8B-Instruu
26.7
20.0
20.0
66.7
5.6
4.2
DeepSeck-R 1-Distill-Qwen-11
86.0
86.0
74.0
96.0
1.4
1.0
1ree22.577-Instruueee
74.0
78.0
68.0
96.0
14.5
11.4
AMC23
DeepSeek --Sttrrrctttt
92.0
96.0
80.0
96.0
1.6
1.6
Llama-3.1 3.1----.1--B-... IB---ruttttt
54.0
64.0
44.0
96.0
5.0
3.6
K, tokens, stopping earlier if an end token is encountered. Appendix A.5 we provide compari- son between continuing sampling and resampling with g. for large-scale scaling performance.
Termination. We iterate the above rejection sampling until final answer is detected, the maxi- mum number of turns is reached, or the overall token limit is exceeded. As highlighted in Fig. increasing the number of turns enables sequential test-time scaling (blue box), while increasing the number of candidates per turn enables parallel test-time scaling (green box).
EXPERIMENT
We provide the hyperparameters and other task details used in our experiments Appendix 4 and evaluate performance under two settings: marginal coverage (Mar Acc.), which measures whether the budget of prediction set align on average across test inputs, and conditional coverage (Con Acc.), which imposes stricter requirement that the guarantee holds for each individual input instance.

---

Published as a conference paper at ICLR 2026
Table Comparison within the same model family, ThETR and Ti denote token- consumption ratios (x) under marginal/conditional coverage relative to SpecThink (Th) and
SpecReason (Re). Values in red indicate lossless acceleration (aceuracy z TM baseline).
Gray rows Skywork-Oh ORl draft model.
Accuracy
Token Consumption (x)
Dataset
Draft / Target Model
Mar.
Cond.
SpecTh.
SpecRe.
T
Tgc
T
Tka
Qwen2.5-7B/32B-
86.0
81.0
79.0
73.7
0.60
0.66
0.68
0.72
s1.1-7B/32B
88.0
87.0
85.0
73.7
0.50
0.54
0.57
0.44
MATHI00
DeepSeek-R1-D RI-
Prssill-Qwee-1.5BB2
88.0
87.0
84,0
76.7
0.48
0,52
0.53
0.39
Skywork-OR1-7B/3BB
88.0
89.0
70.0
75.8
0.42
0.37
0.28
0.31
Qwen2.55-7B322--nnse Instruct
33.3
40.0
33.3
33.3
0.61
0.67
0.69
0.73
s1.1-7B/32B
66.7
73.3
40.0
26.7
0.47
0.52
0.62
0.50
AIME24
Distill-QQen-J..ee Skywork-OR1-77/328
DeepSeek-RI
86.7
80.0
66.7
66.7
0.46
0.50
0.56
0.42
86.7
80.0
60.0
73.3
0.33
0.27
0.19
0.24
Qwen2.5-7B/322-Insse
40.0
33.3
26.7
40.0
0.62
0.68
0.70
0.74
sl.1-7B/32B
53.3
53.3
33.3
40.0
0.49
0.53
0.64
0.52
AIMB25
60.0
53.3
46.7
35.7
0.47
0.50
0.55
0.43
Skywork-ORI-7B/3BB
60.0
53.3
40.0
53.3
0.41
0.36
0.29
0.22
Qwwen2.5-7B/22B-nnnmme
72.0
70.0
74.0
72.0
0.63
0.69
0.71
0.75
$1.1-7B/32B
82.0
78.0
76.0
78.0
0.48
0.52
0.62
0.48
AMC23
92.0
88.0
82.0
80.0
0.46
0.50
0.57
0.44
Skywork-OR1-7B/32B
96.0
94.0
82.0
86.0
0.39
0.34
0.37
0.28
4.1 ASYNCHRONOUS TEST-TIME SCALING ACROSS DIFFERENT MODEL FAMILIES.
Table reports the results of asynchronous test-time scaling when the draft model (DM) and target model (TM) come from different families. We have the following key takeaways: i) ATTS can match the performance of the target model itself. This approach effectively reduces computational overhead while maintaining high-quality outputs, up to 22.22x acceleration. ii) The most challenging datasets, AIME24/25, show strong performance in marginal coverage setup, while the other two datasets (MATH100 and AMC23) demonstrate superior results in conditional coverage setup. This highlights that while marginal coverage allocates more computational resources to the most difficult parts of the tasks effectively, conditional coverage ensures more reliable results at the individual input level, especially in simpler tasks, ensuring that each question is answered correctly. iii) It shows that while reasoning models consume more tokens during inference, using reasoning model as the draft model provides better scaling performance than a non-reasoning model, though the non- reasoning model offers the highest acceleration. iv) When the average length of the reasoning chain output by the draft model exceeds that of the target model, the acceleration ratio is typically less than on simpler datasets such as MATH and AMC23.
4.2 PERFORMANCE OF BUDGET PREDICTION
In this section, we evaluate the accuracy of budget prediction under marginal and conditional cov- erage settings. This shows how well our method controls target model interventions in rejection sampling, reflecting the accuracy of conformal prediction in estimating the rejection rate.
Marginal Coverage. Figure 6b we report the accuracy of the target-model intervention rate under marginal coverage, where the rejection rate is predicted at the dataset level. Budget-prediction accuracy across the full dataset is high, especially with the 6a-sam configuration, whose absolute error stays within 5%. With K₃ 500 tokens for calibration and K₃ 500 for sampling, the error remains within 2%. This directly highlights the importance of constructing a diverse calibration set for maintaining high prediction accuracy.
8

---

Published as a conference paper at ICLR 2026
64 samples predict 500 samples prodict
64 samples predict 701
Take Over
Remaining
crmple pralict
25
20
Perceniage (%)
74.9%
74.2%
74.9%
69.5%
Valoe
qu-0225
25.1%
25.8%
25.1%
30.5%
Problem
Setting
Left: Conditional Coverage.
(b) Right: Marginal Coverage.
Figure 6: Budget prediction accuracy with a rejection rate 0.25. (a) Per-batch conditional
coverage: each bar represents the prediction error per problem under different settings ("K samples predict L" denotes K parallel samples with calibration budget 500 and sampling budget L). (b) Dataset-level marginal coverage: Take Over denotes the fraction of samples handled by the target model, and Remaining denotes those kept from the draft model.
Conditional Coverage. In practice, we require precise per-batch budget control, rather than single aggregate budget over the entire test set. Due to the limited capacity of the target model's server, it cannot process all requests concurrently. As a result, inference is performed in batches, with the token budget enforced for each batch to meet the load constraint. In Figure we report the accuracy of the target model intervention rate under conditional coverage. Under online calibration with a rejection rate of 25%, when Ka = 500 in both the calibration and sampling stages, the 16-sample and 64-sample settings achieve similar accuracy. However, when the calibration stage uses K₂ 500 but the sampling stage uses K₂ 700, the 64-sample setting attains significantly higher budget-prediction accuracy. This indicates that increasing the number parallel samples can improve budget prediction accuracy when the sampling token budget differs from the calibration token budget (i.e., under calibration calibration-samllngg token budget mismatch).
4.3 ASYNCHRONOUS TEST-TIME SCALING WITHIN THE SAME MODEL FAMILIES
In this section, examine the performance across models within the same family, including both reasoning and non-reasoning models in Table 2 In this setting, since the target model and draft models share the same vocabulary, models within the same family, such as Speculative Thinking (Yang et al. 2025), denoted as SPEC- THINK. Our findings are as follows: i): When the draft and target models belong to the same family, in most cases, the best performance is achieved under the setting of marginal coverage, even on simpler datasets like MATH and AMC23. ii): Across datasets, DeepSeek and Skywork show the strongest gains challenging benchmarks (AIME, AMC), while Qwen2.5 performs competi- tively on MATH100 but lags significantly on harder tasks. iii): Moreover, sl. achieves moderate improvements, usually surpassing Qwen but not reaching the level of Skywork DeepSeek. iv): Fi- nally, SpecReason and SpecThink generally underperform compared with ATTS and consume more tokens, especially when the draft model is reasoning model dataset, suggesting that their effectiveness remains limited on more complex reasoning tasks.
can compare against baselines that only applicable to
the more challenging AIME
4.4 ANALYSIS OF TOKEN BUDGET AND LATENCY Latency and Throughput. In this part, we analyze the trade-offs between latency and through- put under different sampling and token budget settings, i): As shown in Figure 7aa the latency of SpecReason and SpecThink consistently increases with the number of samples, highlighting the cost of scaling up sampling. In contrast, our method significantly reduces the sampling latency and achieves the lowest inference latency under the condition coverage setting, ii): The time overhead of online calibration is nearly negligible, particularly at larger sample sizes. iii): Meanwhile, Figure (bb illustrates how throughput varies with the per-sample token budget for methods such SpecReason

---

Published as a conference paper at ICLR 2026
300090
Candiona Coneage
480
Hagna Coveaags Spatfmarcn
250010
Sanciatre Trandng
240
Threugheet tekembi
200
30000
320
110
50800
1084
204-
2350
8150
10366
Bualget
(a) Latency increases with the number of samples.
(b) Throughput variation under the 16-sample setting.
Figure 7: Analysis of latency and throughput trade-offs under different sampling and token budget.
TM
TM Cond.
TM Marg.
Marg
12000
60
10000
Token Budget
8000
Accurccy (%)
6000
30
DerpBock Mar
4000
Quen Mir
20000
18
AIME24
AIMEZS
AMCZ3
MATH100
4
16
20
24 28
Dataset
Turn
(a) Token consumption under 16-sample setting.
(b) Multi-turn evaluation AIME25. Solid/dashed lines denote Mar//Con.. coverage.
TM/DM Target/Draft Model; Cond./Marg. Con- ditional/Marginal Coverage.
DeepSeek:
LSB/SI.1-32B. Qwen: 7B/S1.1-32B.
Figure 8: (a) Token consumption under the 16-sample setting across different datasets. (b) Multi- turn evaluation results on AIME25 with increasing turns.
and SpecThink, revealing diminishing returns as the token budget becomes large. Our method is able to maintain high throughput even under very large budgets, especially in the marginal coverage setting. Overall, these results (Figure 7 provide insights into the balance between efficiency and performance when designing inference strategies. Under the setting of 16 samples and no limit the maximum sequence scaling turns per sample, we achieved 56.7x speedup in inference and a 4.14x throughput improvement compared to the bascline.
pressenss the token consumption under the 16-sample setting. This Token Consumption. Figure[8a presents
includes sampling performed solely by the target model or the draft model as baselines, as well as asynchronous sampling under both condition coverage and marginal coverage settings. Compared with the two baselines, our method can significantly reduce token consumption, especially under the condition coverage setting, it enables budget prediction at the instance level.
4.5 MULTI-TURN EVALUATION RESULTS
In this section, evaluate the results the AIME25 dataset under settings with more turns. Unlike the previous setting, we reduce the token budget per turn but increase the number of iterations, allowing the target model to generate with the number of turns. Both marginal coverage and conditional coverage eventually converge to the value. The results show that under fixed sampling size budget, increasing the number of turns still does not break the performance upper bound.
samples. As shown in Figure 3b the accuracy increases
5 CONCLUSION
We presented ATTS (Asynchronous test-time scaling), framework that addresses the core ineffi- ciencies of test-time scaling in LLMs. By refining arithmetic intensity and introducing online cali- bration with rejection sampling pipeline, ATTS effectively controls rejection rates while reducing latency and memory overhead. Experiments on multiple reasoning benchmarks confirm that ATTS achieves better efficiency and reliability than speculative baselines. This work establishes ATTS as a practical and principled approach for scalable test-time scaling, with potential extensions to dynamic adaptation and real-world deployment.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported in part by the Theme-based Research Scheme (TRS) project T45-701/22-R of the Research Grants Council of Hong Kong, and part by the AVNET-HKU Emerging Micro- electronics and Ubiquitous Systems (EMUS) Lab.
REFERENCES
Anastasios Angelopoulos, Stephen Bates, Jitendra Malik, and Michael Jordan. Uncertainty sets for image classifiers using conformal prediction. arXiv preprint arXiv:2009. 14193, 2020.
Felipe Areces, Christopher Mohri, Tatsunori Hashimoto, and John Duchi. Online conformal predic- tion via online optimization. Forty-second International Conference on Machine Learning.
Art of Problem Solving. Aime problems and solutions. https://artoffroolleeet lving. com/ wwwpwwiiiiiinn
php/AIME Problems and Solutions Accessed: 2025-09-07.
Stephen Bates, Emmanuel Candès, Lihua Lei, Yaniv Romano, and Matteo Sesia. Testing for outliers with conformal p-values. The Annals of Statistics, 51(1):149-178 2023.
Aadyot Bhatnagar, Huan Wang, Caiming Xiong, and Yu Bai. Improved online conformal prediction via strongly adaptive online learning. In International Conference on Machine Learning, pp. 2337-2363. PMLR, 2023.
Tianle Cai, Yuhong Li, Zhengyang Geng, Hongwu Peng, Jason D Lee, Deming Chen, and Tri Dao. Medusa: Simple IIm inference acceleration framework with multiple decoding heads. arXiv preprint urXiv:2401.1011 2024.
Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre, and John Jumper. Accelerating large language model decoding with speculative sampling, arXiv preprint arXivv2302.00118. 2023.
Mouxiang Chen, Binyuan Hui, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Jianling Sun, Junyang Lin, and Zhongxin Liu. Parallel scaling law for language models. arXiv preprint arXiv:2505.10477 2025.
Alex Derhacobian, John Guibas, Linden Li, and Bharath Namboothiry. Adaptive prediction sets with class conditional coverage.
Prasenjit Dey, Srujana Merugu, and Sivaramakrishnan R Kaveri. Conformal prediction sets for ordinal classification. Advances in Neural Information Processing Systems, 36:879-899 2023.
Tiffany Ding, Anastasios Angelopoulos, Stephen Bates, Michael Jordan, and Ryan Tibshirani. Class-conditional conformal prediction with many classes, Advances in neural information pro- cessing systems, 36:64555-64576. 2023.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, al. The llaa 3 herd of models. arXiv e-prints, pp. arXiv-2407, 2024.
In Gim, Seung-seob Lee, and Lin Zhong. Asynchronous Ilm function calling. arXiv preprint arXiv:2412.07017. 2024.
Antonio A Ginart, Naveen Kodali, Jason Lee, Caiming Xiong, Silvio Savarese, and John Emmons. Asynchronous tool usage for real-time agents. arXiv preprint 2rX. 2024.
Gonzalo Gonzalez-Pumariega, Leong Su Yean, Neha Sunkara, and Sanjiban Choudhury. Robotouille: An asynchronous planning benchmark for Ilm agents. arXiv preprint arXiv:2502.05227, 2025.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma. Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, arXiv preprint arXiv:2501. 250...2949.... 12948, 2025.

---

Published as a conference paper at ICLR 2026
Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, et al. Olympiadbench: challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp- 3828 3850, 2024.
Jujie He, Jiacai Liu, Chris Yuhao Liu, Rui Yan, Chaojje Wang, Peng Cheng, Xiaoyu Zhang, Fuxiang Zhang, Jiacheng Xu, Wei Shen, Siyuan Li, Liang Zeng, Tianwen Wei, Cheng Cheng, Bo An, Yang Liu, and Yahui Zhou. Skywork open reasoner 1 technical report. arXiv preprint arXiv:2505.22312. 2025.
Dan Hendrycks. Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurIPS, 2021.
Chengsong Huang, Langlin Huang, Jixuan Leng, Jiacheng Liu, and Jiaxin Huang, Efficient test-time scaling via self-calibration. arXiv preprint arXiv:2503. 00030, 2025.
Jianguo Huang, Huajun Xi, Linjun Zhang, Huaxiu Yao, Yue Qiu, and Hongxin Wei. Conformal prediction for deep classifier via label ranking. arXiv preprint arXiv:2310.064:: 2023.
Ying Jin and Emmanuel J Candès. Selection by prediction with conformal p-values. Journal of Machine Learning Research, 24(244):1-41, 2023.
Schoon Kim, Karttikeya Mangalam, Suhong Moon, Jitendra Malik, Michael W Mahoney, Amir Gholami, and Kurt Keutzer. Speculative decoding with big little decoder. Advances in Neural Information Processing Systems, 36:39236-39256, 2023.
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems princi- plex, pp. 61 6-1622,,20 2023.
Jing Lei, Max G'Sell, Alessandro Rinaldo, Ryan J Tibshirani, and Larry Wasserman. Distribution- free predictive inference for regression. Journal of the American Statistical Association, 113 (523):1094-1111, 2018
Yaniv Leviathan, Matan Kalman, and Yossi Matias, Fast inference from transformers via speculative decoding. In International Conference Machine Learning, pp. 19274-19288 PMLR, 2023.
Dacheng Li, Shiyi Cao, Chengkun Cao, Xiuyu Li, Shangyin Tan, Kurt Keutzer, Jiarong Xing, Joseph Gonzalez, and lon Stoica. S*: Test time scaling for code generation. arXiv preprint arXiv:2502. /4382, 2025.
Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. Eagle: Speculative sampling requires rethinking feature uncertainty. arXiv preprint arXiv:2401.1507 2024a.
Yuhui Li, Fangyun Wei, Chao Zhang, and Hongyang Zhang. Eagle-2: Faster inference of language models with dynamic draft trees. arXiv preprint arXiv:2406.16858, 2024b.
Zichao Li and Zong Ke. Cross-modal augmentation for low-resource language understanding and generation. In Proceedings of the 1st Workshop on Multimodal Augmented Generation via Multi- modal Retrieval (MAGMaR 2025), pp. 90 90-99, 2025.
math-ai.
Amc23:
Math reasoning dataset. Accessed: 2025-09-07.
ai/amc23
R Thomas McCoy, Shunyu Yao, Dan Friedman, Mathew D Hardy, and Thomas L. Griffiths. When a language model is optimized for reasoning, does it still show embers of autoregression? an analysis of openai ol. arXiv preprint arXiv:2410.0179) 2024.
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. s1: Simple test-time scaling. arXiv preprint 2tt::2500119999 2025.
12

---

Published as a conference paper at ICLR 2026
OpenAl. Learning to reason with 1lms, September 2024. URL http:ttpss//oomaa..oo//eteeee
-e wwwhoeewwthhhneeeee-----e
OpenCompass. Aime 2025 dataset (aime i & ii). opencompass/ALMM20.. Accessed: 2025-09-07.
attps///bgggnggaa...c//aaattttt
Jiayi Pan, Xiuyu Li, Long Lian, Charlie Snell, Yifei Zhou, Adam Yala, Trevor Darrell, Kurt Keutzer, and Alane Suhr. Learning adaptive parallel reasoning with language models. arXiv preprint arXiv:2504. 15466, 2025a.
Pengfei Pan, Lizi Chen, Qi He, Keyu Yuan, Han Wang, and Wenchao Zhang. Finscra: An 1lm- powered multi-chain reasoning framework for interpretable node classification text-attributed graphs. Preprints, February 2026. doi: 10. 2r22009///prrrin2220000000000000
URL https:/
doi. www.oogg...... 220.222200 2094 2ttp://...004//pppepnm222222222
0209.
Rui Pan, Yinwei Dai, Zhihao Zhang, Gabriele Oliaro, Zhihao Jia, and Ravi Netravali. Specrea- son: Fast and accurate inference-time compute via speculative reasoning. arXiv preprint arXiv:2504.07891, 2025b.
Yaniv Romano, Matteo Sesia, and Emmanuel Candes. Classification with valid and adaptive cover- age. Advances in neural information processing systems, 33:3581-3591, 2020.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepscekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300 2024.
Charlic Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314, 2024.
Benjamin Spector and Chris Re. Accelerating Ilm inference with staged speculative decoodinn arXiv preprint arXiv:2308..4422 046223 2023.
Qwen Team. Qwen2.5: A party of foundation models, September 2024. UtL/tttps://qwen github..o/bloo//paee.
Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, March 2025. URL hittps://wwenie githum..i............ ho/biog////ggg qwg- pww-32b/
Vladimir Vovk, David Lindsay, Ilia Nouretdinov, and Alex Gammerman. Mondrian confidence machine. Technical Report, 2003.
Vladimir Vovk, Alexander Gammerman, and Glenn Shafer. Algorithmic learning in a random world, volume 29, Springer, 2005.
Guangya Wan, Yuqi Wu, Jie Chen, and Sheng Li. Reasoning aware self-consistency: Leveraging reasoning paths for efficient Ilm sampling. arXiv preprint 1rXiv:2408..7017, 2024.
Xiaoning Wang, Yuyang Huo, Liuhua Peng, and Changliang Zou. Conformalized multiple testing after data-depnndent selection. Advances in Neural Information Processing Systems, 37:58574- 58609, 2024.
Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Inference scaling laws: An empirical analysis of compute-optimal inference for problem-solving with language models. arXiv preprint urXiv:2408.00w
2024.
Jing Xiong, Chengming Li, Min Yang, Xiping Hu, and Bin Hu. Expression syntax information bot- tleneck for math word problems. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2166 2171, 2022.
Jing Xiong, Zixuan Li, Chuanyang Zheng, Zhijiang Guo, Yichun Yin, Enze Xie, Zhicheng Yang, Qingxing Cao, Haiming Wang. Xiongwei Han, et al. Dq-lore: Dual queries with low rank ap- proximation re-ranking for in-context learning. arXiv preprint 2r.iv:2310.02954. 2023a.
13

---

Published as a conference paper at ICLR 2026
Jing Xiong, Jianhao Shen, Ye Yuan, Haiming Wang, Yichun Yin, Zhengying Liu, Lin Li, Zhijiang Guo, Qingxing Cao, Yinya Huang, et al. Trigo: Benchmarking formal mathematical proof re- duction for generative language models. In Proceedings of the 2023 Conference Empirical Methods in Natural Language Processing, pp. 11594-11632. 2023b.
Jing Xiong, Jianghan Shen, Fanghua Ye, Chaofan Tao, Zhongwei Wan, Jianqiao Lu, Xun Wu, Chuanyang Zheng, Zhijiang Guo, Lingpeng Kong, et al. Uncomp: Uncertainty-awan long- context compressor for efficient large language model inference. 2024.
Jing Xiong, Jianghan Shen, Chuanyang Zheng, Zhongwei Wan, Chenyang Zhao, Chiwun Yang, Fanghua Ye, Hongxia Yang, Lingpeng Kong, and Ngai Wong. Parallelcomp: Parallel long-context compressor for length extrapolation. arXiv preprint arXiv:2502.14317, 2025.
Yunpeng Xu, Wenge Guo, and Zhi Wei. Conformal risk control for ordinal classification. In Uncer- lainty in Artificial Intelligence, pp. 2346-2355. PMLR, 2023.
Yuchen Yan, Yongliang Shen, Yang Liu, Jin Jiang, Mengdi Zhang, Jian Shao, and Yueting Zhuang. Inftythink: Breaking the length limits of long-context reasoning in large language models. arXiv preprint arXiv:2503.0000 06692, 2025.
Wang Yang, Xiang Yuc, Vipin Chaudhary, and Xiaotian Han. Speculative thinking: Enhanc- ing smael-model reasoning with large model guidance at inference time. arXiv preprint arXiv:2504. /2329, 2025.
Ming Yin, Minshuo Chen, Kaixuan Huang, and Mengdi Wang. A theoretical perspective for spec- ulative decoding algorithm. Advances in Neural Information Processing Systems, 37:128082- 1281 17, 2024.
Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. Does re- inforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv preprint arXiv:2504.138: 2025.
Zhiyuan Zeng, Qinyuan Cheng, Zhangyue Yin, Yunhua Zhou, and Xipeng Qiu. Revisiting the test-time scaling of ol-like models: Do they truly possess test-time scaling capabilities? arXiv preprint arXiv:2502. 12215, 2025.
Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Livia Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E Gonzalez, al. Sglang: Efficient execution of structured language model programs. Advances in neural information processing systems, 37: 62557-62583, 2024.
Dongsheng Zhu, Weixian Shi, Zhengliang Shi, Zhaochun Ren, Shuaiqiang Wang, Lingyong Yan, and Dawei Yin. cation. arXiv preprint arXiv:2501. /2432, 2025.
An efficient tool learning method via parallel tool invo-
14

---

Published as a conference paper at ICLR 2026
A APPENDIX
A.1 THE USE OF LARGE LANGUAGE MODELS
In accordance with the ICLR policy the use of large language models, hereby declarethat LLMs were employed solely to assist in improving the grammar and enhancing the expressionof this paper. The original research idea, methodological development, and overall structure andcontent of the manuscript were entirely conceived and written by the authors. At no stage was theuse of L.Mss extended to the generation of core intellectual content, and we afirm that there hasbeen misuse of LLMs in the preparation of this work.
A.2 RELATED WORK
Test-time Scaling. Recent works explore test-time scaling- idea that increasing computation
during inference can be more effective than scaling model size (Snell et al. A common strategy is sequential scaling, adopted in models like OpenAl ol (OpenAl 2024) and
2024, Wu et al. 2024).
DeepSeck RI (Guo al. 2025). Other approaches (Muennighoff et al. 2025 Yan et al.
2025
supervised fine-tuning to match a fixed compute budget. In parallel, parallel scaling (Chen et al.)
2025 Zeng et al.] [025 Pan et al., (2025a) improves throughput by distributing inference across replicas or devices, offering latency gains but introducing challenges in memory overhead.
Speculative decoding Speculative decoding Li et al. 2024a Leviathan al. 2023 Kim al.,
2023] Chen et al., (023) is an emerging technique for accelerating LLM inference, which is tra- ditionally limited by slow, sequential autoregressive sampling and memory bandwidth constraints. There are three main strategies for sampling draft tokens: token-level sumpligg(eviattin
et al.)
2023 Kim al. 2023 Chen et al. 2023). where the large model directly verifies the token outputs feature-level f eration paths using intermediate representations; and step-level sampling (Pan al.) [2025b] Yang et al., 2025), which operates at coarser granularity by validating multiple tokens or computation steps together improve throughput.
of the draft model;
sampling (Cai et al. 20244] Li et al., 2024a|b) (b. which verifies gen-
Asynchronous Tool Calling, The synchronization issue in batch inference with tool calls (Zhu al. 2025) is a known obstacle to efficient reasoning. However, remains underexplored in the when large model inference is treated as form of tool
context of speculative decoding-partieee call. In asynchronous scheduling, controlling the frequency of large model intervention challeng- ing due to synchronization overhead. Recent approaches (Ginart et al.) 2024 Gonzalez-Pumman Pumariega jet al., (2225) employ event-driven finite-state machine architectures to manage asynchronous tool calls more flexibly and efficiently.
Conformal Prediction. To avoid synchronization and to accurately predict the request budget in the scaling process, we introduce conformal prediction (Derhacobian et al.) Angeloppulos et al.) 2020 Huang et al. (023) to provide a theoretical guarantee for the budget times our target model intervenes. The prediction set is then used to ensure that the large model's interventions remain consistent with the desired coverage and reliability, aligning with the validation process. How- ever, these methods all require the model to perform complete softmax operation (which requires synchronization), and this becomes challenging in modern inference engines with asynchronous scheduling mechanisms, thus conflicting with these methods. Some online conformal prediction algorithms (Areces et al. Bhatnagar et al. 2023) attempt to ensure the coverage of future data the context of online learning.
A.3 OLYMPIADBENCH
We evaluate the results on the more challenging OlympiadBench et. 2024 dataset as evidence of the robustness of our method. Under two of our settings (results shown in bold), we were even able to surpass the performance of the original target model's sampling. Under conditional coverage, we achieved a more efficient allocation of computational resources compared to marginal coverage, resulting in improved test performance. Table 3 shows the results the OlympiadBench dataset.
15

---

Published as a conference paper at ICLR 2026
Table 3: Results on the OlympiadBench benchmark. Each setting uses 16 samples and turns, with a = 0.4, 500-token budgets, and temperature 0.8.
Draft Target Model
Draft Model
Target Model
Marginal Cov.
Conditional Cov.
DeepSeek-RI -ii---D-tttllllammaa / S1.1-32B
28
44
DeepSeek-RI Prsstill-Qwen-.... Qwen-1.5B S1.1-32B
26
48
38
38
Mamna-.1--B-Insttuue S1.1-32B
26
48
44
50
Qwen2.5- 1ren2.5-7BIInsstreett S1.1-32B
32
48
48
48
QwQ-32B
28
46
38
40
DeepSeck-RI-Dissill-ONee
QwQ-32B
26
46
38
40
Llama-3.1 3.1-8BB-nntBtnuee QwQ-32B
26
48
Qwen2.5-77 IB-Instruct/ QwQ-32B
32
46
50
46
A.4
EXPERIMENTAL SETUP
We evaluate a diverse set of draft models, including DeepSeek-RI- RI--istill - (Gu et al,
2025),
DeepSeck-R1-DistillJJmmmm -amma-BB (Guo al. (0255,, Qwen2.5-7B-Insttee (Team] 2024)
Llama-3. --88BB (Dubey et al. 2024), 81.1-7B (Muennighoff et al. 2025). and Skywork-OR! 7B (He
al.
2025 Each draft model is paired with one of the large target models: QwQ-32B Team 3-3eBB
2025
s s1.1-32B [Muennighoff et al. 2025).
(Team 2024 DeepSeek-RI-
Distill-Qwen-t
(Guo
2025), or Skywork -RInK-OOR--2D (He al. 2025). We use SpecRea-
son (Pan et al. 2025b) and Speculative Thinking YYng et al., (2225 as baselines for speculative decoding, with a maximum length 8192. Both of them focus on acceleration under serial scaling.
Our evaluation covers four reasoning benchmarks. We use 100 randomly sampled problems from MATH (Hendrycks et al., 202T) (denoted as MATH100) for grade-school arithmetic word problems, AIME24 (Art of Problem Solving) and A AIME25 (OpenCompass) for high-school competition-levee mathematics, and the first 50 problems from AMC23 (math-ai) for the American Mathematics Com- petitions. These datasets require multi-step reasoning and are particularly suitable for testing the effectiveness of asynchronous sampling with rejection. To ensure consistency, we set token budget of 8192 across all settings and adopt deterministic decoding with temperature set to zero. We set the maximum number of turns to 10.
Similar to prior work Yuue al. 2025 the hest@ 16 metric we calculate is intended to measure the upper bound of performance for both the method and the baseline model. Unless otherwise specified, the miscoverage parameter is set to statistical guarantees. We use SGLang (Zheng et al., 2024) version 0.4.3.post4 as the inference engine. The sampling temperature is set to 0.8. We set the target model's per-turn token budget to K, 500 and the draft model's per-turn token budget to K₂ 500.
0.4, ensuring that the prediction sets constructed with
A.5 LARGE-SCALE ASYNCHRONOUS TEST-SCALING
In this section, we set the maximum number of test-time scaling turns of the model 20, and then gradually increase the number of sam- ples per turn up to 128. We conduct compar- ative experiments under editing coverage, con- ditional coverage, well as under the standard rejection sampling and our designed rejection sampling settings. We set the draft model to DeasSeck-R--iisiil----eeea 1.5B, and the tar- get model to DeepSeck- RI L----itttl--aaaaee 70B.
Fauetpa
Haginel Coverge
Perfurmnte
Scating Factor
Figure 9: Accuracy improvement with increasing sample size AIME 2025.
According to Figure 9 we observe that as the number of samples increases, the perfor- mance of our model gradually improves. With 8 samples, model reaches the performance of 03-mini(low); with 64 samples, reaches the performance of m3-mini(medium); and with 128 sam- ples, it reaches the performance of miigh,, which closed-source reasoning models. Due
16

---

Published as a conference paper at ICLR 2026
Conds end Goverae TTa
Margia Coverag hicatey
10060
Time 54
2003
0.0
0.2
Alpha
(a) The change in final accuracy as increases.
(b) The increase in total time overhead as grows.
Figure 10: Ablation study on the hyperparameter a using the DeepSeek-Qwen-I.e -1.5B and QwQ-32B models on the AIME2025 dataset.
limited computational resources, we did not conduct experiments with larger-scale sampling, which results in still falling short of GPT-5 performance.
Continue Sampling. In our experimental setup, we did not strictly adhere to the standard rejection sampling procedure (i.e., discarding the samples generated by the draft model and resampling with the target model) when performing scaling. Instead, under the continue sampling setting, if a sample produced by the draft model is included the prediction of the current turn, the target model subsequently continues the sampling in the following turn conditioned on this sampled result. Ac- cording to Figure(9) both our conditional coverage and editing coverage adopt the continue- samplling scheme. The conditional coverage demonstrates relatively high sampling efficiency, reaching the level of o3-mini-medium under the sample setting.
Resampling. We also conducted experiments that scale the number of samples under the standard rejection sampling setting. In this setting. at each scaling turn, if the draft model's 's sample is included in the prediction set for the current turn, then within the same turn the target model draws a prediction set whose size matches that of the current prediction set. As indicated by the red curve in Figure (9) this scheme cxhibits substantially higher sampling efficiency than the alternatives; however, for the same nominal number of samples it consumes more tokens (since part of the draft model's tokens are discarded). Under the 12m-sa setting, it achieves performance comparable to 03-mini-high.
A.6 ABLATION STUDY
In this section, we present the ablation experiments on the hyperparameter a. As shown in Figure I0 as the hyperparameter a increases, the overall accuracy and time overhead of the system both risc. This reflects that the intervention of the target model increases both accuracy and time overhead. However, we can find a balance between accuracy and time overhead, such as when a is 0.2, where the condition coverage setting achieves 60% accuracy with relatively low time overhead. When a is 0.4, both the marginal coverage and condition coverage settings reach 60% accuracy, making it more robust hyperparameter setting.
A.7 CONTROLLING THE REJECTION RATE IN MULTI-TURN INTERACTIONS
In this section, we study the variation of model perplexity with sequential scaling and the change in the number of take-overs by the large model during rejection sampling. This directly reflects whether the conformal prediction algorithm we adopt can control the intervention rate of the target model within an acceptable range. i): According to Figure Taa each generation by the draft model leads to an increase in the overall PPL., while the intervention of the target model effectively miti- gates this trend. As the number of turns increases, the PPL of the scaling process can be gradually reduced, which indirectly results in a decrease in the rejection rate, ii): Meanwhile, in Figure [ib we directly visualize the change in the number of take-overs by the target model as the number of interaction turns increases. We observe that with sequential scaling, the target model overall main- tains a rejection rate around a fixed level, which gradually decreases and eventually approaches zero. ii):: The target model continuously adjusts the convergence behavior of the draft model's 's perplexity during sequential scaling.
17

---

Published as a conference paper at ICLR 2026
2.25
Diuf Modo Thrget Model
Oeet Court
2.30
10
W M
2.25
Take Over Count
40
1.75
20
1.50
10
2.00
(a) PPL variation with increasing turns.
(b) Take-over counts across multiple turns.
Figure 11: Analysis model behavior multi-turn interactions. The subfigure shows how Perplexity (PPL) of draft and target models evolves as the number of turns increases, while the right subfigure presents the take-over counts across turns.
A.8 DEFINITION AND PROOF
Proposition 1. Suppose that {(x;, Y:))== are exchangeable random variables from the test dataset, and & ~ Uniform 2....... ,n. dataset, where k denotes the k-th sample of that data point, then the marginal conformal p-values defined as,
represents randomly sampling one data point from the test
1
5/)
PE
(17)
is valid in the sense that for the miscoverage rate E (0, 1), we have
P(P < a) aa.
(18)
Moreover, if the conformity scores { are distinct surely, we have,
2
(19)
num + nm
Proof of Proposition 1. Suppose, for any given values of conformity scores, 01...., Unm++1 they be rearranged 01 < Ür with repetitions ny of ē; such that Σ T4: 1. Let E,
denote the event of {s} 1/,......... sm + (UI,.... )-
Then, under E., for 1,..., l, we have
P( = 8 FT))
n2:
(20)
nm + 1
due to the exchangeability of conformity scores.
We also note that under E, and s 0; we have from Equation equation 17
Ei m
PC
(21)
TiM
Then, for any E [0, 1] and i = 1,..., E, we have
so if
Σi m mm+1
Plp < a E., 2/ σz)
(22)
otherwise.
18

---

Published as a conference paper at ICLR 2026
Yi my
-im nu um++1
Thus, for any == b....,e, and
S a <
we have
am+11
P(o
Bu)
Σ
P(P a E -t)) 0z Ex)
(23)
1=1
11=
+1 a.
(24)
nm
By taking the expectation over the above inequality, follows that the conformal p-value p is marginally valid.
Specifically, if conformity scores f= U ((tt distinct surely, then l nm++1 and ni for L, L...., + 1. Thus,
i 1
if i-1 nm +1 nm + 1
F(PE S aal Ex) nm +
(25)
that is, P E, U mm+11 mm++1 } This completes the proof. Next, we address the theoretical part that guarantees conditional coverage.
Proposition 2. Suppose that {(X;, Yo) are exchangeable random variables from the test dataset, then for any sample y E V, given a conditioning set I, S {O...., m - 1} for each X; constructed based on the specific sample y for test input and YE y. the corresponding condi- tional conformal p-value as defined in Equation equation 17 is conditionally valid in the sense that for any [0, 1).
P(P S Iy, Yg = a.
(26)
Moreover, if He,, are distinct surely, we have that conditional on Iy and Yg
2
l P
(27)
i 1m+1 m
where m = [Iy] is the size of the conditioning set for output y.
Proof. For any given sample y V, the corresponding conditional conformal p-value is given by
a m +
Σ i=1
Σ 16 ++)++1 JEE
(28)
where Ig S mo 1} is the conditioning set for sample y. m = IZyl. sf represents the conformity score for the j-th candidate of the i-th test instance, and sf is the conformity score for the k-th candidate of the new test instance.
Given Z. and Y - y, the conformity scores (ot) U (sE) are exchangeably distributed, which follows from the assumption that [(X,, 1) are exchangeably distributed and the con- struction of candidate outputs.
Using similar arguments as in the proof of Proposition 1, for any given values of conformity scores U1...., Unm++, suppose that they can be arranged as &1 < ... 02 with repetitions m, of é; such that E m, nm Let E, denote the event be {s { Unm+1
Then, given E,, Ig. and Ye = y, we have
m; P(s = ūz Eo, Yx-) nm +
(29)
for 1,...,e, l, due to exchangeability
conformity scores.
Note that given E.,. Iyp Yg y. and a ú we have from Equation equation 28
E my
3 Tem
(30)
19

---

Published as a conference paper at ICLR 2026
Thus, for any [0, and i I....,e,
0 i <
Ei=1 my
F(p S a Bvg Iy Ye = 56 Ú.}
am+1
(31)
otherwise.
Then, for any given i = I...., and تريح nm++ I a <
-
we have
nm+
P(p sa a Ev. Ty. Yg = y)
(32)
P(ok a
Ev, Iyp Ye -v = ū,)) P( "j Ev. Iy: Ye =v)
(33)
i Σj my nm + < a.
(34)
By taking expectation, it follows that p is conditionally valid given Ye This completes the proof.
Discussion. After proving the validity of individual conformal p-values, in order to obtain the rejection rate for the entire test set, that is, to ensure that the overall error rate is controlled when simultaneously testing K hypotheses, we propose the following Proposition.
exchangeable random variables, and let & Uniform{ 1,2....,h represent a randomly selected test instance, then Al based on marginal con- formal p-values all provide simultaneous coverage guarantees across the entire test dataset at level 1 a, i.e.
Proposition 3. Suppose that {(X;,Y:))
P(Vi E [l,...,m} Y, E C(X.)) 21-a.
(35)
1a. Specifically, if the conformity scores {s² Ji=lij=1 are distinct surely, then for A1, we also have,
P(vi E {L..... n] Y E C(X.)) 2 a+ (n 1)m
(36)
Proof of Proposition 3. Consider A1 based on marginal conformal p-values. Note that among the tested hypotheses Hı...., Hm, there is exactly one hypothesis Hy, to be true. Thus, the probability that all samples are correctly covered by A1 satisfies:
P(Vi E (l......]] n): Y, E C(X)))
= Paaccept Hy;) P(reject Hy:)
(37)
(38)
> P(Pe Ys <a)
(39)
2 a.
(40)
where the last inequality follows by Proposition 1.
Specifically, for A1, if the conformity scores {s are distinct surely, by Proposition we
have
P(V e (L....,n) n) Y. C(X,)) Paaccept Hy:)
(41)
Yc P(P5
5 5a)
(42)
1
(43)
(n - 1)m+ +1
+
(44)
(n 1)m +1
which gives the desired result.
20

---

Published as a conference paper at ICLR 2026
Theorem 1. Suppose that Y:) exchangeable random variables, and let
Uniform 1.2....,77 represent a randomly selected test instance, then the prediction set C(Xç) =
{ pp > a} determined by Al both satisfy
P(YE E C(X))) 2 a.
(45)
Proof. Note that the prediction set is given by C(Xe) =AI A2. Thus, by Proposition 1,
P(Y: E C(X))) 2 P(Pe 1/s > a) 2 1 1
(46)
Similarly, its prediction set is given by
C(X+)= = (y EV: : > a}.
(47)
By Proposition it is easy to check that
P(Y: E C(X))) P(pe Ye > a) 2 aa.
(48)
Specifically, if the conformity scores { m,m (mi]iml are distinct surely, we have
P(Y: E C(x+)) =1 P(PE aa) s a+ 1 (n 1)m +1
(49)
This completes the proof.
Discussion.
Now that have completed the proof of marginal coverage, we proceed prove the conditional coverage for the entire test dataset. Under the exchangeability assumption, we have:
Proposition 4. Under the same exchangeability assumption as in Proposition 2, Al based on condi- tional conformal p-values p all provide conditional coverage guarantees for the entire test dataset at level - a, i.e., any y E V,
nt: Specifically, if the conformity scores {8) jiIm,.=i.... are distinct surely, then for A1 based on r we have that for any E y and Iy (O....,. -1..
P(Vi E [L....,m] E ((())))) Y2 -y)21-0 a.
(50)
P(vi E [1....,,i) n} Y, C(X.) | Y = =y,Iy) 21 a
(51)
(n (n- III,++++
Proof. Consider Procedure 1-3 based on conditional conformal p-values. For any y E Y, given
Yg = y, the conditional coverage probability for the entire test dataset satisfies:
P(Vi E (1,...,,n) m] Y, E C(X.) Y = y) = 1 P(reject H, Yg = v)
(52)
2 1 P(P2" s a Ye = y)
(53)
2 1 - 0,,
(54)
where the inequalities follow from the definitions of Procedure 1-3 and Proposition
Specifically, for Procedure 3, if the conformity scores {5/ Jyer, :=11...... are distinct surely, then by Proposition 2, the coverage probability conditional on Iy and Y5 = satisfies:
P(Vi E n). Y, ((X,) I, Yg = y) = P'reject H, Iy. Yg = y)
(55)
1 P(P s Iy. Y/s y)
(56)
2 1 - a (n 1) +
(57)
1 a +
(58)
(n - 1)|z, +1
This completes the proof.
21

---

Published as a conference paper at ICLR 2026
Theorem 2. Under the same exchangeability assumption as in Theorem 1, the prediction set C(xg) y) based on conditional conformal p-values p satisfies
P(Y: C(x 1 v) Ye = y) 21~1 a
(59)
for any E V. Specifically, for the prediction set C(Xg y) based on p the conformity scores (ST) are distinct surely, we have
1
P(Y: C(Xs v) Y = y, S a+ (n 1) +1
(60)
for any y E y and I, S {0, .......m.. 1}.
Proof. By using Proposition 4 and similar arguments in the proof of Theorem 1, the prediction sets ( y) based on the conditional conformal p-values P all satisfy:
IP(YE E C(Xe v) Iy YE y) 1-a
(61)
for any E y.
Specifically, if the conformity scores {sr jet are distinct surely, we have:
P(Y: E C(X (v) I. Ye y) P(pp > a Igp YE = y) S 1 a + - 1n-1)IIn + +1
(62)
(63)
which gives the desired result.
A.9
PROMPT
Completion Question: Please answer the following problem using step-by-step reasoning. Please separate your reasoning steps with two newline characters ())m \(n). Please must put your final answer within (/boxe( Question: {question]
Multiple Choice Question: This is a multiple-choice question. Please answer the follow- ing problem using step-by-step reasoning. Separate each reasoning step with two new- line characters ()'n \nn) You must put your final answer within (boxed (f), such as (boxed((A]] (boxed( (B]). \.boxed([C)) (CC), (D))) No other formats are allowed.
Question: {question}
Choices: A. {choice[1]) B. {choice[2]} C. {choice[3]) D. {choice[4])
22


Reviews:
[{"review_id": "HqbK5DjFzW", "reviewer": "Reviewer_LsxV", "summary": "The authors propose ATTS (Asynchronous Test-Time Scaling), designed to alleviate the memory peak and synchronization overhead issues in speculative decoding under parallel and sequential test-time scaling settings.\n\nThe authors characterize system bottlenecks using asynchronous arithmetic intensity, explicitly incorporating synchronization time into the performance metric. The authors find that they synchronous time is the main bottleneck. Therefore, ATTS adopts conformal prediction: it computes a p-value for each candidate and compares it with a threshold α to decide acceptance or rejection, instead of using global softmax and ranking which needs synchronous. This provides statistical control over the target model intervention rate without requiring distributional assumptions.\n\nBuilding on this principle, ATTS implements a three-stage rejection sampling pipeline that scales along both the parallel and sequential axes. It further employs online calibration and ordered decision-making to avoid global synchronization.\n\nExperiments on MATH, AMC23, AIME24/25, and various draft/target model combinations show that ATTS achieves up to 56.7× end-to-end acceleration and 4.14× throughput improvement without sacrificing accuracy, while significantly reducing peak memory usage and latency. Notably, on the AIME benchmark, a 1.5B draft / 70B target setup reaches performance close to that of strong closed-source reasoning models.\n\nThe main limitations lie in the sensitivity to online calibration quality and the choice of α, the lack of guarantees for global optimality in each round, and the evaluation focus on mathematical reasoning and specific inference engine configurations.", "questions": "How can online calibration maintain long-term stability of the rejection rate under non-stationary workloads?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. This paper addresses a highly important and practical problem in inference infrastructure, which has significant implications for optimizing the efficiency of model deployment.\n2. The authors introduced a new performance metric to analyze specific bottlenecks, which guided the subsequent algorithm design.\n3. The paper is well-structured and easy to understand.\n4. The proposed method achieves a significant improvement in efficiency while maintaining stable performance.", "weaknesses": "1. The proposed method heavily relies on the stability of online calibration. The rejection rate control and accuracy are quite sensitive to the calibration distribution, the number of parallel samples, and the value of α.\n2. The proposed method does not aim for globally optimal candidates and cannot guarantee obtaining the top-1 candidate in each round, which may be suboptimal for applications that strictly require optimal ranking.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UOgwHeUIyp", "reviewer": "Reviewer_KbTN", "summary": "This paper proposes ATTS, an asynchronous test-time scaling framework for large language models (LLMs) that addresses high inference latency and memory bottlenecks in test-time scaling. By redefining asynchronous arithmetic intensity to identify synchronization as the key bottleneck, integrating online calibration with conformal prediction, and designing a three-stage rejection sampling pipeline, ATTS enables both sequential and parallel scaling. Experiments on MATH, AMC23, AIME24/25 datasets show up to 56.7x speedup, 4.14x throughput improvement, and no accuracy loss—even allowing 1.5B/70B draft/target models to match o3-mini (high) performance on AIME.", "questions": "See weakness", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The introduction of \"asynchronous arithmetic intensity\" effectively quantifies synchronization overhead, a critical but understudied issue in test-time scaling.\n2. Leveraging conformal prediction for ranking and rejection sampling ensures controlled rejection rates and coverage (marginal/conditional), adding theoretical rigor absent in many efficiency-focused works.\n3. Significant speedup and throughput gains across model families (e.g., Qwen, DeepSeek, Llama) and challenging math benchmarks demonstrate real-world utility.", "weaknesses": "1. ATTS frames sequential scaling (increasing turns) as a strength, but experiments only test up to 10 turns. The paper fails to analyze when sequential scaling stops adding value—e.g., whether 20+ turns lead to accuracy plateaus while inflating token costs. \n2. The evaluation on more challnaging benchmarks would make the results more robust\n3. While conformal prediction enables statistical guarantees for rejection sampling, the paper overlooks a critical tension: its p-value calculation (Eq.13) requires comparing test samples to the full calibration set. For large batch sizes (128+), this comparison could introduce hidden per-sample latency—undermining ATTS’s core goal of reducing overhead.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tZ5UXbkVkq", "reviewer": "Reviewer_ct8q", "summary": "This paper proposes ATTS (Asynchronous Test-Time Scaling) to address the high latency problem in large model inference. The authors introduce the concept of asynchronous arithmetic intensity to measure the ratio between the time spent sampling candidates and the time waiting for the large model to verify them, showing that as the number of samples increases, the cost from waiting for verification becomes more significant. The method leverages ideas from conformal prediction to reduce the candidate set that requires verification by the large model, thereby lowering compute costs. The resulting ATTS framework applies conformal prediction for theoretically lossless speedup of test-time inference.", "questions": "1.Validity of theoretical assumptions\nThe omission of normalization in the implementation may break the core assumptions of conformal prediction, potentially leading to excessive rejections for some tasks and too few for others in mixed or real-world scenarios. How does the method ensure fairness and validity of the rejection rate when handling multi-task or large-scale deployments?\n\n2.Lack of comparison with strong baselines\nThe experimental section does not compare ATTS with other competitive methods such as SpecReason or Speculative Thinking. What are the practical or theoretical advantages of ATTS over these methods, and is there any empirical evidence of its superiority or unique use cases?\n\n3.System/engineering considerations\nThe paper does not discuss the impact of modern inference engine features, such as supporting asynchronous verification requests. Can ATTS be integrated with inference backends that support asynchronous calls, and is there any experimental evidence for further gains from such integration?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper provides valuable insights into how to reduce the time cost of test-time scaling (TTS), especially through the lens of asynchronous arithmetic intensity.\n\nEffectively applies conformal prediction to minimize the workload of the large model during inference, and demonstrates both theoretical and practical speedup.\n\nThe experimental section covers several representative math reasoning tasks, and the results are fairly comprehensive for the considered setting.", "weaknesses": "The method is not empirically compared against other prominent recent approaches in the field, such as SpecReason and Speculative Thinking, making it difficult to objectively evaluate the unique advantages of ATTS.\n\nExperiments suggest that when the small and large models are trained on data from different distributions, the benefits of ATTS decrease noticeably, highlighting a potential issue: the method does not fundamentally address the extra rejection cost caused by distribution mismatch between the small and large models.\n\nTheoretical guarantees rely on key assumptions from conformal prediction (e.g., exchangeability and normalization of scores). However, the implementation omits score normalization, which may break these assumptions and cause the rejection rate control to fail, especially in multi-task or distribution-shifted settings.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)