Best-of-Infinity: Asymptotic Performance of Test-Time LLM Ensembling
Authors: Junpei Komiyama
Daisuke Oba
Masafumi Oyamada
Venue: ICLR 2026 Poster (2026)
Keywords: LLM
test-time compute
majority voting
LLM ensemble
Primary area: foundation or frontier models, including LLMs

Abstract: We study best-of-$N$ for large language models (LLMs) where the selection is based on majority voting. In particular, we analyze the limit $N \to \infty$, which we denote as best-of-$\infty$. While this approach achieves impressive performance in the limit, it requires an infinite test-time budget. To address this, we propose an adaptive generation scheme that selects $N$ based on answer agreement, thereby efficiently allocating inference-time computation. Beyond adaptivity, we extend the framework to weighted ensembles of multiple LLMs, showing that such mixtures can outperform any individual model. The optimal ensemble weighting is formulated and efficiently computed as a mixed-integer linear program. Extensive experiments demonstrate the effectiveness of our approach. Our code is available at https://github.com/jkomiyama/BoInf-code-publish/.

Published as a conference paper at ICLR 2026
BEST-OF-oO ASYMPTOTIC PERFORMANCE OF TEST- TIME LLM ENSEMBLING
Junpei Komiyama
Daisuke Oba
Mohamed bin Zayed University of Artificial Intelligence New York University / RIKEN AIP
Institute of Science Tokyo da i suke. cba@nlp.come ssct. ac. jp
junpe Canpei@komi @komi yama. info
Masafumi Oyamada
NEC Corporation
st i 1 @cillppeantt@meeee i 1. ccom
ABSTRACT
We study best-of-N for large language models (LLMs) where the selection is based on majority voting. In particular, we analyze the limit N - co, which we denote as best-of-oo. While this approach achieves impressive performance the limit, it requires infinite test-time budget. To address this, propose adaptive generation scheme that selects N based on answer agreement, thereby efficiently allocating inference-time computation. Beyond adaptivity, extend the framework to weighted ensembles of multiple L.MMs, showing that such mix- tures outperform any individual model. The optimal ensemble weighting is formulated and efficiently computed as mixed-integer linear program. Extensive experiments demonstrate the effectiveness of our approach. Our code is available at /tntts::/githien com/ jkomiy yama/ Bo InInf- code code-publl lilbiish.
INTRODUCTION
The last few years have witnessed remarkable advancements in large language models (LLMs), in their industrial successes including closed models such as Gemini (Gemini Team, 2025), GPT (OpenAI, 2023), and Claude (Anthropic, 2025) as well as open-weight models such as Llama (Ilama Team, 2024). Deepseek (DeepSeek-Al, 2025), Qwen (Qwen Team, 2025; Ye et al., 2025; Cheng et al., 2025) and many others including Liu et al. (2023); Almazrouei et al. (2023); Gao et al. (2023); Jiang al. (2023a); Biderman et al. (2023); BigScience Workshop (2023); OpenAl (2025); Wang et al. (2025b); NVIDIA (2025); Abdin et al. (2025); Ji et al. (2025); LG AI Research (2025). One of the largest interests in the realm of LLMs is on their ability perform complex reasoning tasks. A breakthrough in the reasoning of LLMS was the introduction of chain-of-thought prompting (Wei al., 2022; Kojima al., 2023), which allows models generate intermediate reasoning steps before arriving at an answer. Instruction-tune -uuned LLMs optimized to generate longer chains of thought have drastically increased performance in these tasks (Muennighoff al., 2025).
Spending more computational resources at test time, in particular by generating multiple answers, leads more reliable inference (Snell et al., 2025; Brown et al., 2024). A simple yet effective strategy is the best-of-N (BoN) approach, where we generate N answers and select the best one based on some criteria. There are several ways to implement the BoN strategy. One common approach is to use reward model to select the best answer (Uesato et al., 2022; Rafailov et al., 2023; Wan et al., 2024; Dong et al., 2024; Liu et al., 2024; Wang et al., 2024; Wu et al., 2025) or asking LLM to choose preferable answer (Mahan et al., 2024; Son et al., 2024; Guo et al., 2025; Chen et al., 2025a). Another approach is majority voting (Wang et al., 2023) in which the most frequent answer is selected.
Despite its simplicity, majority voting has several advantages. First, it does not require any additional modeling further text generation. Second, compared with other methods, majority voting is robust to reward hacking and benefits from additional generations with minimal risk, unlike reward-base based models where increasing N' can lead to overfitting (Huang et al., 2025). Third, for reasoning tasks,

---

Published as a conference paper at ICLR 2026
majority vote is reported to be very effective (Chen et al., 2024a). Across datasets, with few exceptions (Chen et al., 2024b), majority voting performance generally increases with (Figure I).
While we desire achieve such Best-of-N performance of N + co, which we call best-of-co per- formance, it requires an infinite number of generations (samples), which is infeasible in real-world scenarios. Yet, for the same test-time budget, As shown in Figure 2, we can generate samples adaptively until we determine the majority with some confidence level. We introduce a principled method to determine when to stop generating answers and when to continue using Bayesian modeling (Section 2).
utilize the available budget more effectively.
Our scheme can be naturally extended to ensembles of multiple LLMss. Importantly, ensemble ma- jority voting can naturally benefit from complementarity. For example, in the AIME2025 dataset, the best-of-co performance GPT-OSS-20B (OpenAI, 2025) and Wemotron-Nano-P900 2025) are 90.0% and 73.0%. respectively, but their ensemble achieves 93.3%. A weak LLM can contribute the ensemble if it has complementary strengths.
(NVIDIA,
A key theoretical contribution of this work is the formulation of optimal ensemble weighting tractable optimization problem. We show that finding the optimal weight vector that maximizes best-of-co accuracy can be reduced to a mixed-integer linear program (MILP) (Section 3). This formulation enabled by considering the asymptotic limit: while optimizing weights for finite N requires enumerating an exponentially large number of answer combinations, the best-of-oo frame- work yields a polytope structure that allows efficient optimization via standard MILP solvers. To our knowledge, this is the first work to provide a computationally tractable method for finding provably optimal ensemble weights in the context LLM majority voting.
Finally, we evaluate the performance of the proposed method (Section 4). Our experimental results include 11 instruction-tuned L.MS and four heavy-reasoning problem sets (AIME2024, AIME2025, GPOA-DIAMOND, MATH500), with least 80 generations for each L.MM-prbllem set com- bination. This represents significantly larger scale of test-time computation than prior work. We demonstrate that the MILP-optimized ensemble weights consistently outperform both uniform weighting and single-model selection across all benchmarks. We release our generation results for subsequent research. Related work is discussed in Appendix
BEST-OF-SC IN FINITE SAMPLES
While Best-of-oc defines an idealized best-of-N ensemble in the limit N + co, its literal realization would require unbounded test- tist compute. We now develop anite-sample procedure that closely tracks this limit. Our core idea is to adaptively samples (i.e., ask LLM to generate the answers) until we are sure the population majority vote with a desired confidence level. other words, we aim to terminate the answer generation process soon as sufficient statistical evidence has been obtained to support the conclusion that the currently most frequent response corresponds to the true majority, which allows different number of N across problems. A distinctive challenge of this problem lies in the fact that the support of the answer distribution generated by large language models (LLMs) is
AIME2024
AIME2025
GPOA-DIAMOND
MATH500
0.95
0.90
0.90
0.725
0.90
0.85
0.700
0.96
Accuracy
0.80
0.675
0.80
0.75
0.650
0.625
0.92
0.75
0.70
0.600
10a
10
10a
10'
10a
of gens 1- N, log scale!
of gens (= N. log scale)
# of gens (- N, log scale)
of gens (= N, log scale)
Figure 1: Accuracy of ofst-of-N with majority voting as a function of N (GPT-OSS-20B (Medium)) with four datasets (Maxwell-Jia, 2024: OpenCompass, 2025; Rein et al., 2023: Hendrycks et al., 2021). Green line indicates the asymptotic accuracy of N - oc. For each problem, BoN benefits from increasing N, least from N 10' to 102.
2

---

Published as a conference paper at ICLR 2026
Frequency
0
(A)
Terminate
(B)
(C)
(D)
Frequency
0.0
00
0_00
0
(C)
(A)
(C)
(D)
(C)
Terminate
Figure 2: An illustration of adaptive sampling (Algorithm 1). The histogram shows the distribution of answers generated by LLM for single problem. Each answer generation be viewed as a sample from the underlying distribution. Blue indicates the most frequent answer, and orange indicates the others. In the top example. three generations agrec, so sampling stops. In the bottom example, more samples are needed to determine the majority. This maximizes the accuracy under given compute budget. Confidence in the majority is based the Bayes factor.
Algorithm Approximated Best-of-oo: Determining answer for single problem
Require: Maximum samples Nuuxx concentration parameter a, Bayes factor threshold B.
1: for 1, 1,2,... do
2: if we use LLM Ensemble (Section 3) then
3: Choose LLM with probability {w: twihex.
4: end if
5: Ask the L.MM for the answer of the problem to obtain answer.
6: if Nmax or BF 2 B then
7: break
8: end if
9:
end for
10: return The most frequent answer.
unknown. For instance, in one case LLM may produce candidate answers, such 42 with probability 70% and 105 with probability 30%, whereas in another case it may yield four distinct outputs, such 111 with probability 40%, 1 with probability 25%. 2 with probability 20%, and 702 with probability 15%. Given such uncertainty in the variation of generated responses, a particularly well-suited approach is employ nonparametric Bayesian modeling. In particular, Dirichlet process DP H. a) prior over the answer space that captures the unknown distribution of answers. Here, H base distribution' over the answer space, and a > is a concentration parameter that controls the likelihood of generating new answers. Intuitively speaking, is the strength of prior belief in existence of new answers. Assume that, at round n, we observe s(n) different answer A1, A1,A..... A A.(n) with corresponding counts N₁ N2 Ng... Na(n)- Then, the posterior distribution is
adopt
DP a+n
H
Σ
N
a n
buse distribution
empirical distribution
'The base distribution can have possibly infinite support, such as GPQA, the answer is given a finite domain (e.g., A. B, C, D), and thus the base distribution is of finite support. In such cases, Dirichlet process is exactly the same as the Dirichlet distribution. The advantage of the Dirichlet process is to unify the treatment for both finite and infinite answer spaces, well having reguralization with hyperparameter a.
as all possible integers. For some tasks,
3

---

Published as a conference paper at ICLR 2026
The first argument of the posterior above states that the posterior is increasingly concentrated around the observed answers as more data is collected.
We use the Bayes factor (Jeffreys, 1935; Good, 1967; Kass & Raftery, 1995; Lindon & Malck, 2022) to measure the evidence true majority.'
Formally, define the hypotheses as follows:
Ho : The most frequent answer A₁ is not the true majority.
(2)
H1 The most frequent answer Al is the true majority.
(3)
and define the Bayes factor (BF), which quantifies the strength of evidence in the data for H1, as (4)
BF P(D(n)) Ho)
where D(n) is the observed data so far. Here, P(D(n)) ((D())))))) are the evidence (marginal likelihood) based on the observed data. Then, the Bayes factor equation 4 can be computed as follows:
BF()) ((D(m) H1) ((H)) P(Ho)
P(D (1) Ho) P(H) P(H)) (Bayes' theorem)
(5)
s (1)) P( (n)))
(approximating the prior ratio by uniform prior)
(6)
((Ho)D(()) (D))
P(H, (D(n)
s((2) 1- (D())
(Ho H is the entire space)
(7)
where (((H((((n))) (Ho))) (HHo))D(r)) (D())) are the corresponding posteriors. Note that, in the second line, we approximated the DP prior with uniform prior the existing answers.
When n is sufficiently large compared with a, ((H))D(n)) of the DP posterior can approximated by Dirichlet distribution as:
(D = Pr[X, 2 max X,, X Marichlet(N1,,W) Ne,..., a)),
i
by approximating the probability of Aı appearing in the base distribution H to be zero. The Dirichlet distribution is a conjugate distribution of the categorical distribution of s(n) + 1 of answers, where the last dimension corresponds to the unobserved answers. Here, the final component of weight is added to account for the base distribution H. While this quantity is not trivial to compute, it can be estimated using Monte Carlo methods by sampling from the Dirichlet distribution.
The following theorem states that, if we set Nmax and sufficiently large, the performance of Algorithm converges to the best-of-xo performance. The proof is given in Appendix C.
Theorem 1. (Consistency) Assume that the LLM generates a finite number of answers 1, ....... For ease of discussion, let P5 be the probability of answer and assume that p1 > P2 Ps 2
2 Ps > O. Namely, there are no ties for the most frequent answer, and each answer is generated with non-zero probability. Then, Nmax, B cc, the algorithm'ss 's performance converges the best-of-cx performance almost surely. Namely, the algorithm returns the true majority answer with probability 1.
3 LLM ENSEMBLE
Algorithm is naturally extended to use more than one LLM. Let i E K. index the LLMs, and let (w), W.,..., ., wx) be the weight vector, where w; 2 and Σiex W, Algorithm with an LLM ensemble proceeds as follows: for each generation, we first select an LLM with probability wi and then ask the selected LLM for the answer.
Let us consider the optimal weighting scheme for the BoN inference. Let 9 E Q be the problem. Each problem associated with answer domain Ag-
Bayes factor for categorical data is not new, Unlike their case, our case starts from an unknown number of categories, which is handled by the Dirichlet process prior and via some approximation. Aggarwal et al. (2023) applies Dirichlet distribution majority voting in the context of LLM consistency, and approximated the posterior prohability with Beta distribution on top-two majority answers. Wang et al. (2025a) applies frequentist confidence interval for adaptive stopping.
"The use
4

---

Published as a conference paper at ICLR 2026
Example (AIME2025). For AIME2025, Ag S 10,1, .9.......9..... JJ, where U denotes either out-of-range integer, fractional number, or a failure to emit a final answer; is always incorrect.
For each problem, 9g E Ag be the gold answer. Each LLMM-roblle pair (4, q) the probability distribution Pig over Ag. For each problem take a majority vote to produce ag. The total number correct answers
obtaín multiple generations from the LLMs and
f(ta))) = Σ 1a = -ol-
QEQ
We aim maximize it in expectation: B[f({a,))] Here, the expectation taken over the random- ness in the generation of LLMs.
3.1
BEST-OF-ONE
Before going into Best-of-co, we first consider the best-of-one (Bol) policy, which first selects an LLM with probability proportional to w, and then uses the LLM generate single answer. immediate observation is that the optimal weight is to put all the weight on the best LLM.
Lemma 1. (Optimal Bol) The accuracy of Eq. equation 9 is maximized when we choose Wi and W 0 for all + i*. where wi- the weight for the best LLM it Namely, let p (P,,1-P.22 Pr.2........... .P..........PIAAA) E AA, be the probability distribution on Aq of the answers that LLM i generates. Then, Pigg be the probability that LLM generates the gold answer 94 for problem q. The average accuracy of LLM i is S3 and the best LLM, which maximizes this quantity, is i* arg max,ex Eg P..."
Proof. It is easy see that
f((ag)) = =E wi
Σ IE
pi
I max
Σ pi.g. V
For Bol, the optimal weight is put all the weight the best LLM. However, this longer the case for BoN with N > 1. Put differently, under multi-generation majority voting, appropriately mixing non-optimal LLMs can be beneficial.
3.2 BEST-OF-OX
As in the Bol setting, our design choice is to take weighted majority vote with (w (wi.-... -.r..... wx) When we consider the large-sample limit, the answer for problem is deterministic."
0g arg max
WiPi.
IEK
with ties broken arbitrarily. Consequently f(a,) is also deterministic:
f({a,)) Σ Iag 9yl-
PEQ
Here, for ease of discussion, we omit the consideration for a tie. Henceforth, since our design choice on the weight vector w, we denote (ww and use f((a,.) and f(w) interchangeably.
Our central question is how to choose a weight vector that maximizes the accuracy f(w). The following lemma implies the hardness of optimizing f(w).
Lemma 2. (Non-concavity) f (w) a non-concave function on the simplex space of w.
We the deterministic deseribe non-random quantity.

---

Published as a conference paper at ICLR 2026
Proof. Consider dataset of just question with two LLMs, where LLM correctly answer the question and the other LLM fails. Namely, f((1.0) 0)) = 1 and f((0, 1)) = O. Then the weighted combination is 0 at somewhere in between, which implies it is non-concave.
While the proof above is an extremely simple case of two LLMs with a single problem, we will demonstrate the non-coneavity in more complex cases.
optimal solution
Figure 3: Visualization of the non-concave objective function f(w) over the weight simplex The yellow simplex corresponds to w in the simplex of the weights of the three LLMs. The gray region of the five polytopes (= five problems) weight correctly answer to the problem. The optimal solution is the intersection of four polytopes at the center, which corresponds to the case where four out of five problems correctly answered.
the region where the weighted majority of the corresponding
Although non-concavin non-concavity implies sub-optimality of gradient- based methods, a combinatorial opti- mization approach can be adopted for instances of typical scale. The in optimizing f(w) is that the summand in equation takes value one within polytope.
Lemma 3. (Polytope lemma) Let hEt,eE, be the arbitrary distributions of the answers. Then, the following set, which implies that answer is the most frequent answer, is polytope:
E AK
Σ
wapl
max J'
WiP'y
(10)
Proof. The region of equation 10 is an intersection of the following half-spaces:
W Σ w.pp'
w.ply
for all j' jj. j. which polyhedron. Since the desired space is intersection of polyhedron and a simplex AA,, is finite. Therefore, is a polytope.
Lemma 3 states that the maximization on the number of correct answers is equivalent the max- imization on the number of polytopes that contain w (Figure 3). By introducing auxiliary variable Y, that indicates the correctness for each answer, this can be formulated as mixed-integer linear programming (MILP) problem.
Lemma 4. (MILP formulation) The equation 9 is equivalent to the following MILP problem:
max WEA (0.101)
Σ U4
(11)
s.
w: 2 4A,
(12)
Σ w; =
(13)
Ayw Z m(1 3q) Vq
(14)
where Ay is matrix of size BlAAK such that its j, entry is pi.g. Pl.g- and the j-th row corre- sponds the fact that the total weight of the gold answer 9q is larger than that of wrong answer

---

Published as a conference paper at ICLR 2026
Table 1: Statisties of the large-scale generation dataset that we used in our experiments.
Each file corresponds to a single answer. We release it at eal @al0a6bd766bcf4ee000
com/ s/
LLM
of files
total generated tokens
total file size (MB)
AM-Thinking-1
4,800
79,438,111
185.95
Datarus-RI- -------I------reiee
4,800
49,968,613
127.03
EXAONE-Decs -2e
60,640
478,575,594
1,372.35
GPT-OSS-20B
68,605
244,985 253
98.594
LIMO-v2
6,095
77,460,50
219.45
MetaStone-S1-32B
60,757
806,737 ,009
2,458.48
NVIDIA--NemotronNName
60,640
295,466 466,626
897.82
Phi-4-reasoning
168, 138,
558, 358,980,037
1,841.06
Qwen3-4B
20,640
547.170,887
1,704.28
Qwen3-14B
44,800
666,466,780
1,822.13
B-BB--30BAA3--- 2507
60,640
436,865,22
1,234.28
j. The vector m > is chosen sufficiently large, so that Agw 2 m is never satisfied when Agw has negative component.
The size of the problem instance depends on the number of LLMs K, the number of problems N, and the size of the possible set of answers Ag. General MILP solving is NP-hard; in practice, however, open-source solvers scale smoothly to K 10 LLMs and N 103 problems, where typical size of Ag is s 10a.
Max margin solutions As we illustrated in Figure 3, the objective function f(w) has continuous region of optimal solutions. While any interior point on these position is optimal in best-of-co, its finite-N performance can vary. In this paper, we adopt "max margin" solution, that is at the most interior of the solution. Namely, we introduce margin >> and replaces A, in equation A, ş. We choose the supremum of the margin such that the objective value E, Mi does not decrease, and adopts the solution on such margin. The optimization of margin can be done binary search on the space of & E [0, m] where m is sufficiently large constant. This binary search problem of monotone objective, which is practically feasible.
with
EXPERIMENTS
This section reports our experimental results. We considered heavy-reasonnng tasks on open-weight LLMs that we can test on our local environment. We set Algorithm I's hyperparameter = 0.3 for all the experiments. To solve MILPs, we use highspy, an open-source Python interface to the HiGHS optimization suite (Huangfu & Hall, 2018), which provides state-of-the-art solvers for large-scale LP, MIP, and MILP. We adopt the max-margin solution described in Section 3.2. Unless specified otherwise, all results are estimated from 100 independent runs. We use the Dirichlet posterior with each count (t) + rather than Nt( to have faster stopping. The Bayes factor is calculated with 1,000 Monte Carlo samples from the posterior. Due to page limits, show only several experimental results the main text. More results are available in Appendix G.
4.1 TESTED OPEN-WEIGIIT LLMS AND DATASETS
We evaluate open-weight LLMs (< 32B parameters) across four reasoning benchmarks. We the following problem sets: AIME2024 (Maxwell-Jia, 2024), AIME2025 (OpenCompass, 2025). GPQA-DIAMOND (Graduate-Level Google-Proof Q&A Benchmark; Rein et al. 2023), and MATH500 (Hendrycks et al., 2021). Details the LLMs and datasets are provided in Appendix D. These datasets are challenging mathematical and scientific reasoning tasks. We did not test GSM8K (Cobbe et al., 2021) as is too easy for the L.LMs we tested.
WWe do not use chain-of-thoune thought (CoT) in our experiments and thus the file size is small; however, we also include an updated dataset that contains CoT.
7

---

Published as a conference paper at ICLR 2026
Large-scale generation dataset We generate set of candidate answers by querying the LLM with the problem statement. For each pair of (LLM, problem), we generate at least 80 answers an order of magnitude greater than the typical 8 generations reported in most LLM technical reports. We believe the difficulty of the problems as well as the scale of generated tokens are significantly larger than existing work test-time computing. Table 1 shows the statistics of the datasets used in our experiments. Base performance (Bol, best-of-oc) of these LLMs are shown in Appendix E. Every sample of answer in our subsequent experiments is drawn from this dataset. Best-of-co performance is also estimated from these samples. We remove the unparseable answers, which benefits some of the LLMs with lower performance.
4.2
EXPERIMENTAL RESULTS
Acaptiva
0.900
0.960
Fieed
0.933
0.955
Accuracy
0.950
Accuracy
0.950
0.945
0.945
0.940
0.940
0.935
0.935
0.930
0.930
10a
10 Average # Gens (log scale)
10
10
Average Total Tokens (lag scale)
Figure 4: Cost-analysis of our proposed method and fixed BoN. GPT-OSS-20B on MATH500. "Adaptive" Algorithm with average sample size of N 3 achieves the same accuracy as "fixed" sample of N = 10, and the algorithm with average sample size ze 10 achieves the same accuracy fixed N 100. Thus, the adaptive sampling in this plot reduced the computation times by 2x-5x order. Both approach the best-of-ox performance (green dashed line).
Experimental Set Effectiveness of adaptive sampling First, we investigate the impact of adaptive sampling scheme of Algorithm 1 on the performance of majority voting. We set N = 100 and tested varying Bayes factor {2, 3, 5, 7, 10, 30, 100, 300, Figure (left) compares the performance of Algorithm with fixed budget of samples (BoN), where r-axis is the number of average samples per problem (log-scale), and y-axis is the accuracy. The figure clearly shows that the blue curve (Algorithm 1) achieves the same accuracy as the red curve (fixed BoN) with substantially fewer samples. Figure (right) shows the average total number of tokens as a function of accuracy. The adaptive method again demonstrates significant reduction in token usage to achieve the same accuracy level compared to the fixed method, although the gap is smaller than that of the sample count. This is because the adaptive method tends stop sampling early for easier problems, which often require fewer tokens per generation.
Experimental Set 2: Advantage L.LM ensemble over single L.M Second, we investigate the advantage of LLM ensemble over single LLM. We compare the performance of the single LLM with the optimal mixture LLMs. The results in Figure show that the ensemble method achieves higher accuracy than any single LLM, demonstrating the effectiveness of combining multiple models.
Experimental Set 3: Learning good weight Third, we investigate the generalization ability of our weight optimization method (Section 3). Figure shows the performance the learned weights function of the number of training problems on AIME2025. With five training problems, the learned weights approach the best single-LLM performance.
Experimental Set Transfer learning of the optimal weight assess transferability, we trained weights AIMF2002 and tested on AIME2025; across 165 three-model combinations, the ensemble matched or exceeded the strongest individual model in 106 cases (64.2%).
Also note that, for adaptive sampling scheme, around 80 samples are usually sufficient to achieve accuracy fairly close the best-of-co performance.
8

---

Published as a conference paper at ICLR 2026
Ensemblc Adaptive
0.775
0.775
gpl-oss-200 Adaptive
MetaStone Adaptive
0.750
0.750
Adaplive
Qwen3-300--4 30B-A3BT Asaptive
Accuracy
0.725
Accuracy
0.725
Phié-8, Adaptive
0.700
0.700
0.630
0.625
0.625
0.600
0.600
10a
10
10a
10*
10a
10e
Average # of Gens (log scale)
Average Total Tokens (log scale)
Figure 5: Performance comparison of the LLM ensemble of EXAONE-Dec DEAEEE--eep--222 12cpp, MetaStone-SI- 32B, Phi-4-reasoning,
and GPT-OSS-200 on GPQA-Diamond. The weight is optimized to = (0.0176, 0.0346, 0.2690, 0.4145, 0.2644). The LLM ensemble outper- forms any single LLM with N 2 5 and approaches the blue dashed line of best-of-co performance.
AM-Thinking-v11: 0.867
Ensemble Accuracy
0.9
Datarus-r1: 0.600
EXAONE-D: 0.767
LIMO-v2 0.700
0.8
MetaStone: 0.800
Nemotron-N98: 0. 0.733
0.7
Phi4-R: 0.833
Qwen3-144 3-n:: 0.800
Qwen3-30B-Ae 0.900
0.6
Qwen3-4B: 0.733
gpt-oss-20b: 0.900
5
10
15
20
25
30
Ensemble
Number of Training Samples
Figure 6: The number of samples to determine the weight (x-axis) as performance of best-of-co (y-axis) on AIME2025. The r-axis indicates the number of problems used to learn the weight and the y-axis indicates the best-of-oo performance with all problems. The score is averaged over 100 The optimal weight has achieved the limit accuracy of 93.3%, whereas the best single LLM has the limit accuracy of 90.0%. Dashed lines indicate the best-of-co performance of each LLM.
Experimental Set 5: Comparison with other answer-selection methods We finally compared the majority voting scheme with other selection scheme in the best-of-five (Bo5) test-time inference (Table 2). AIME2025, majority voting outperforms random selection, self-certainty, reward models, and LLM au-uuggee,- full tables and settings provided in the appendix (Appendix G.5).
CONCLUSION
In this paper, we view the majority voting as sampling from the underlying answer distribution, Its limit-N performance naturally defined. To approximate this limit with a finite number of samples, introduce adaptive sampling method based on the Bayes factor. We also study the problem of aggregating responses from multiple L.MSs and propose a majority voting that effectively leverages the strengths of individual models. The limit performance for large N has advantage because the weights of LLM ensembles can be optimized by solving mixed-integer linear programming prob- lem. Through extensive experiments
challenging reasoning tasks, we have verified robustness across LLMs and problem sets. The scale of our generated answers is substantially larger than in existing work, and we release these generations for future research.
ACKNOWLEDGMENTS
We thank Naoto Twase for providing the guidance the MedRECT dataset.

---

Published as a conference paper at ICLR 2026
The accuracy of several selection methods on the best-of-five (Bo5) setting on the AIME2025 dataset. Answers are generated by GPT-OSS-20B. The scores are averaged over 16 trials and we report the two-sigma confidence intervals. Omniscient is hypothetical upper bound that always selects the correct answer if it is present in the candidate answers, which requires the gold answer. Random, which selects one of N answers uniformly at random, should match the per- formance of Bol. Details of each method are described in Appendix G.5.
Table 2:
Method
Mean + CI
Omniscient
91.04 + 1.32
Majority voting
85.42 + 2.01
LLM-as-aajjudge (tournament)
82.92 + 2.57
LLMM-as-a-judge (set)
81.25 + 2.42
INF-ORM-LLlmaa3.1725
79.79 2.54
Skywork-Reward-V22 --a---e-rrrd-----aaa---- 1-8BBBBBBBBBBBBBB
79.79 2.47
-kywottRewwrrd-2---wellll
80.00 2.51
Self-certainty
75.83 2.47
Random
76.25 2.71
ETHICS STATEMENT
We acknowledge the potential ethical concerns surrounding the use of large language models (LLMs) in our research. Our work aims improve the performance of LLMs in reasoning tasks, which may have implications for their deployment in real-world applications. Since we do not ask for LLM work on any sensitive or harmful tasks, we believe that our work does not directly contribute to the generation of harmful content. We are committed to ensuring that our research is conducted responsibly and that the benefits of our work are shared broadly.
REPRODUCIBILITY STATEMENT
We release the code and the generated answers used our experiments to facilitate reproducibility. Detailed descriptions of the datasets, models, and experimental setups provided in the Appendix.
REFERENCES
Marah Abdin, Sahaj Agarwal, Ahmed Awadallah, Vidhisha Balachandran, Harkirat Behl, Lingjiao Chen, Gustavo de Rosa, Suriya Gunasekar, Mojan Javaheripi, Neel Joshi, Piero Kauffmann, Yash Lara, Caio César Teodoro Mendes, Arindam Mitra, Besmira Nushi, Dimitris Papailiopoulos, Olli Saarikivi, Shital Shah, Vaishnavi Shrivastava, Vibhav Vineet, Yue Wu, Safoora Yousefi, and Guoqing Zheng. Phi-4-reasoning technical report, 2025. URL 2504.213 21318.
abs/
Pranjal Aggarwal, Aman Madaan, Yiming Yang, and Mausam. Let's sample step by step: Adaptive- consistency for efficient reasoning and coding with LLMs. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Lan- guage Processing, pp. 12375-12396 Singapore, December 2023. Association for Computational Linguistics. doi: 10. 1e..18653/112222.coo 2023. comalp-mmiin 761/
761. URL http://:/aaaaathhoooooeo org/
Ebtesam Almazrouei, Hamza Alobeidli, Abdulla Alshamsi, Alessandro Cappelli, Ruxandra Cojo- caru, Sultan Alneyadi, Matteo Maggioni, and The falcon series of open language models. arXiv preprint arXiv:2311 1..66737....677, 2023. URL https::pp//.......... / /eteii///ivvvva/ org/ abs 2311 1. 16867.
Anthropic. Claude opus 4 & claude sonnet 4 system card. System card, May 2025, 2025. Includes model descriptions, safety testing, hybrid reasoning modes.
Stella Biderman, Hailey Schoelkopf, Quentin Anthony, Herbie Bradley, Kyle O'Brien, Eric Hal- lahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, Aviya
10

---

Published as a conference paper at ICLR 2026
Skowron, Lintang Sutawika, and Oskar van der Wal. Pythia: A suite for analyzing large language models across training and scaling, 2023. URL https:
org/ abs ab///3/a..///....//// 2304. 01373.
BigScience Workshop. Bloom: A 176b paramete open-access multilingual language model, 2023.
URL https: //axxiivvv//./////////a/////aava org/ abs / 2211.221111.1 05100.
Bradley C. A. Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoe V. Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with re- peated sampling. CoRR, abs/2407.21787, 2024. doi: 10. 10-48550AAXXIV22472227773 URL
https: /tw.//doii//////////// wrggii.rgl..... 48550/ arxiv. 2407 21787.
Lingjiao Chen, Jared Davis, Boris Hanin, Peter Bailis, Ion Stoica, Matei Zaharia, and James Zou. Are more Im calls all you need? towards the scaling properties of compound ai systems. In Proceedings of the 38th International Conference on Neural Information Processing Systems, NIPS '24, Red Hook, USA, 2024a. Curran Associates Inc. ISBN 9798331314385.
Lingjiao Chen, Jared Quiney Davis, Boris Hanin, Peter Bailis, lon Stoica, Matei Zaharia, and James Zou. Are more LLM calls you need? towards the scaling properties of compound Al systems. In The Thirty-eighth Annual Conference Neural Information Processing Systems, 2024b. URL. net / forum?? d=m51 C6RRLgX.
/ttp://ppeeeeeeeeeeeeeee https:
Xiusi Chen, Gaotang Li, Ziqi Wang, Bowen Jin, Cheng Qian, Yu Wang, Hongru Wang, Yu Zhang, Denghui Zhang. Tong Zhang, Hanghang Tong, and Heng Ji. Rm-rl: Reward modeling as reason- ing, 2025a. URL https: /wtxxv.xivvv.///////ea//aaaaa org/ abs 2 2505. 02387.
Zhijun Chen, Jingzheng Li, Pengpeng Chen, Zhuoran Li, Kai Sun, Yuankai Luo, Qianren Mao, Dingqi Yang. Hailong Sun, and Philip S. Yu. Harnessing multiple large language models: A survey on Ilm ensemble, 2025b. URL https: http///xxvvv.or/ee abs / 22002 18036.
Zhoujun Cheng, Richard Fan, Shibo Hao, Taylor W. Killian, Haonan Li, Suqi Sun, Hector Ren, Alexander Moreno, Daqian Zhang, Tianjun Zhong, Yuxin Xiong, Yuanzhe Hu, Yutao Xie, Xudong Han, Yuqi Wang, Varad Pimpalkhute, Yonghao Zhuang, Aaryamonvikram Singh, Xuezhi Liang, Anze Xie, Jianshu She, Desai Fan, Chenggian Gao, Liqun Ma, Mikhail Yurochkin, John Maggs, Xuezhe Ma, Guowei He, Zhiting Hu, Zhengzhong Liu, and Eric Xing. K2-think: A parameter-efficient reasoning system, 2025. URL https: ptttt::://axii...t// //aa/xxxvvv.rr.. org/ 07604.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint 10.1:2110.11168 2021.
A. P. Dawid and A. M. Skene. Maximum likelihood estimation of observer error-rates using the em algorithm. Applied Statistics, 28(1):20-28, 1979. URL /brokenur##http: / //links... jstor. http://icissccii13399955533 (wt.:222222229999992229999999992 119999999292211122 $3C20 3AMLE00E33E2. C.C0333---- 3B2-0.
DeepSeck-AL. Deepseck-rl: Incentivizing reasoning capability in Ilms via reinforcement learning.
2025. URL. https: //axxiv org 2201 .25.. 12948.
Hanze Dong, Wei Xiong, Bo Pang, Haoxiang Wang, Han Zhao, Yingbo Zhou, Nan Jiang, Doyen Sahoo, Caiming Xiong, and Tong Zhang. RLHF workflow: From reward modeling to online RLHF. Transactions on Machine Learning Research, 2024. ISSN 2835-8856 URL https: / //openreview. wttp:/iwwwweeeeettteeeeeeeceeeee htt./forumm/iio
YuU
Yichao Fu, Xuewei Wang, Yuandong Tian, and Jiawei Zhao. Deep think with confidence, 2025. URL wtttttttss///ii....
htsp//bbs22553 15260.
Leo Gao, Sid Black, Phil Wang, and et al. Cerebras-gpt: A family of open, compute-efficiintt, large language models. arXiv preprint arXiv:2304. 03204. 2023. URL https: http:///xxvvvvvo ort/a////.rg////////////o 0304.03200
Aurelien Garivier and Emilie Kaufmann. Optimal best arm identification with fixed confidence. CoRR,
2016. URL htt.::ttp////atee abs 1602..4588 04589.
11

---

Published as a conference paper at ICLR 2026
Gemini Team. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long text, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261, 2025. Submitted July 7, 2025; revised July 22, 2025.
Irving John Good. A bayesian significance test for multinomial distributions. Journal of the Royal Statistical Society Series B: Statistical Methodology, 29(3):399-418, 1967.
Neel Guha, Mayee F Chen, Trevor Chow, Ishan S. Khare, and Christopher Re. Smoothie: Label free language model routing. In The Thirty-eight Annual Conference on Neural Information Processing Systems, 2024. URL https::t//// htttps://openrevieee
net. fote//ormmmmm id-ppddppiiiie gqR.
Jiaxin Guo, Zewen Chi, Li Dong, Qingxiu Dong, Xun Wu, Shaohan Huang, and Furu Wei. Reward reasoning model, 2025. http::t::///att
www.oogggaaba/a/oee 14674.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurIPS, 2021.
Audrey Huang, Adam Block, Qinghua Liu, Nan Jiang, Akshay Krishnamurthy, and Dylan J. Foster. Is best-of-n the best of them? coverage, scaling, and optimality inference-time alignment, 2025. URL https:: / aarxivvii...//..///aaa///aaaavvt org/ abs/2503 21878.
Q. Huangfu and J. A. J. Hall. Parallelizing the dual revised simplex method. Mathematical Programming Computation, 10(1):119-142, March 2018. ISSN 1867-2957. doi: 10.1007/ s12532-017-0110-5 URL /tettps://do..oog/////
Hiroshi Inoue. Adaptive ensemble prediction for deep neural networks based on confidence level. In Kamalika Chaudhuri and Masashi Sugiyama (eds.), Proceedings of the Twenty-Second Interna- fional Conference on Artificial Intelligence and Statistics, volume 89 of Proceedings of Machine Learning Research, pp. 1284-1293. PMLR, 16-18 Apr 2019. URL /ttpp//eoceee.....ee mlr. prwss/ / v inouen 9a. html.
Naoto Iwase, Hiroki Okuyama, and Junichiro Iwasawa. Medrect: A medical reasoning benchmark for error correction in clinical texts, 2025. URL http:t /ttp:s://///iv/aa/ arwia. org/
wrtp://rrr/ab/////// http//bs///111 00421.
Harold Jeffreys. Some tests of significance, treated by the theory of probability. Mathemati- cal Proceedings of the Cambridge Philosophical Society, 31(2):203-22 222,, 1935. doi: 10.1017/ 50305004400011300X
Yunjie Ji, Xiaoyu Tian, Sitong Zhao, Haotian Wang, Shuaiting Chen, Yiping Peng, Han Zhao, and Xiangang Li. Am-thinking-vl: Advancing the frontier of reasoning at 32b scale, 2025. URL https:/ / arxiv. org/ alss /2005 08311
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chap- lot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud. Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023a. URL https: //arxiv.org// abs/ 2210 06825.
Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. LLM-blender: Ensembling large language mod- els with pairwise ranking and generative fusion. Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.). Proceedings of the 6lst Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers), pp. 14165-14178, Toronto, Canada, July 2023b. Association for Computational Linguistics. doi: 1/.18663/112222..acll 792/.
URL
https: /aaccaaanii wwwaacnnnnoooyyy. org/ acl-
Hiroshi Kajino, Yuta Tsuboi, and Hisashi Kashima. A convex formulation for learning from crowds. In Jörg Hoffmann and Bart Selman (eds.), Proceedings of the Twenty-Sixth AAAI Con- ference on Artificial Intelligence, July 22-26, 2012, Toronto, Ontario, Canada, pp. 73-79. AAAI Press, 2012. doi: 10.1609/AAAL V261L. 8105. URL https: /ttp//i.oogo...///// 1609/ aaai. ....1.8100 8105.
Robert E. Kass and Adrian E. Raftery. Bayes factors. Journal of the American Statistical Associa- fion, 907430):773-795, 1995. doi: 10. (0000006211455 199...997....77.
12

---

Published as a conference paper at ICLR 2026
Emilie Kaufmann and Wouter M. Koolen. Mixture martingales revisited with applications sequen- tial tests and confidence intervals. J. Mach. Learn. Res., (1) January 2021. ISSN 1532-4435.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners, 2023. URL https://aaxiia org/ 11916.
J. Zico Kolter and Marcus A. Maloof. Dynamic weighted majority; An ensemble method for drifting concepts. Journal of Machine Learning Research, 801))2755--790, 2007. URL http:// jmlr. wwtp://ppppppp./aeeeeeeee http//ooleeeaaaaaaa
LG Al Research. Exaone deep: Reasoning enhanced language models.
arXiv preprint
arXiv:2503. 12524, 2025.
Jiyi Li, Yukino Baba, and Hisashi Kashima. Hyper questions:
Unsupervised targeting few experts in crowdsourcing. In Proceedings of the 2017 ACM Conference Information and Knowledge Management, CIKM 17, pp. 1069-1078, New York, NY, USA, 2017. Association for Computing Machinery. ISBN 978 9781450349185. doi: 10. (0.114553132884.33322977 URL https:
oro/1...g//1... 11t...1111/1111 11155/330..144//134477777 .10.8447.311977 329297
Junyou Li, Qin Zhang, Yangbin Yu, Qiang Fu, and Deheng Ye. More agents is all you need. Transac- tions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreview
net / Eorum? i www.bgguuuuBeee
Michael Lindon and Alan Malek. Anytime-vall inference for multinomial count data. In Proceed- ings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA, 2022. Curran Associates Inc. ISBN 9781713871088
Chris Yuhao Liu, Liang Zeng, Jiacai Liu, Rui Yan, Jujie He, Chaojie Wang, Shuicheng Yan, Yang Liu, and Yahui Zhou. Skywork-rewards Bag of tricks for reward modeling llms. urXiv preprint 0rXiv:2410.18451, 2024.
Zhengzhong Liu, Aurick Qiao, Willie Neiswanger, Hongyi Wang, Bowen Tan, Tianhua Tao, Junbo Li, Yuqi Wang. Suqi Sun, Omkar Pangarkar, Richard Fan, Yi Gu, Victor Miller, Yonghao Zhuang, Guowei He, Haonan Li, Fajri Koto, Liping Tang, Nikhil Ranjan, Zhiqiang Shen, Xuguang Ren, Roberto Iriondo, Cun Mu, Zhiting Hu, Mark Schulze, Preslav Nakov, Tim Baldwin, and Eric P. Xing. Llm360: Towards fully transparent org/abs 2212. 06550.
llms, 2023. URL https://arxiv:
Llama Team. The llama 3 herd of models. arXiv preprint arXiv:2407.21788, 2024. Released July 23, 2024.
2024.
Thang Luong, Dawsen Hwang, Hoang H. Nguyen, Golnaz Ghiasi, Yuri Chervonyi, Insuk Seo, Junsu Kim, Garrett Bingham, Jonathan Lee, Swaroop Mishra, Alex Zhai, Clara Huiyi Hu, Hen- ryk Michalewski, Jimin Kim, Jeonghyun Ahn, Junhwi Bae, Xingyou Song, Trieu H. Trinh, Quoe V. Le, and Junehyuk Jung. Towards robust mathematical reasoning. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025. URL https: /aa::///calaa antho oogy org/ 2025 emnlp-ma i 1794/.
Bo Lv, Chen Tang, Yanan Zhang, Xin Liu, Ping Luo, and Yue Yu. URG: A unified rank- ing and generation method for ensembling language models. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar (eds.), Findings of the Association for Computational Linguistics: ACL Bangkok, Thailand, August 2024. Association for Computational Linguis- tics. doi: 10. 10.18653/v1/0244 http:///..2222.//seeeeeae findings-acl.266 URL https: /acl anthe org/2024. findings- acll 261/.
2024, pp.
Dakota Mahan, Duy Phung, Rafael Rafailov, Chase Blagden, Nathan Lile, Louis Castricato, Jan-Philipp Fränken, Chelsea Finn, and Alon Albalak. Generative reward models. CoRR, abs/2410.12832. 2024. doi: 1448550/ARXII.2410..555. URL ttttts://doli org/10. 48550/arXiv 41100 12832.
Maxwell-Jia. Aime 2024 dataset.
https:/ httpss//hhuggnggac.../////// Maxwel J-Jia// ATMME, AAM4__004, 2024. Accessed: 2025-09-07.
13

---

Published as a conference paper at ICLR 2026
Xiaoyu Tan Minghao Yang, Chao Qu aif-orm-llama311-20 2024. URL [https:// hncgg//g..ce..co/iifll//////
(https: ( http://////ttttse///////aaaaaa http//gggggggg....e////
co/intty//NN--OR--Iaanaaaa
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time /2501. 19393.
scaling, 2025. htLLht/:///axke
NVIDIA. Nvidia nemotron nano 2: An accurate and efficient hybrid mamba-transformer reasoning model, 2025, URL https: //tp:///aavv..//.i htrggaab/225033 14444.
OpenAI. Gpt-4 technical report. arXiv preprint att::mmm20X00:::000000000 2023. URL httttt::://arxiv org/abs/2303 08774,
OpenAl. gpt-oss-120b & gpt-oss-20b model card, 2025. URL https://arzavv.orr/a/ 2508. 10925.
OpenCompass. Aime 2025 dataset.
https: httpp:://mogghhngff.....e/tteee
opencompass /AIMPas0/// 2025. Accessed: 2025-09-07.
Qwen Team. Qwen3 technical report, 2025. URL AtLL:tppp//axxx..or///////
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D. Manning. Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference Neural Information Processing Systems 2023, NewrIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023. URL http: /tap::://paapeeee httppppprs..sss.pppppaaeeeftaaee wttpeee/.wlleeeppeereeeee pappp/pppppp3p 2223/hash// a85b4 2555405ed65c66 77a oeee -onnra--tttttttttooo--o-oeee
html.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. Gpqa: A grad google-proof q&a benchmark, 2023. URL https: http://aaxvvvvv.. org/ abs 2311.
Victor S. Sheng, Foster Provost, and Panagiotis G. Ipeirotis. Get another label? improving data quality and data mining using multiple, noisy labelers. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '08, pp. 614-622, New York, NY, USA, 2008. Association for Computing Machinery. ISBN 978 1605581934 doi: 10.1145/1401 890. 1401965. URL https: /oo. www.oo.g/ooo..ooo 114.
1401965.
Chenglei Si, Weijia Shi, Chen Zhao, Luke Zettlemoyer, and Jordan Lee Boyd-Graber. Getting moRE out of mixture of language model reasoning experts. In The 2023 Conference Empir- ical Methods in Natural Language Processing, 2023. URL https: http://ppneeeeeeeeeeeeeeem net./
forum? Dorum?id-0Mywla 3r.3n.
Charlie Victor Snell, Jachoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute optimally can be more effective than scaling parameters for reasoning. In The Thirteenth Interna- fional Conference Learning Representations, 2025. URL https: / /ttttps://ppereeeeee wpw..eview.eeeeeeeme/eeee..ee.ee nnt./
forum? ddw.22dd=FFWww22222
Guijin Son, Hyunwoo Ko, Hoyoung Lee, Yewon Kim, and Seunghyeok Hong. Llm-ae-a-judge and reward model: What they can and cannot do, 2024. URL 11239.
Victor Soto, Alberto Suárez, and Gonzalo Martinez-Muñoz. An urn model for majority voting in the 30th International Conference on Neural Informa-
classification ensembles. In Proceedings the tion Processing Systems, NIPS' 16, pp. 4437-4445, Red Hook, NY, USA, 2016. Curran Associates Inc. ISBN 9781510838819.
Shingo Takamatsu, Issei Sato, and Hiroshi Nakagawa. Reducing wrong labels in distant supervision for relation extraction. In Haizhou Li, Chin- Yew Lin, Miles Osborne, Gary Geunbae Lee, and Jong C. Park (eds.), Proceedings of the 50th Anmual Meeting ofe the Association for Computational Linguistics Volume I: Long Papers), pp. 721-729, Jeju Island, Korea, July 2012. Association for Computational Linguistics. URL. htt antantoolnoooooooooo oog..oo.oooooooooooo...... org/ 076/.
14

---

Published as a conference paper at ICLR 2026
Selim Furkan Tekin, Fatih Ilhan, Tiansheng Huang, Sihao Hu, and Ling Liu. LLM-TOPLA: Ef- ficient LLM ensemble by maximising diversity. In Yaser Al-Onaizan, Mohit Bansal, and Yun- Nung Chen (eds.), Findings of the Association for Computational Linguistics: EMNLP 2024, pp. 11951-111 1966, Miami, Florida, USA, November 2024. Association for Computational Lin- URL https: /ttps///aaaaatthhaaaee org/
guisties. doi: 10.
2024. f.l.finniin findings emnlp. 698/.
Jonathan Uesato, Nate Kushman, Ramana Kumar, H. Francis Song, Noah Y. Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. Solving math word problems with process- and outcome-based feedback. CoRR, abs/2211.14275. 2022. doi: 0044850/ARXIV2211.14777 URL https: /doii.i..//// org / /00. (05555//rXiv 2212.
Neeraj Varshney and Chitta Baral. Model cascading: Towards jointly improving efficiency and accuracy of nlp systems, 2022. URL https:://aaaaae
wwg/abs//////oooo...r 2210. 05528.
Ziyu Wan, Xidong Feng, Muning Wen, Stephen Marcus McAleer, Ying Wen, Weinan Zhang, and Jun Wang. Alphazero-Bic like tree-search guide large language model decoding and train- ing. In Forty-first International Conference on Machine Learning ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024. URL https: http:s://ppeeeeveee http///pppnwee..eeeeeeeee net forum?idd CA0pREezgj.
Binghai Wang, Rui Zheng. Lu Chen, Yan Liu, Shihan Dou, Caishuang Huang, Wei Shen, Senjie Jin, Enyu Zhou, Chenyu Shi, Songyang Gao, Nuo Xu, Yuhao Zhou, Xiaoran Fan, Zhiheng Xi, Jun Zhao, Xiao Wang, Tao Ji, Hang Yan, Lixing Shen, Zhan Chen, Tao Gui, Qi Zhang, Xipeng Qiu, Xuanjing Huang, Zuxuan Wu, and Yu-Gang Jiang. Secrets of RLHF in large language models part II: reward modeling. CoRR, 000.2224000000000 2024. doi: 10.48550/ARXIV24000 24000000000000 URL https:
/ /ooii , .rr.orgggoooooo... wwtgg//..00 48550 wa8855//aXiii... 2401. 06080.
Fei Wang, Xingchen Wan, Ruoxi Sun, Jiefeng Chen, and Sercan Ó. Ank. Dynscaling: Efficient verifier-free inference scaling via dynamic and integrated sampling, 2025a. URL https://
arxiv.org/ abs/ 2506. 6043.
Xuezhi Wang. Jason Wei, Dale Schuurmans, Quoe V Le. Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, 2023. URL net/
htttss:/cppereete wtwp:rrwwwwwweeeeeeoeeoueeeeaeee
Zixiao Wang, Yuxin Wang, Xiaorui Wang, Mengting Xing, Jie Gao, Jianjun Xu, Guangcan Liu, Chenhui Jin, Zhuo Wang. Shengzhuo Zhang, and Hongtao Xie. Test-time scaling with reflective generative model, 2025b. URL htts: aaariiiiivv.///iv///aaa/aaaaaavv org/ 2201.00111.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoe V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. arXiv preprint 1900 2022. URL htttt:///aaxii org/ abs/ /2001 11903.
Jacob Whitehill, Ting-fan Wu, Jacob Bergsma, Javier Movellan, and Paul Ruvolo. Whose vote should count more: Optimal integration of labels from labelers of unknown exper- tise. In Y. Bengio, D. Schuurmans. J. Lafferty, C. Williams, and A. Culotta (eds.), Advances in Neural Information Processing Systems, volume 22. Curran Associates, Inc., 2009. URL https::/proccssttt www.eeddddnnnnnnneneeeeeeeeeeeee neurips. cc/ file/ File/f883
Pap.r.pdf.
Yangzhen Wu, Zhiqing Sun, Shanda Li. Sean Welleck, and Yiming Yang. Scaling inference compu- tation: Compute-optimal inference for problem-solving with language models. In The 4th Work- shop on Mathematical Reasoning and Al at NeurlPS 2024. htLhttps:/ /oppnrevie...e.e net/ / Eorum? iid= WWD2WScBu..
Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Inference scaling laws: An empirical analysis of compute-optimal inference for LLM problem-solving. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025. URL https: http:////eeeeeeeeeeeeeeeeeeeeeee wwwp/fott///eeeee http://wwwwm.m..ccooo/ooo.....m
d=VNcckp.JH..
15

---

Published as a conference paper at ICLR 2026
Yuchen Yan, Yongliang Shen, Yang Liu, Jin Jiang, Mengdi Zhang, Jian Shao, and Yueting Zhuang. Inftythink: Breaking the length limits of long-context reasoning in large language models, 2025.
URL htttps//aaxxi..rg/ abs/2503 06692.
Yixin Ye, Zhen Huang, Yang Xiao, Ethan Cher, Shijie Xia, and Pengfei Liu. Limo: Less is more for reasoning, 2025. URL https://axxii. org/ abs 03387.
Murong Yue, Jie Zhao, Min Zhang, Liang Du, and Ziyu Yao. Large language model cascades with mixture of thought representations for cost-efficient reasoning. In ICLR, 2024. URL https:
/ /tt.://ooooomm/dddooooaf........
Wenting Zhao, Pranjal Aggarwal, Swarnadeep Saha, Asli Celikyilmaz, Jason Weston, and Ilia Ku- likov. The majority is not always right: RI training for solution aggregation, 2025a. URL
https: /tapis:://rxiv... org/ abs///b//////// 2509. 66700.
Xuandong Zhao, Zhewei Kang, Aosong Feng, Sergey Levine, and Dawn Song. Learning to reason without external rewards, 2025b. URL battps://aaae orbb 2tt.//555....999
16

---

Published as a conference paper at ICLR 2026
A USAGE OF LLMS
We have used LLMs to write paragraphs as well as find relevant literature. Programs are written with the aid of LLMs based on agentic programming tools (i.e., Cursor). However, all the responsi- bility of the content lies with us. Theorems and proofs are written by us.
B RELATED WORK
This section describes the related work on aggregating multiple answers from different individuals and LLMs.
B. AGGREGATION OF MULTIPLE ANSWERS FROM INDIVIDUALS
Controlling N in majority voting Ensembling multiple predictions is widely used technique for improving the accuracy of various machine learning tasks. One of the classic papers by Kolter & Maloof (2007) considers online ensemble learning, where the system remove experts based on their performance. Regarding the optimal control of N, the idea closest ours the Urn model by Soto et al. (2016), which calculates the Bayesian probability that the empirical majority matches the true majority. However, their model samples without replacement, whereas LLM generation fits sampling with replacement. Their method also requires candidate answers and is thus not directly applicable to our setting. Motivated by ensembling methods for deep image classifiers, Inoue (2019) proposed an adaptive ensemble prediction method that adaptively aggregates the outputs of multiple probabilistic classifiers.
dynamically add or
Opinion aggregation in crowdsourcing A relevant lines of works in pre-LLM era is the opinion aggregation crowdsourcing (Sheng al., 2008: et al., 2017). One of the most popular methods in opinion aggregation is the method by David and Skene (Dawid & Skene, 1979). They introduced probabilistic model that estimates the true labels of items by leveraging the agreement among multiple annotators. It comprises confusion matrix π, whose jk entry represents the probability such that each annotator's label is j when the true label is Such confusion matrix is not directly applicable to setting because generation. For example, in the AIME datasets, building a confusion matrix of 1,000 rows (possible answers are integers from O 999) is not very practical. The Dawid-Skene model also assumes that the most frequent answer is the correct one. Subsequent works (Whitehill et al., 2009; Kajino al., 2012: Takamatsu et al., 2012) addressed this issue by introducing difficulty parameter for each problem. One of the largest difference between crowdsourcing and LLM ensemble is that the former typically assumes single answer from each annotator, whereas the latter can generate multiple answers from the same LLM.
cannot generally assume fixed domain of answers in LLM
B.2 AGGREGATION OF MULTIPLE ANSWERS FROM LLMS
Our method belong to a large umbrella of LLM Ensemble methods, where the forecaster uses multi- ple LLMS for a better output. A comprehensive survey on this topic (Chen et al., 2025b) categorizes ensemble LLM methods into several categories." 6 Our method falls into the category of "ensemble after inference", where we aggregate the outputs of multiple L.MS after they have generated their responses.
Within this category, Chen et al. (2025b) classified methods into three sub-categories: (1) selection, (2) selectioonthhen-rggeenaaiio and (3) cascade. The first directly selects the answer from generated outputs (our setting). The second selects a subset of LLMs and then merges their outputs using another LLM or trained model. The third uses a cascade of LLMs, invoking a stronger model only when needed to save cost. Methods in (1) and (2) typically assume a fixed number of generations per I.IM and optimize aggregation. In contrast, we primarily consider dynamically controlling the number of generations. Methods in (3) focus minimizing total cost of calling LLMs.
FFigure 2 therein.
17

---

Published as a conference paper at ICLR 2026
(1) Selection Li et al. (2024) proposed AgentForest that aggregates the predictions of multiple agents by using similarity agreement. Guha et al. (2024) introduced Smoothie, a graphical based method to choose the best LLM for each problem. Si et al. (2023) introduced Mixture of Rea- soning Experts (MORE) framework that adopts multiple prompting strategy to obtain a mixture of experts and aggregates them by random forest classifier. Our methods belongs this sub-category. Compared with these methods, our method is simple average while others may use more com- plex aggregation strategies. Note also that these methods primarily consider a single generation per each LLM, whereas our paper primarily considers large number of generations per each LLM. A recent paper by Zhao et al. (2025a) proposes an aggregation method of multiple solutions by using reinforcement learning from verifiable rewards.
Jiang al. (2023b) introduced LLM-Blender, ensemble LLM method that comprises two modules: PAIRRANKER and GENFUSER. PairRanker chooses K among N L.LMs, and GenFuser merges the outputs. Tekin et al. (2024) introduced LLM-TOPLA, an L.MM ensemble method that maximizes the diversity of the answers. Based on the answer dis- tribution of N LLMs, they choose K subset of LLMs that maximizes the diversity, and then train an aggregator (like multi-layer perceptron) that minimizes the cross-entropy loss. Lv et al. (2024) proposed an end-to-end method that integrates the subset selection and regeneration. Most of these methods are based on the idea of using many LLMs (or same LLM with different prompts) and single generation per each prompt, whereas our paper primarily considers a relatively small subset of LLMs for each prompt, and large number of generations per each LLM.
(2)
(3) Cascading Varshney & Baral (2022) is one of the earliest work that introduced cascading. Yue et al. (2024) proposed an aggregation of weak and strong LLMs. In their model, if the weak LLM and the cascade LLM disagree with the answer, then the strong LLM is invoked. The primal motivation in cascading is to save the cost of calling strong LLMs, which is orthogonal to our goal of improving the accuracy of maximizing the accuracy given large amount of computation.
Answer selection based on reward models and LLM-as-a-judge A common approach to aggre- gate multiple answers from LLMs is to use reward models LLMM-asa-juddee methods. Typically, reward models are constructed on top of language models. These approaches can be broadly cate- gorized into two groups: those in which the reward model directly outputs scalar value (Rafailov et al., 2023; Liu et al., 2024), and those in which the reward model provides comparative judgments or rankings over multiple responses (Mahan et al., 2024; Dong et al., 2024; Son al., 2024; Guo et al., 2025; Chen et al., 2025a). The methods of the latter category are referred to as generative reward models, reward reasoning models, or LLM-as---judge Compared to our approach, these methods incur additional computational cost due to the reliance on reward models. Also, in our experiments, we did not observe particular advantage of using reward models (see Table 2).
C PROOF OF THEOREM 1
Proof of Theorem 1. Let (n) = N, (n)/n be the empirical mean of answer j at round n. Hoeffd- ing's 's inequality implies that
(n) pal 2 c) < 2 2exp(-22x2)
Let A = I - p,)) > 0 be the gap between the most frequent answer and the second most frequent answer. Then it holds that
A
8 P
A
(m)(m)
p5l N
(m(m))
- Ppl
N 2
(Union bound)
Nn=No
n=
80
A
2exp
-11
(Hoeffding's inequality)
2
N=
2c No
(15)
2-2212
18

---

Published as a conference paper at ICLR 2026
and by choosing Nu (o) sufficiently large, the hight-hand side can be made no larger than 8/s. Union bound over all s answers implies that, with probability at least 1 - 6. it holds that
A 2 0. Vj 1, vn 2 No(6).
(1(n) (m) 2 Pj - 2
2
Namely, at least with probability 1-6, the empirical most frequent answer is indeed the true majority answer for all 2 No((), and thus, if stopping time is longer than N6) the algorithm returns the true majority answer. By choosing Nax 2 (6) and B sufficienlly large', the algorithm stops after (6(6) with probability 1, and thus, the algorithm returns the true majority answer with probability at least - 5. Since > O is arbitrary, the algorithm returns the true majority answer with probability arbitrarily close Proof of Theorem is complete.
Remark 1. (Frequentist stopping criteria) While Dirichlei posterior naturally fus with our task, we may consider frequentist stopping criteria based on the observed data. Advantages of the frequen- tist approach include its closed formula as well as rigorous guarantee in view of a frequentist. A drawback is that its configuration of the hyperparameter tends to be conservative: the confidence level that it requires is often higher than what actually is, potentially leading to oversampling. To bound the error probability, it needs consider the correction due adaptive sampling (Kaufimamn & Koolen, 2021), ), as well as a multiple-testing correction with respect to the size of answer set s(t). The latter seems particularly problematic, s(l) is unknown and potentially unbounded. For this reason, we do not see any existing work that adopts a frequentist approuch to lesting adaptive ma- jority voting. For example, existing methods on majority voting, such Soto et al. (2016), which we will elaborate in Section B.1, also adopt Bayesian approach. Therefore, we do not pursue this direction in this paper.
C. FINITE-TIME ANALYSIS OF STOPPING TIME
In this section, we conduct finite-time analysis of the stopping time of Algorithm 1. Theorem 2. (Finite-time stopping) Algorithm stops within
1 A
O
log (1-A) max] 1/0))
rounds with probability at least where the gap between the most frequent answer and the second most frequent answer, and A is the set of possible answers by LLM.
Note that this rate is optimal for 8-correct identification because of the lower bound of the best-arm identification problem (e.g., Garivier & Kaufmann (2016)) with two arms, which is about identifying the larger of two Bernoulli distributions with gap A, is 1/A3) as well.
Proof of Theorem 2. Since we focus on a particular problem q we drop and denote ag be the gold answer and A be the possible answers by LLM.
Let P(n) Pr[X, 2 max,311 X;, X ~ Dirichlet( [N₁ +1, N2+1, Na +1, a)). Let Pa (n) be the Beta posterior probability such that the parameter of answer ag is larger than that of answer a. By using the fact that Dirichlet distribution restricted to two dimensions is Beta distribution, is equivalent to
1 P(X 2 1/2], X Beta( Noy))
(16)
"This is because, the possible combination of answers with the first No samples is finite, and thus, the possible value" of BF that it can take until the first No sample is finite. If we set B larger than that the largest. of such values, then the algorithm stops before the No samples.
19

---

Published as a conference paper at ICLR 2026
A sufficient condition for stopping at round n is (c.f., Eq. equation and Eq. equation 8) is
P(n)
[B 2 P(n)
BB g(n) 1- P(n))
(17)
1 P(n))
2 {2B 2 1 P(n))
(for 2 2)
(18)
2 (P(n) 2 1 - 1
(19)
2B
2 Po (n). 21 2|
(20)
azpa
By Hoeffding's inequality, for any
A
[n( Pal I 4
(21)
holds with probability at least exp( 2 If we fix such that
[A]]
n 2
log
(22)
2
8
then equation 21 holds for all a E A with probability at least 1
Under equation 21, it holds that
A
A
A
Pa, (n) Pa (n) 2 Pa, Po 2
A
(23)
2
2
for all a + ag. Therefore, by letting (n) (Da, (n) + Pa (n))
< 1/2, we have
7
Pa (n) 2
exp( nd(( 1/2)) (by Lemma 5)
(24)
1/2 μ
If we choose such that
1
B μ
2
log
(25)
d(u, 2)
then
7 exp(-nd(u4 1/2)) 2 1
(26)
1/2 μ
2|A
In summary, if we choose n such that both equation 22 and equation 25 hold, then the algorithm stops at (or before) with probability at least 5. Regarding the order of equation 25, we have
B log 2 - p
n
(27)
d(u, 1
= A
log (IA1B)
(Pinsker's inequality: d(p, q) 2 2(p 9)
(28)
(29)
and thus, such that both equation 22 and equation 25 hold is
n O 2
log (IA) max(B, 1/8))
(30)
Lemma (Beta tail). Let Beta(1 ( + np, 1 + n(1 - 4)) be a random variable followingg the Beta distribution. Then, for any μt, holds that
P[x 2 a) <
exp(-nd(p,,) a)).
where d(yr,a) y log # +(1 11) log ي-د is the KI. divergence between two Bernoulli distributions.
20

---

Published as a conference paper at ICLR 2026
Proof of Lemma S. Let B(a, b) and (2) be the Beta function and the gamma function, respectively.
P[X al
(31)
B(1 + ny, 1 + n11 - 1)) [ a= dx
(32)
1
.(
np + n(1 11)
nn x))(1-a)
x² TH
x)
dx
B(1 + nu, + n11 - 1))
-(1-1] m1
a(
)2
x (1-x)
(1-x)
(33)
(by integration by parts)
(34)
1 B(1 + ri, 1 + n1 - 11))
a TM (1 - ()m(l--))
(35)
n11
(1-1)
+2)
a(1 - a) a 10 a) ()(((=H)
(36)
[(1 + (1 + n(1
11)) n(a (1)
(37)
and by Stirling's 's formula 2 S 1--1/ S 22Te 1/12 we have
1(x)
P[X N al - a(1 - 2/22/12
(n 22s
(n + 2)
(nu + 1)(n(1 1) + 1) (mp) =))m((-p) (1 -))
n( - 11) 2m
(38)
a(1 1/12/12
27n
è -ndlun))
(39)
n(a- P) 2m
n(1 1
06 25/12
27
1-a exp( exp(-nd(H,a))
(40)
a 11) 2
25/11 12
54
-nd(ana))
(41)
5aa= n)
π
-n(ppa)
(42)
and the proof complete.
D LIST OF LLMS AND PROBLEM SETS
We tested the following LLMs. The model temperature is 0.6 unless otherwise specified. We fol- low the model recommendation to set the temperature and other hyperparameters. The maximum model length is min(X, maximum context length of LLM) 2500 tokens, where X 100000 all whereas X = 50000 for reserved for the prompt: we believe that this does not matter MATH500 and GPQA-DIAMOND at all, and to A1ME2024/2025 very slightly.
but
The 2500 token margin is
Phi-4-reasoning (Abdin et al., 2025) is a 14-billion-parand model developed by Microsoft, released in April 2025. It builds on the Phi-4 base model us- ing supervised fine-tuning on a dataset of chain-of of-thought traces and reinforcement learn- ing. We set temperature to 0.8.
(14B) reasoning-oriented
GPT-OSS-200 (OpenAI, 2025) is the smaller version of the two LLMs released in October 2025 by OpenAI. This model has 21B parameters total. We set the reasoning effort to be medium (default setting).
AM-Thinking-vl (Ji et al., 2025) is a 32B dense model released in May 2025 by the a- m-team. It is built upon the pre-trained Qwen 2.5-32B-Basee then enhanced through specialized post- tosintraining pipeline featuring Supervised Fine- Tinn-Tnnng (SFT) followed by re- inforcement leaming (RI..
21

---

Published as a conference paper at ICLR 2026
SLAONE-Deep-255 32B (LG Al Research, 2025) is 32B model released in May 2025 by LG AI Research as part of the EXAONE Deep series. Built with 64 Transformer layers, a 102K vocabulary, and 32K-token context window, is designed to excel in reasoning-intensive intensive tasks such as mathematics and coding.
Nemotroo--Nano-98 (NVIDIA, 2025) is Aabillion-paraneee hybrid reasoning model by NVIDIA. released in August 2025. It features Mamba-2 + Transformer hybrid archi- tecture, replacing most attention layers with efficient Mamba-2 layers. was pretrained from scratch (using 12B base model over 20 trillion tokens) and then compressed via distillation. Post-training includes SFT, GRPO, DPO, and RLHF
SltaStone-S1-32B (Wang et al., 2025b) is a 32B reflective generative reasoning model, released around July 2025. It introduces novel Reflective Generative Form, merging policy generation and process reward modeling within a single shared backbone, enabled by lightweight Sulf-super Process Reward Model (SPRM).
Qwen3, released in April 2025 by Alibaba Cloud (Qwen Team, 2025), is the third- generation open-source large language model family featuring hybrid reasoning, long context support, agentic capabilities, and multilingual fluency. We use three versions of Qwen3. Namely, Qwen3-4B, Qwen3-14B, and Qwen3-30B-A3B A3B-Thinking-
LIMO-v2 (Ye et al., 2025) a 32B Owen2.5-based reasoning model released in July 2025, fine-tuned on ~800 carefully curated samples to achieve top-tier math reasoning with re- markable data emweeeyyyeebobiiingn the -s-ss- principle.
We tested the following datasets:
AIME2024 (Maxwell-Jia, 2024) consists of 30 problems that were used American Invi- tational Mathematics Examination (AIME) held during January 31 and February 1, 2024. AIME2024 tests mathmatical moollmm-slvingg skills vast field of mathmatical topics. High-scoring high-school students are invited to participate in the United States of Amer- ica Mathematics Olympiad (USAMO). All answers are integers between 1-999.
AIME2025 (OpenCompass, 2025) consists of 30 problems that were used American Invi- tational Mathematics Examination (AIME) held from February 10 to February 12, 2025. Its format identical AIME2024.
GPQA-DIAMOND (Graduate-Leved Google-Proof Q&A Benchmark, Rein al. 2023) is a set of multiple-choice questions crafted by PhD-level experts in biology, physics, and chemistry. The Diamond is a subset of 198 GPQA problems that distinguishes Ph.D. level experts from the others. The answers are in multiple-choice format (A-D).
MATH500 (Hendrycks et al., 2021) is a benchmark derived from the MATH dataset, which contains challenging competition-levad mathematics problems covering algebra, geometry, number theory, probability, and other advanced topics. The MATH500 subset consists of 500 carefully selected problems used in recent evaluation studies, and is designed to test mathematical problem-solvie skills beyond high-school level. All problems require generating detailed reasoning and solutions rather than multiple-choice responses. The answer format varies, including numeric integers, fractions, complex numbers, and vectors.
Among these datasets, AIME2024/2025 benefits for a long chain of thought (CoT) reasoning. as the problems are challenging and require multi-step reasoning." 9 GPQA-DIAMOND and MATH500 also require long COT, but the benefit of is less significant than AIME2024/2025. We did not include GSM8K (Cobbe et al., 2021) because these problems are relatively easy and finishes with short CoT for the tested LLMs, and thus the benefit of ensemble was not significant.
Regardinng the scaling of performance as function of CoT length, see, e.g., Figure of Muennighoff et al. (2025) and Figure 7 of Yan al. (2025).
22

---

Published as a conference paper at ICLR 2026
Table 3: Summary performance per model across datasets. The scores are estimated from least 80 generation for each model and dataset. GPQA-D is an abbreviation of GPQA-DIAMOND,
LLM
AIME2024
AIME2025
GPQA-D
MATH500
Bol
Boco
Bol
Booo
Bol
Booo
Bol
Boxc
AM-Thinking-vl
0.789
0.900
0.762
0.867
Datarus-RI-144
0.516
0.733
0.370
0.600
EXAONE-DDep-$2B
0.715
0.867
0.627
0.767
0.661
0.692
0.945
0.962
GPT-OSS-20B
0.780
0.900
0.744
0.900
0.642
0.722
0.928
0.960
LIMO-v2
0.620
0.800
0.527
0.700
MetaStone-S1-32B
0.820
0.867
0.747
0.800
0.670
0.707
0.947
0.950
IVIDDIA-Nemotton-Wan---B
0.716
0.867
0.600
0.733
0.584
0.626
0,938
0.956
Phi-4-reasoning
0.729
0.867
0.643
0.833
0.658
0.727
0.878
0.944
Qwen3-4B
0.735
0.800
0.655
0.733
Qwen3-14B
0.830
0.867
0.744
0.800
0.946
0.956
Been3-30B-AA3--Thiniingg
0.905
0.933
0.858
0.900
0.720
0.732
0.954
0.960
E Bol AND BEST-OF-3c PERFORMANCE OF EACH MODEL
We list Bol (averaged) and best-of-co performance of each model Table 3. We have used the same prompt (Section E. for all models, which might be sub-optimal for some models. The evalu- ated performance also depends on the answer parser. While we used the consistent and a reasonably acknowledge some examples" where the parsing is imper-
flexible answer parser for all models, fect. We have not specified any tool call option. Also note that GPT-OSS-20B3 reasoning mode set medium (default setting), which is the second best setting. Finally, clarify our goal is to argue superiority of some models over the others, but to give some idea on the performance of each model that use for the verification our methods of adaptive sampling (Algorithm 1).
E. PROMPTS FOR ANSWER GENERATION
We send the following request to a LLM that we launched as a vllm process:
("role": "user", "content": prompt)
where the examples of the prompt are given below: The first prompt is from AIME2024, and the second prompt is from GPQA-DIAMOND.
Let Sx, ys and $z$ be positive real numbers that satisty the following system of equationa:
\ \[\log_2\left ((x \over yz]lright) = (1 \over 2
\[1log2\\eett ((y \over x2/\right) {1 lover 3:\ A[Alog_2\lets
\over xyllright)
lover 4:
Then the value of $1left//log (x" (x"4" 22) \right where $mş and $n8 are relatively prime positive integers. Find Smtns.
(Atfrac|m) (n)s
Flease reasen step by step, and put your final answer within Aboxed 11.
Among the following exoplanets, which one has the highest density?
a) An Rarth-mass and Karth-radi us planet.
b) A planet with 2 Earth masses and density of approximately 5.5 /.
In particular, MATH500 where the format varies.
23

---

Published as a conference paper at ICLR 2026
planet with the same composition as Barth but times more massive than Earth.
d) A planet with the same composition as Barth but half the mass of Earth.
C. b
D.
Please reascn step by step, and put your final answer the letter choice (A), (R), (C), etc. within \boxed().
For NVIDIA Nemotron-Nano-99R, we prepend the recommended system message "/think".
E.2 PROMPTS FOR LLM-AS-A-JUDGE
The following illustrates a prompt used to instruct an LLM-as-a-judge to select the best answer among set of candidates. In this prompt, last-part.1, last jast-part-a. denote the final 5000 characters of each answer preceding the < </think> tag.
Please evaluate following answer excerpts for this mathematical problem and determine which answer you think is the most correct.
Problem:
let yx, and $2$ be positive real numbers that satisfy the following system of equations:
\/Alog_2\lett 2/left (Ix \over yz)lright) = (1 \over 2
M/Vlog 21left (ly \over xz)\right)
lover 3]
[[llg__\\eftt \over ky/\right)
\over 4:
(x (x'o__(x"44))) 42 \right is (n] Then the value of $1left Allet//log \log_2
where $mş and $n$ are zelatively prime positive integers. Find Sm+n$.
Answer (Last 5000 chars before </think>): [last_part_1]
Answer 2 (Last 5000 chars before </think>); (last_part_2)
Answer 3 (Last 5000 chars before </think>);
Answer (Last 5000 chars before </think>): (last_part_4]
Answer 5 (Last 5000 chars before </think>); (last_part_5)
Among the above 5 answer excerpts (showing the last parts before </ think> tag), which answer do you think is the most correct, Logical, and complete?
Please provide detailed reasoning for your judgment, and then output the number of the answer you think is correct (1, 2, 3, 4, 5) enclosed in \boxed!).
Example: \boxed([1
Judgment:
24

---

Published as a conference paper at ICLR 2026
Table 4: Basic performance for each problem. The final line column "accuracy" indicates Bol performance, and the final line at "majority answer" indicates best-of-co performance. LLM-Phi- 4-reasoning, DItaset=AIME2005.
Problem No.
Total answers
Correct answers
Accuracy
Gold answer
Majority answer
160
159
0.994
70
70
2
160
112
0.700
588
588
3
160
154
0.963
16
16
160
150
0.938
117
117
5
160
146
0.912
279
279
6
160
158
0,988
504
504
7
160
96
0.600
821
821
8
160
147
0.919
77
77
9
160
134
0.838
62
62
10
160
58
0.362
81
81
11
160
120
0.750
259
259
12
160
137
0.856
510
510
13
160
0.031 5
204
487/3
14
160
0.031
60
63
15
160
0.000
735
147
16
160
158
0.988
468
468
17
160
157
0.981
49
49
18
160
87
0.544
82
82
19
160
154
0.963
106
106
20
160
114
0.713
336
336
21
160
143
0.894
293
293
22
160
45
0.281
237
60671
23
160
66
0.412
610
610
24
160
77
0.481
149
149
25
160
132
0.825
907
907
26
160
111
0.694
113
113
27
160
136
0,850
19
19
28
160
0.006
248
625
29
160
75
0.469
104
104
30
160
0.300
240
240
total
4800
3085
0.643
0.833
E.3
SOURCE CODE
Our source code is available at
F COMPLEMENTARITY IN LLM ENSEMBLES FOR AIME 2025
In the AIME2025 dataset, we explored the combination of Phi-4-reasoning (Table 4) and GPT-OSS- 20B (Table 5) enhance performance on complex reasoning tasks. By leveraging the strengths of both models, we aimed to achieve better accuracy and robustness in our predictions. In this case, Phi-4-reasoning can solve Problem 30 that GPT-OSS-20B cannot solve, and can complement the performance. As result, its LLM ensemble achieved 0.933 best-of-oo accuracy, which is higher than the individual accuracies of Phi-4-reasoning (0.733) and GPT-OSS-20B (0.900). This demon- strates the effectiveness of combining different models improve overall performance on challeng- ing tasks.
25

---

Published as a conference paper at ICLR 2026
Table 5: Basic performance for each problem. The final line column "accuracy" indicates Bol performance, and the final line at "majority answer" indicates best-of-ox (limit) performance. LLM=-GPT-OS--00 Dataset=AlME2025.
Problem No.
Total answers
Correct answers
Accuracy
Gold answer
Majority answer
85
85
1.000
70
70
2
85
76
0.894
588
588
3
85
85
1.000
16
16
4
85
83
0.976
117
117
5
85
81
0.953
279
279
6
85
85
1.000
504
504
7
85
52
0,612
821
821
8
85
0.941
77
77
9
85
0.882 75
62
62
10
85
0.624
11
85
0.718 61
259
259
12
85
0.659 17 0.200
510
510
13
85
204
204
14
85
0.035 3
60
74
15
85
0
0.000
735
147
16
85
0.953 81
468
468
17
85
85
1.000
49
49
18
85
62
0.729
82
82
19
85
0.988
106
106
20
85
0.929 79
336
336
21
85
0.847
293
293
22
85
85
1.000
237
237
23
85
45
0.529
610
610
24
85
58
0,682
149
149
25
85
81
0,953
907
907
26
85
72
0.847
113
113
27
85
81
0.953
19
19
28
85
0.424
248
248
29
85
71
0.835
104
104
30
85
14
0.165
240
188
total
2550
1898
0.744
0.900
26

---

Published as a conference paper at ICLR 2026
G ADDITIONAL EXPERIMENTS
To verify the robustness of our findings, we conducted similar experiments on other LLMs and datasets. The results are consistent with the main experiments the paper, confirming the robustness of our proposed methods across different settings. As is the main paper, all error bars are standard two-sigma confidence intervals.
G.1 EXPERIMENTAL SET 1: EFFECTIVENESS OF ADAPTIVE SAMPLING
In the following pages, present the performance comparison between our proposed adaptive algorithm (Algorithm 1) and the fixed-sample BoN across various LLMs and datasets (Figures 8- 11). The results consistently demonstrate that our adaptive approach outperforms the fixed-sample- size method given the same number of generation (= samples) or the same token budget. This is because algorithm is adaptive; for easy problems where the model always outputs the same answer, uses fewer samples, while for hard problems where the model's answers vary, it uses more samples. This adaptivity leads better overall performance compared fixed-sample-size approach.
27

---

Published as a conference paper at ICLR 2026
Adaptive
Accuracy
0.85
0.85
0.80
0.80
0.75
0.75
10a
10
10
10a
Average # of Gens (log scale)
Average Total Tokens (log scale)
(a) AIME2025
Adaptive
0.925
0.900
0.900
Accuracy
0.875
Accuracy
0.875
0.850
0.850
0.825
0.825
0.800
0.800
0.775
0.775
10a
10a
10
10'
10a
10a
Average # of Gens (log scale)
Average Total Tokens (log scale)
AIME2024
Adapttive
0.72
0.72
0.70
0.70
Accuracy
Accuracy
0.68
0.68
0.66
0.66
0.64
0.64
10"
10'
10'
10a
10a
Average Gens (log scale)
Average Total Tokens (log scale)
(c) Diaamdd
Adeptive Rxed
0.960
0.960
0.955
0.955
Accuracy
Accuracy
0.945
0.945
0.940
0.940
0.935
0.935
0.930
0.930
10"
10º
10a
10a
10"
Average of (log scale)
Average Total Tokens (log scale)
(d) MATH500
Figure 7: Cost-analysis of our proposed method and fixed BoN for GPT-OSS-201 20B. The error bars are standard two-sigma confidence intervals. Green dashed line indicates the best-of-co performance.
28

---

Published as a conference paper at ICLR 2026
0.85
0.85
Adaptive Fixed
Accuracy
Accuracy
0.75
0.75
0.70
0.70
0.65
0.65
100
10r
10
Average Gens (log scale)
Average Total Tokens (log scale)
(a) AIME2025
0.875
0.875
Fixed
0.850
0.850
Accuracy
0.825
Accuracy
0.625
0.800
0.800
0.775
0.775
0.750
0.750
0.725
0.725
10a
10'
Average # of Gens (log scale)
Average Total Tokens (log scale)
(b) AIME2024
Adaptive
0.73
0.73
0.72
Accuracy
0.69
0.69
0.68
0.68
0.67
0.67
0.66
0.66
10"
10
10a
10a
10*
Average # of Gens (log scale)
Average Total Tokens (log scale)
(c) GPQA- Diamond
0.95
0.95
Adaptive
Accuracy
Accuracy
0.93
0.93
0.92
0.92
0.91
0.90
0.90
10a
10:
10a
10a
10a
Average Gens (log scale)
Average Total Tokens (log scale)
(d) MATH500
Figure 8: Cost-analysis of our proposed method and fixed BoN for Phi-4-reasoning. The error bars standard two-sigma confidence intervals. Green dashed line indicates the best-of-2o perfor- mance.
29

---

Published as a conference paper at ICLR 2026
Adaptive
0.750
0.750
Fised
0.725
0.725
Accuracy
0.700
Accuracy
0.700
0.675
0.675
0.625
0.625
0.600
0.600
0.575
0.575
10a
10
10c
Average of Gens (log scale)
Average Total Tokens scale)
(a) AIMF2025
0.875
0.875
Fixed
0.850
0.850
Accuracy
0.825
Accuracy
0.825
0.800
0.800
0.775
0.775
0.725
0.725
0.700
0.700
10a
10a
Average # of Gens (log scale)
Average Total Tokens (log scale)
(b) AIME2024
0.64
0.64
Adaptive
0.63
0.63
Accuracy
Accuracy
0.60
0.59
0.59
10"
10
10*
10a
Average # of Gens (log scale)
Average Total Tokens (log scale)
(c) GPQA-Diamont Diamond
Adapiive
0.955
0.953
Accuracy
0.950
Accuracy
0.950
0.945
0.945
0.940
0.940
0.935
0.935
10a
10a
10a
10a
10a
Average of Gens (log scale)
Average Total Tokens (log scale)
(d) MATH500
Figure 9: Cost-analysis of our proposed method and fixed BoN for Nemotronn-Nao--99. v2. The error bars standard two-sigma confidence intervals. Green dashed line indicates the best-of-co performance.
30

---

Published as a conference paper at ICLR 2026
0.91
0.91
Adaptive Fised
0.90
0.90
Accuracy
Accuracy
0.88
0.89
0.87
0.86
0.86
0.85
0.85
100
10'
Average Gens (log scale)
Average Total Tokens (log scale)
(a) AIME2025
0.94
0.94
Fixed
Accuracy
Accuracy
0.92
0.91
0.91
0.90
0.90
10"
10a
Average # of Gens (log scale)
Average Total Tokens (log scale)
(b) AIME2024
Adaptive
0.735
0.735
Accuracy
0.730
Accuracy
0.730
0.725
0.720
0.720
10a
10a
10a
10
10
10e
Average # of Gens (log scale)
Average Total Tokens (log scale)
(c) GPQA-Diamond
Adeeptive
Ried
0.960
0.960
Accuracy
0.958
Accuracy
0.958
0.956
0.956
0.954
0.954
10a
10a
10a
10'
10a
Average of Gens (log scale)
Average Total Tokens (log scale)
(d) MATH500
Figure 10: Cost-analysis of our proposed method and fixed BoN for Qwen3-30B-AS3 -3-h-ii-- 2507. The emror bars standard two-sigma confidence intervals. Green dashed line indicates the best-of-co performance.
31

---

Published as a conference paper at ICLR 2026
Adaptive Fised
0.76
0.76
Accuracy
Accuracy
0.74
0.74
0.70
100
10a
Average Gens (log scale)
Average Total Tokens (log scale)
(a) AIMF2025
0.88
0.88
lixed
Accuracy
0.86
Accuracy
0.86
0.84
0.84
10a
10a
10?
10
Average # of Gens (log scale)
Average Total Takens (log scale)
(b) AIME2024
Adopptvve
0.695
0.695
0.690
0.690
Accuracy
0.685
0.685
0.680
0.680
0.675
0.675
0.670
0.670
0.665
0.665
0.660
0.660
10a
10a
10a
10a
10a
10
Average # of Gens (log scale)
Average Total Tokens (log scale)
(c) GPQA-Diamoni
Adaptive Fixed
0.9625
0.9800
0.9600
Accuracy
0.9575
Accuracy
0.9575
0.9550
0.9550
0.9525
0.9500
0.9500
0.9475
0.9475
10a
10'
10a
10"
10a
Average # Gens (log scale)
Average Total Tokens (log scale)
(d) MATH150
Figure 1I: Cost-analysis of our proposed method and fixed BoN for EXAONE-Deep-32B. The bars standard two-sigma confidence intervals. Green dashed line indicates the best-of-co performance.
32

---

Published as a conference paper at ICLR 2026
G.2 EXPERIMENTAL SET 2: ADVANTAGE LLM ENSEMBLE OVER SINGLE LLM
Figure 12 demonstrates several more examples where the ensemble of LLMs outperforms the best single LLM. The weights are optimized by the MILP introduced in Section 3. We used Algorithm to adaptively select and ask LLM for the answers.
Ensemble Adeptiive
gpt-oss-200 Aceptive Phid- Adaptive
0.90
0.90
Accuracy
0.85
Accuracy
0.85
0.80
0.75
0.75
0.65
0.65
10a
10
10a
10a
102
10a
10'
Average of Gens (log scale)
Average Total Tokens (log scale)
(a) Performance of two-LLM ensemble. We used GPT-OSS-20D and Phi 4-rssoninng AIME2025, We tested with weight (0.7, 0.3). The best-of-co performance of GPT-OSS-20B 0.900 (90.0%), whereas the ensemble's 's best-of-oo performance 0.933 (93.3%).
0.85
0.85
Ensembie Adaptiive
Adapplive
Datarus-rt Accptive
Accuracy
0.75
Accuracy
0.75
0.70
0.70
0.65
0.65
0.60
0.60
0.55
0.50
0.50
10
10'
10'
Average # of Gens (log scale)
Average Total Tokens (log scale)
(b) Performance of two-LLM ensemble. used LIMO-v2 and Datarus- R1-14B on AIME2024. The weight was optimized to w 00.4316.0 0.5684).
0.97
Ensemble Adaptive
-t Adaptivv
0.96
0.96
MetaStone Adaptive
gpt-oss-20o Adaptive
Accuracy
Accuracy
Owen3-108-AAtt Adaptive
0.94
0.94
0.92
0.92
0.91
0.90
0.90
10"
10
10
10"
Average Gens (log scale)
Average Total Tokens (log scale)
(c) Performance of four-LLM ensemble (MetaStome-S1-32B. Phi-4-reasoning. Qwen3-30B-A3B Thinking-
GPT-OSS-20B) on MATHH500. The weight was optimized w (0.0193, 0.0411, 0.623.))))).....
Figure 12: Performance of LLM ensembles compared with single-LLM performance. We used Algorithm choosing the LLM. Blue dashed line indicates the best-of-co performance of the LLM ensemble.
33

---

Published as a conference paper at ICLR 2026
G.3 EXPERIMENTAL SET 3: LEARNING A GOOD WEIGHT
Figure 13 shows several additional examples of sample efficiency of learning the optimal weights in LLM ensembles. Dashed lines the best-of-o0 performance of the individual LLMs. One can that, with a small number of gold answers, the learned weights can outperform the best single LLM.
0.875
LIMO-v2 0.800
Ensemble Accuracy
0.850
Datarus-r1: 0.733
Ensemble
0.825
0.800
0.775
0.750
0.725
10
15
20
25
30
Number of Training Samples
(a) The mixture of LIMO-v2 and Datarus R1- 141 AIME2024. Note that the best-of-co performance of the two base LLMs exactly the same and thus overlaps the figure.
0.78
EXAONE-D
Ensemble Accuracy
MetaStone: 0.712
0.76
Phi4-R: 0.727
Mmven3-30--AB5 0.732
0.74
gpt-oss-20b- 0.722
Ensemble
0.72
0.70
Number of Training Samples
(b) The mixture of Phi-4-reasoning, Qwen3-30BB B-8-AAB--hhniing 2507, and GPT-OSS-20B on GPQA- Diamond.
EXAONE-D: 0.962
Ensemble Accuracy
0.965
MeteStone: 0.950
Nemotron-Nee 0.956
0.960
Phi4-R: 0.946
Qwen3- 14B: 0.956
0.955
Owen3-30B-A335 0.960
gpt-oss-20b: 0.960
0.950
Ensemble
0.945
50
100
150
Number of Training Samples
(c) The mixture of seven LLMS on MATH500.
Figure 13: The training of weights in an LLM ensemble. We show the number of samples to determine the weight (x-axis) versus the best-of-co performance (y-axis, i.c., performance of best- of-oo with the weight). The æ-axis indicates the number of problems used to learn the weight and the y-axis indicates the best-of-co performance. The score averaged over 100 runs.
34

---

Published as a conference paper at ICLR 2026
G.4 EXPERIMENTAL SET 4: TRANSFER LEARNING OF THE OPTIMAL WEIGHT
We do not have additional experiments for this set of experiments.
G.5 EXPERIMENTAL SET 5: COMPARISON WITH OTHER ANSWER-SELECTION METHIOSS
This section reports our comparison of majority voting with other aggregation methods. This ap- pendix section complements Table 2 of main paper by providing additional experimental results on AIME2025 and other LLMs, as well as more details on the compared methods. The results are shown in Table 2. The compared methods are as follows:
Omniscient is the hypothestical selection method that can always select the correct answer it included in the candidates, which infeasible unless we know the gold answer. By definition, this is the best possible performance of any selection method.
Majority voting is the method that selects the most frequent answer among the candidates. Ties are broken randomly.
LLM-as ju-daeee is the answer selection method that uses the target LLM itself to select the best answer among the candidates. Since the concatination of the all answers can exceed the context length, extracted the last 5,000 characters before the < </think> tag of the answers for each answer." ll To avoid uninterpretable answer, we ask the LLM twice, which slightly increased the accuracy. There are two variants: (tournament) compares the answers pairwise and selects the best one, and (set) compares all answers at once and selects the best one.
INF-ORM-Llams3. 1-70 is one of the state-of- fif-the-an art reward model (Minghao Yang, 2024), which marked the 9th in the RewardBench leaderboard as of September 8 2025.
and Skywork-Reward- V2-Qwen3-8B are two of the state-of-the-ard reward model (Liu et al., 2024), which marked the 1st and the 6th in the RewardBench leaderboard as of September 2025.
-ewoo--Reward--- -2---V-a--.m..m1.mm
Self-certainty is the method that selects the answer with the highest self-certainty score (Zhao et al., 2025b), which measures intrinsic confidence by how the likelihood differs from the uniform distribution per token. Note that we used the sequence average of self- certainty. Very recently, (Fu et al., 2025) introduced a version of self-certainty that weights more on the latter part of the sequence, which we have not tested and may improve the performance.
Random is the model that randomly selects one of the candidates, whose performance should be close to the accuracy of Bol.
We use the same set of answers for comparing these selection methods, which reduces the variance due to the randomness in answer generation. Table 6, Table 7, and Table 8 show the compari- son of these methods GPT-OSS-2OB, Phi-4-reasoning, and -0-m--30B--3D--TTiinn------- respectively. The results are consistent with the main experiments in the paper. All results are Bo5 settings.
Fooo an answer without </think> tag, used the final 5,000 characters. We also tested an alternative method that asks L.I to summarize its own answer before the comparison, which, in our preliminary analysis, did not outperform the proposed method.
35

---

Published as a conference paper at ICLR 2026
Table 6: The accuracy of several selection methods on the best-of-five (Bo5) setting across three datasets (AIME2024, MATH500, GPQA-Diamond), Answers are generated by GPT-OSS-2000 The scores are averaged over trials and we report the two-sigma confidence intervals.
Method
AIME2024
GPQA-Diamond
MATH500
Omniscient
91.25 + 1.03
85.98 1.19
95.56 0.23
Majority voting
88.12 + 1.49
70.07 2.02
95.31 + 0.17
LLM-as-a-judge (set)
85.42 + 1.48
69.14 1.60
94.31 0.28
LLM-as-a-judge (tournament)
70.22 1.96
INF-ORM-LLama3.1-200
85,42 + 2.18
68.38 = 1.84
94.21 0.29
Syywof-RewarrdV22-Jlaa----
85.42 + 2.10
68.13 1.95
68.42 1.93
Self-certainty
81.67 + 2.98
67.65 1.38
93.50 0.47
Random (= Bol)
79.17 + 2.89
67.65 1.38
93.91 + 0.40
Table 7: The accuracy of several selection methods the best-of-five (Bo5) setting on the AIME2025 and AIME2024 datasets. Answers are generated by Phi-4-reasoning. Scores are av- eraged over 16 trials and we report the two-sigma confidence intervals.
Method
AIME2025
AIME2024
Omniscient
85.00 + 1.72
85.21 + 1,21
Majority voting
76.67 2.58
80.00 1.72
LLM-as-a-judge (set)
72.92 3.10
80.42 1.81
INF-ORM-LLama3
70.42 2.78
78.54 2.51
Neywor-RReaard-22--eelll
70.62 + 2.87
77.29
2.60
Self-certainty
63.12 3.36
73.54-22 2.31
Random (== Bol)
63.96 2.45
73.54 2.31
Table 8: The accuracy of several selection methods on the hest-of-five (Bo5) setting on the AIME2025 and AIME2024 datasets. Answers generated by Scores are averaged over 16 trials and we report the two-sigma contidence intervals.
Method
AIME2025
AIME2024
Omniscient
92.71 + 1.09
93.54 0.74
Majority voting
88.75 + 1.20
92.92 + 0.57
LLM-as-a-judge (set)
88.13 1.49
92.29 + 0.80
LLM-as-a-judge (tournament)
87.50 1.29
91.25 1.48
INF-ORM-Llama3.. 1--0B 1-70B
89.38 + 1.09
4 1.00
Skywork Keywork-Reward-Wo Sooward-V2-Q02 V2-Qwen3-8B
89.38 + 1.09
92.71
Self-certainty
87.50 + 2.06
91.25
Random ( Bol)
86.04 2.04
90.00 1.36
G.6 COMPARISON WITH BETA STOPPING
In this section, we compare our proposed adaptive sampling algorithm (Algorithm 1) with the Beta stopping method introduced by Aggarwal al. (2023), which state-of-the- adaptive sampling method for best-of-NV (BoN) in LLMs. The Beta stopping method uses Bayesian approach to determine when to stop sampling based on the posterior distribution the majority and the second majority. Such posterior projected to dooddmmeniooal space follows Beta distribution, and they uses the posterior probability such that the second majority exceeds the majority to decide whether stop or not. Key findings twofold. First, our proposed method dominates in the sense that method is always as good as the Beta stopping. Second, there are several case where our method outperforms the Beta stopping. Results for GPT-OSS-20B, Phi-4-reasoning, EXAONE-Deep-32) are shown in Figure 14, Figure 15, and Figure 16, respectively.
36

---

Published as a conference paper at ICLR 2026
Adaplive Acaptive Beta
0.90
0.90
Fixed
Accuracy
0.85
Accuracy
0.85
0.80
0.80
0.75
0.75
Average # of Gens (log scale)
Average Total Tokens (log scale)
(a) AIME2025
0.925
0.925
Acaptive
0.900
0.900
Fixeet
0.875
0.875
Accuracy
0.850
Accuracy
0.050
0.825
0.025
0.800
0.800
0.775
0.775
0.750
0.750
0.725
0.725
10a
10a
10
Average # of Gens (log scale)
Average Total Tokens (log scale)
(b) AIME2024
Acaptiiee Acaptive
Fixed
0.70
0.70
Accuracy
Accuracy
0.68
0.68
0.66
0.66
0.64
0.64
10
10a
10a
10
Average # of Gens (log scale)
Average Total Tokens (log scale)
(c) GPQA-Diamont
Adsptive
0.955
0.955
Adaptive
Rieed
0.950
0.950
Accuracy
Accuracy
0.945
0.940
0.940
0.935
0.935
0.925
0.925
10"
10
10a
10
10
Average of Gens (log scale)
Average Total Tokens (log scale)
(d) MATH150
Figure 14: Cost-analysis of our proposed method ("adaptive"). Beta stopping (Aggarwal et al., 2023), and fixed BoN for GPT-OSS-20B. The error bars are standard two-sigma confidence inter- vals. Green dashed line indicates the best-of-oo performance.
37

---

Published as a conference paper at ICLR 2026
0.85
0.85
Adaptive Acaptive Beta
Fived
Accurecy
0.70
0.70
0.65
0.65
10
Average # of Gens (log scale)
Average Total Tokens (log scale)
(a) AIME2025
Acaptive
0.850
0.850
Fixeet
Accuracy
Accuracy
0.800
0.800
0.750
0.750
10a
10a
Average # of Gens (log scale)
Average Total Tokens (log scale)
(b) AIME2024
Acaptiive Acaptive
0.73
0.73
Fixed
Accuracy
0.69
0.69
0.68
0.68
0.67
0.67
0.66
0.66
10
10.
10
10a
10*
Average # of Gens (log scale)
Average Total Tokens (log scale)
(c) GPQA-Diamout Diamond
0.95
Acaptive
Acaptive
0.94
0.94
Accuracy
Accuracy
0.93
0.92
0.92
0.90
0.90
10c
10b
10a
10a
10a
Average of Gens llog scale)
Average Total Tokens (log scale)
(d) MATH150
Figure 15: Cost-analysis of our proposed method ("adaptive"). Beta stopping (Aggarwal et al., 2023), and fixed BoN for Phi-4-reasoning. The error bars are standard two-sigma confidence inter- vals. Green dashed line indicates the best-of-oo performance.
38

---

Published as a conference paper at ICLR 2026
Adaplive Acaptive Beta
Fixed
0.76
Accuracy
Accuracy
0.74
0.74
0.72
0.72
0.70
0.70
10
Average # of Gens (log scale)
Average Total Tokens (log scale)
(a) AIME2025
Acaptive Adapptive Fixee
0.88
0.88
0.86
0.86
Accuracy
Accuracy
0.80
10a
10a
Average # of Gens (log scale)
Average Total Tokens (log scale)
(b) AIME2024
Adaptive Roed
0.695
0.695
Accuracy
0.685
0.685
0.680
0.680
0.675
0.675
0.670
0.670
10"
10
10a
10a
10"
Average # of Gens (log scale)
Average Total Tokens (log scale)
(c) GPQA-Diamous Diamond
0.964
0.964
Adsptive Adaptive
0.962
0.962
Rieed
Accuracy
0.958
Accuracy
0.958
0.956
0.956
0.954
0.954
0.952
0.952
0.948
0.948
10"
10
10a
10a Average Total Tokens (log scale)
10a
Average of Gens (log scale)
(d) MATH150
Figure 16: Cost-analysis of our proposed method ("adaptive"), Beta stopping (Aggarwal et al., 2023), and fixed BoN for EXAONE- MHONNE-eepp-22B. The error bars are standard two-sigma confidence intervals. Green dashed line indicates the best-of-oo performance.
39

---

Published as a conference paper at ICLR 2026
H ORACLE ADAPTIVE SAMPLER
Finally, we compare the performance the proposed method with "oracle adaptive sampler" that stops when least one of the samples are correct reaches N Nmax. Of course, such an adaptive algorithm requires an oracle (omniscient) verifier (Brown et al., 2024; Wu et al., 2024) that requires the knowledge of the gold answer and is unavailable in practice. Too see the performance of such oracle adaptive sampler, take an example of GPT-OSS --T-OOS--0BBB on the AIME2025 dataset (Figure 14 (a)). For example, an oracle adaptive sampler with N Nmax 5 has average N 1.6 and accuracy 2 0.90, which, by large margin, exceeds the performance of the proposed algorithm. This is reasonable given that oracle does not need to depend on majority and one correct sample suffices to stop.
I ADDITIONAL DATASETS AND EXPERIMENTS
To further verify the robustness of the results, we tested two additional datasets.
The MedRECT dataset (Iwase et al., 2025) is a cross-lingual dataset that measures the ability of LLMs to detect and correct errors in clinical texts, where reasoning capability improves accuracy. In particular, we used the Japanese dataset which consists of 663 texts on medical diagnosis. Following Iwase et al. (2025), extracted 367 texts that contain misdiagnoses and asked the LLM to detect the number of sentences that require correction.
The IMOBench dataset (Luong et al., 2025) contains 400 mathematical questions extracted from International Mathematical Olympiad (IMO) problems." 13 Unlike AIME, answers in this dataset may have multiple valid expressions (e-g.. "1/2" equivalent to " "\rraell")) and our answer normalizer is imperfect. For evaluation, we used only the subset of questions for which at least one of the 100 generated answers matched the gold answer.
The results on these two datasets, which are consistent with our other experimental findings, are shown in Figure 17.
seeredt-ji-sttep--ccppeee.. json
Phttps://githan wwtp//////oooooo hta / http:///wwwa..a.aaaaa///////aaaa
answerbench_v2.com
40

---

Published as a conference paper at ICLR 2026
0.76
0.76
lixed
Accuracy
0.72
Accuracy
0.72
0.70
0.70
0.66
0.66
10"
10a
Average # of Gens (log scale)
Average Total Tokens (log scale)
(a) McdRECT
Adaptive
0.62
0.62
Accuracy
0.60
Accuracy
0.60
0.56
0.56
0.54
0.54
0.50
0.50
10"
10a
10a
10*
Average of Gens (log scale)
Average Total Tokens (log scale)
IMOBench
Figure 17: Cost-analysis of our proposed method and fixed BoN for GPT-OSS-20B. The error bars are standard two-sigma confidence intervals. Green dashed line indicates the best-of-oc perfor- mance.
41


Reviews:
[{"review_id": "JkEMf3gJr9", "reviewer": "Reviewer_CXnE", "summary": "The apparel provides an early stopping mechanism for sampling in the majority-voting algorithm in the LLMs to achieve the same performance as the infinite-budget case with finite generations. The mechanism maintains a Drichlet Process posterior over output distribution and stops according to the Bayes factor. Also, the paper studies the optimal weighting for LLM ensembles in the infinite budget case.", "questions": "1. I could not follow the line 117 derivation and would appreciate more explanations. There might be typo there. To me, it should be\n\n$$\n\\mathbb{P}(H_1 \\mid \\mathcal{D}(n)) \\approx \\Pr[X_1 \\ge \\max_{i \\ne 1} X_i,\\, X \\sim \\mathrm{Dirichlet}(N_1, N_2, \\ldots, N_{s(n)}, \\alpha)].\n$$\n\nAlso, the assumption is that the probability of all $A_i$ together (instead of just A1) is zero under H.", "limitations": "", "rating": 2, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is written rigorously with an accurate language. The paper is mostly easy to follow.\n\nThe introduced early stopping mechanism can be very impactful in practice \n\nI find the ensemble problem very interesting and novel. \n\nThe experiments are extensive and cover many models and benchmarks. The algorithms work well across all settings.", "weaknesses": "I mostly have two major concerns about the paper;\n\n1. It appears to me that the Drichlet stopping criteria highly resembles the work of [1]. I am not aware of the full details of that paper and would appreciate a comparison by the authors. However, it looks like both model the problem as a dirichlet process and stop using a threshold. This reduces the novelty of this part of the paper.  \n\n2. The ensemble method seems to depend on the knowledge of the ground truth answers. The matrix in the constraint in equation 13 depends on the ground truth answer. If this is true, it severely reduces the practicality of this part of the paper. \n\n\nMore minor issues:\n\n3. In a footnote, It is mentioned that the method can be applied to infinite possible final answers (which is the case in MATH500) problems. But it breaks the approximation in equation 6. More clarification about the applicability to this setting strengthens the paper. \n\n4. As a reference point, it would be informative to compare the stopping criteria with an oracle one. The oracle could be stopping at the moment the output becomes and remains correct. \n\n\n\n[1] [Let’s Sample Step by Step: Adaptive-Consistency for Efficient Reasoning and Coding with LLMs](https://aclanthology.org/2023.emnlp-main.761/) (Aggarwal et al., EMNLP 2023)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ut4mibK3iG", "reviewer": "Reviewer_6Yj4", "summary": "This work investigates test-time scaling of large language models (LLMs) via majority voting. The authors analyze the infinite-sample limit of accuracy and propose an instance-level dynamic compute allocation method based on answer agreement. They also consider weighted ensembles of multiple LLMs and formulate a mixed-integer linear program to determine optimal weights. Experiments are conducted to validate the proposed analysis and methods.", "questions": "- How is asymptotic accuracy computed in the experiments? In the first plot of Figure 1, there is a noticeable gap between asymptotic accuracy and empirical accuracy at $N$ as large as 1000; how should this be interpreted?  \n- Regarding terminology: literature convention generally considers majority voting and best-of-N as separate methods (as far as I can tell), whereas this paper treats majority voting as a special case of best-of-N. Nonetheless, I do not object strongly to this choice if other reviewers think this is fine.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- The proposed dynamic compute allocation method based on answer agreement is sensible, intuitive, and straightforward to implement.  \n- The release of a dataset containing LLM-generated responses may be useful to researchers studying LLM test-time scaling.", "weaknesses": "- The procedure for optimizing the weights in the LLM ensemble is unclear. In particular, it appears that the same data is used for both optimization and evaluation, which would amount to \"training on test data\" and thus compromise the validity of the results.  \n- Some theoretical results (e.g., Theorem 1 and Lemma 1) are relatively obvious and could be better presented by briefly summarizing the key insights in the main text, with full details and formal notation moved to the appendix.  \n- \"Experiment Set 5\" on page 9 seems out of place, as it compares standard majority voting with other test-time scaling methods, none of which is directly related to the proposed methodology or analysis.  \n- The discussion on asymptotic performance may give the impression that majority voting's accuracy increases monotonically with the number of samples, which is not always correct. This paper misses an important related work [1] demonstrating that the accuracy curve can exhibit diverse behaviors.  \n\n  [1] Are More LLM Calls All You Need? Towards the Scaling Properties of Compound AI Systems (NeurIPS 2024)  \n\n- Minor issues:  \n  - Line 184: replace “the algorithm” with “Algorithm 1.”  \n  - Footnote 4 on page 5 seems redundant and could be removed.  \n  - Table captions should be placed above the tables, per ICLR format guidelines.  \n  - Line 482 contains a grammatical error: “we have verified robust across LLMs and problem sets”.  \n  - Table 6 has missing entries without explanation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2lBeVJules", "reviewer": "Reviewer_XSCw", "summary": "This paper studies adaptive repeated sampling methods for more compute-efficient test-time scaling. This paper also proposes a new approach to learning optimal weighting for multi-LLM ensemble based on mixed-integer linear programing.", "questions": "- Can you give some examples of the base distributions $H$ when the answer space is infinite?\n\n- The MILP in Lemma 4 assumes access to $p_{i,j}^q$. How is that implemented in practice?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The research topic is important and well motivated.\n\n- The proposed methods are grounded with principled analysis.\n\n- The experiments show that the proposed methods lead to performance improvement.", "weaknesses": "## Main weaknesses\n- I find some terminologies in the paper confusing or inappropriate:\n  - Test-time compute: Whenever an LLM is used in the test-time, the compute should be categorized as test-time compute. Based on my understanding, this submission indeed focuses on test-time scaling (TTS), or different methods of scaling up test-time compute. Therefore, test-time scaling should be the right term to use (e.g., in the title), and this terminology has been widely adopted in existing papers [1].\n  - Best-of-$N$: In early works on test-time scaling, best-of-$N$ specifically refers to selecting the best generation trajectory with an external verifier -- either an oracle or a learned verifier [2,3]. \"Majority voting\", to the best of my knowledge, is a different method from best-of-$N$ in existing studies [3,4]. After all, majority voting cannot select the single \"best\" response from $N$ candidates -- it just returns an output answer based on consensus. The focus of this paper is purely on majority voting and I cannot agree that this is a \"variant\" of \"best-of-$N$\". Besides, I think [2,3,4] -- [4] in particular -- are all very relevant to this work and should be discussed.\n\n[1] s1: Simple test-time scaling\n\n[2] Large Language Monkeys: Scaling Inference Compute with Repeated Sampling\n\n[3] Inference Scaling Laws: An Empirical Analysis of Compute-Optimal Inference for Problem-Solving with Language Models\n\n[4] Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systems\n\n- The exposition of the method part lacks clarity:\n  - Algorithm 1 is not even self-contained. The input $\\alpha$ is not explicitly used. Moreover, the approximation of $\\mathrm{BF}(n)$ lies in the center of the analysis in Section 2. The algorithm needs to elaborate on it rather than calling the function without any explaination.\n  - Line 182 mentions that $\\mathbb{P}(H_1\\mid \\mathcal D (n))$ can be estimated using Monte Carlo methods. More details on the method are anticipated, especially when the answer space is large or even infinite.\n  - The \"algorithm\" in Theorem 1 is very vague. Is it the presented Algorithm 1? Is $\\mathrm{BF}(n)$ approximated? Should the result also rely on a large number of Monte Carlo samples?\n  - In Equation (8), the notation is wired. People naturally expect $f(\\\\{a_q\\\\})$ is defined for a single problem $q$ but it is indeed for the entire problem set. Why not denote it by $f{\\mathcal Q)$?\n\n\n## Nits\n- Lines 28-30: `\\citet` $\\to$ `\\citep`.\n- Line 175: When $n$ is sufficiently large compared with $\\alpha$ $\\to$ When $n\\gg \\alpha$.\n- Line 201: AIME problem solutions can be 0.\n- Line 222: The \"=\" should be $\\in$ since the maximizer is not necessarily unique.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "muybRgNdS3", "reviewer": "Reviewer_jczD", "summary": "This paper studies a test-time-scaling instance, where N answers are first generated by 1 LLM, and then one answer is selected by majority vote. The authors propose and analyze an algorithm that adaptively adjust number of answers to mitigate the overall LLM generation cost, which later is extended to the case where the N answers are generated by multiple LLMs (i.e., an LLM ensemble). Experiments using real-world LLMs are conducted to justify the results.", "questions": "- Where is the finite-sample analysis? Theorem 1 analyzes the asymptotic performance of Algorithm 1, i.e., showing it converges to best-of-infinity when the budget goes to infinity. This is not quite useful, as the standard majority-vote using N samples also converges to best-of-infinity as N goes to infinity. What is useful is the finite-sample analysis, i.e., the convergence rate of Algorithm 1 compared with majority-vote with the same number of samples.\n\n- Why is the choice of Dirichlet process in Algorithm 1? As far as I understand, the purpose of Dirichlet process in Algorithm 1 is to offer a prior distribution for uncertainty estimation. There are several possible prior options, and I am not convinced that Dirichlet process is the simplest or optimal. At least one should provide some ablation study on the choices of the prior distribution.\n\n- \"The equation 8 is equivalent to the following MILP\". How could an equation be the same as an optimization problem? Do the authors mean \"maximizing equation 8\"?\n\n- Many closely related papers are completely ignored. For example, [J. Li et al 2025] and [L. Chen et al 2025] have already studies the case of majority vote with many generations and offered detailed analysis and empirical experiments. A. Huang studies how to optimize the best-of-n approach from a theoretical aspect. F. Wang studies almost the exactly same problem and proposes a similar algorithm to improve sample efficiency. The novelty and technical contributions cannot be easily accessed without clearly positioning the paper in the fruitful literature. \n\nJ. Li et al, More Agents Is All You Need, 2025.\nL. Chen et al, Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systems, 2025.\nA. Huang et al, Is Best-of-N the Best of Them? Coverage, Scaling, and Optimality in Inference-Time Alignment, 2025.\nF. Wang et al, DynScaling: Efficient Verifier-free Inference Scaling via Dynamic and Integrated Sampling, 2025.\n\n- The empirical results are based on only 3 datasets, namely, AIME(2024+2025), GPQA, and MATH500. While relevant, GPQA and MATH500 have been released for a while and often considered contaminated. AIME (2024 or 2025), on the other hand, only contains 30 questions, making it extremely hard to draw any significant conclusions. Large-scale experiments are essential for statistically significant conclusions.\n\n- What is the training runtime cost? Obtaining the optimal weights for LLM ensemble is via solving an MILP, which can be extremely expensive. What solvers are used in the experiments? How fast/slow are them?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "- Test-time-scaling is a timely topic, and its research impact may potentially last for a long time.\n\n\n- The paper is well-written and easy to follow.\n\n\n-The proposed majority vote approach is generic enough to be applied to a range of problems.", "weaknesses": "- The technical contributions are limited. The main contribution, Algorithm 1, simply views the LLM generated answers as samples from some unknown distributions and then estimates the uncertainty from the drawn samples to decide whether to go further or not. The theorems and lemmas all seem very straightforward too. See my questions for more details.\n\n- Both majority-vote and LLM ensemble has been extensively studied in the literature. However, the paper ignores many (important) papers and fails to position itself appropriately. See my questions for more details.\n\n- Motivations with open-source models: One main reason developers and researchers would use majority-vote is black-box access to the LLM models. Otherwise, instead of majority vote over multiple answers, one can simply fine-tune the model weights to optimize the performance. Hence, showing the performance with black-box LLM models (e.g., GPT-5 or at least GPT-5-mini) is essential to demonstrate the practical usefulness of Algorithm 1.\n\n- The empirical results appear to be statistically insignificant and outdated due to dataset selections, and thus is hardly convincing that the proposed Algorithm 1 is efficient and effective.  See my questions below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)