Reasoning Planning for Language Models
Authors: Bao Nguyen
Hieu Trung Nguyen
Ruifeng She
Xiaojin Fu
Viet Anh Nguyen
Venue: NeurIPS 2025 spotlight ()
Keywords: reasoning method selection
two-tower framework
accuracy-cost trade-off
inference time scaling
Primary area: deep_learning

Abstract: Selecting an appropriate reasoning method for a given query remains a key challenge in language model generation. Existing approaches typically generate multiple candidate responses and use an aggregation strategy to select the output answer, often assuming that more candidate answers yield higher accuracy. We revisit this assumption through a rigorous theoretical analysis, deriving accuracy bounds for standard aggregation methods under fixed generation distributions and candidate sizes. Building on these insights, we introduce EPIC, an Ensemble Planning with Contrastive learning framework to learn a shared representation space that captures both model reasoning abilities and query-method compatibility. EPIC incorporates our probability bounds as a regularizer in a utility-driven optimization that balances accuracy and computational cost. Experiments on diverse mathematical reasoning tasks show that EPIC consistently selects optimal reasoning methods, improving accuracy while reducing computational overhead. Our code can be found at https://github.com/nguyenngocbaocmt02/EPIC.

Reasoning Planning for Language Models
Bao Nguyen'
Hieu Trung Nguyen'
Ruifeng She?
Xiaojin Fu?
Viet Anh Nguyen'
The Chinese University of Hong Kong
2 Huaweii Noah's Ark Lab
nbnguyenese. cwh.. hk, thnguyen@se. www.ieeeeeucc...... edu.
she. ruifeng@huawei...... com, fuxiao in32@hotmail...mmmmmmmmammmmm com, nguyendse. cuhk. edu. hk
Abstract
Selecting an appropriate reasoning method for a given query remains a key chal- lenge in language model generation. Existing approaches typically generate multi- ple candidate responses and use an aggregation strategy to select the output answer, often assuming that more candidate answers yield higher accuracy. We revisit this assumption through rigorous theoretical analysis, deriving accuracy bounds for standard aggregation methods under fixed generation distributions and candidate sizes. Building on these insights, we introduce EPIC, an Ensemble PlannIng with Contrastive learning framework to learn a shared representation space that captures both model reasoning abilities and query-method compatibility. EPIC incorpo- rates our probability bounds as a regularizer in a utility-driven optimization that balances accuracy and computational cost. Experiments diverse mathematical reasoning tasks show that EPIC consistently selects optimal reasoning methods, improving accuracy while reducing computational overhead. Our code can be found httppp:// http:///gatuuu...ooongnggegeeee/
com http:////.//oooooocoooooo/////C/
Introduction
Large Language Models (LLMs) have demonstrated remarkable abilities to understand and reason in human natural language. These advancements have transformed applications, including travel planning [Xie et al. 2024a]. Al teaching platforms [Jin et al. 2024). and human population simula- tions (Park al.) 2023 Bui al. 2025). However, even with pre-trained LLM,, the computational expense of serving LLM-powered systems remains significant bottleneck due to the massive scale of the models [Lin et al. 2004), the quadratic complexity of the attention mechanism [Dao et al.)
2022], and the token-by-token nature of auto-regressive generation [Zhang et al. 2025 This high computational cost significantly hinders the broader application of L.LMs in practical scenarios, particularly in resource-constraing environments such edge devices, real-time applications, and small-scale businesses.
This challenge becomes even more pronounced in tasks that require advanced reasoning, such as automated theorem proving [Wu al. 2022 mathematical problem solving [Trinh al., 2024). code generation Diang et al., 2024 Li et al. 2025a], or heuristic discovery (Romera-Paree Paredes et al.) 2024]. LLMs often fail to produce accurate responses these scenarios single pass. Instead, they rely on iterative generation strategies combined with aggregation search techniques, such as best-of-N sampling [Stiennon et al., 22220) or Monte Carlo Tree Search [Xie et al., [024b). to refine and select the most appropriate response. Throughout this paper, we refer to these iterative strategies as Reasoning Methods.
A key limitation of current approaches lies in their static application of reasoning methods, where the same technique is applied uniformly across all user queries. However, not all reasoning methods
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

equally effective or efficient for every query. This observation leads to our central research question: Could we select the most suitable reasoning method for given user query to balance the trade-off between accuracy and efficiency before generating the answer?
As a starting point, consider universe of methods, denoted by M. Each technique in M is formally characterized by a tuple (LM, ReStrat, Agg, Config, N),, where LM denotes the base language model, ReStrat denotes the reasoning strategy (e.g. Monte Carlo Tree Search, Beam Search, Best-of-N), Config is collection of relevant configuration parameters (e.g., temperature of a sampling-bast based decoding strategy), Agg denotes an aggregation technique (e.g., majority voting or score-based voting), and N is the number of candidate answers for aggregation. Importantly, this formulation is broad enough to subsume a wide variety of test-time compute methods [Snell et al. 2025], ranging from simple prompting techniques (Wei et al., [202 Yao ct al. [023] Brown et al.
2020] and standard decoding strategies [Xie et al. 2024b| Wang et al. [022] to more specialized intervention-basee approaches [Li et al., 2023 Nguyen et al. 2025a bj. However, in this work, we
focus on a representative subset of methods rather than exhaustively covering the entire space.
Contributions. We introduce EPIC, an Ensemble PlannIng with Contrastive learning framework that recommends matching an input question and an appropriate reasoning method in the universe of methods M. EPIC learns jointly the embedding of each reasoning method and a neural mapping from the input question the embedding space. Two main components guide the learning process:
contrastive loss, which pulls the question embedding towards the reasoning method with the highest utility for that question. The utility value is composed of a weighted combination of the accuracy and the inference cost, measured by the number of tokens generated. The user controls the accuracy-cost trade-off through a scalar parameter, balancing the preferences across different confficting deployment criteria.
a regularizer term, which exploits the commonality among methods that share four components (LM, ReStrat, Config, Agg). but differ only by the number of candidate answers N. This regularizer aims to improve the sample efficiency of the training procedure by grounding these methods relatively on the scale of N.
At inference time, EPIC maps the test-time input question to the embedding space and selects the reasoning method with the highest similarity (or scores) for answer generation. Extensive experiments on the MATH dataset demonstrate EPIC's advantage: compared to individual reasoning models in the universe of methods, EPIC can reduce the number of tokens (or cost) by 75% while maintaining the same level of accuracy.
Our paper unfolds as follows: Section discusses related work on LLM reasoning. Section studies the probabilistic bounds of common aggregation methods. Section 4 delineates our EPIC framework for matching reasoning methods with input questions, and Section presents the extensive numerical results of the mathematical reasoning task.
2 Related Work
We review advances in LLM reasoning algorithms and inference-time scaling, highlighting their emerging impact on output quality and computational efficiency.
Reasoning algorithms and inference-tine scaling, A naive reasoning process may not generate the correct solutions for complex reasoning tasks. To identify and choose the correct solution within the distribution, Self-Consistency (SC) samples multiple outputs from the LLM and selects the final response by majority voting [Wang et al. 2022). Another similar approach is best-of-N sampling, which uses a reward model or function to choose the answer with the highest reward [Stiennon et al., 2020). Both methods enhance the quality of the output, but increase the computational cost by a factor of sampling times. To explore potential reasoning paths, tree-search-ber
methods are proposed, such as Tree-of- of-Thughh Yao et al. 2024 J. Monte Carlo Tree Search (MCTS) (Wan et al. 2024 (Bi et al., 2025 Damani et al. 2025
Zhang et al.) 2024 Guan et al. 2025| Forest-of-
indicates that searching over a tree structure is more effective in discovering a correct solution than
simply sampling responses in parallel for more complex tasks.
Despite applying different reasoning methods to problems with various levels of complexity, inference- time scaling these methods also significantly improves the output quality. [Beeching et al. (2024)

---

demonstrates that the accuracy on the MATH-500 benchmark improves as the amount of test-time computation (number of generations per problem) increases for algorithms such as best-of-N, beam search, and diverse verifier tree search (DVTS). Guan et al.] (2025) conduct extensive MCTS rollouts and achieve an average accuracy of 53.3% on questions of AIME24 benchmark.
Cost-effective reasoning. Although inference-time scaling significantly enhances LLM's reasoning capabilities, this approach incurs substantial computational overhead and often leads inefficient use of computational resources, Recent work finds that performance gains from various inference-time scaling strategies exhibit significant variability across different levels of prompt difficulty [Snell] et al. 2025). Drawing from this evidence, they effectively allocate inference- teeence--mmeeeee compute according to question difficulty, with four times less computation than the best-of-N baseline. However, the method incurs considerable computational costs to assess question difficulty. Damani et al. [2025] train lightweight probes built upon I.LM's hidden representations to quickly predict allocating more computation to a question will improve the response quality. To efficiently scale best-of-N sampling, Manvi et al. (2024) introduces a highly cost-effective self-evaluation paradigm that does not rely external reward model, incurring costs only from generating a single token.
Whereas most studies focus on effectively and efficiently scaling a particular reasoning algorithm, we focus on pairing suitable reasoning methods with various questions, considering both accuracy and cost. We conduct our study based on OpenR [Wang et al., 2024a] J. an open-source framework for LLM reasoning that integrates multiple strategies, including greedy decoding, best-of-N, beam search, and MCTS.
3 Probabilistic Analysis of Aggregation Accuracy
We observe that many methods in the universe M could share common features: they could use the same base language model, reasoning strategy, configuration parameters, and aggregation methods, and they could differ by only the amount of test-time compute, or how many samples N they need to generate before aggregation. To exploit this information, we first need to understand how different sample sizes N affect the quality of the output. We analyze the probabilistic performance of an aggregation method for a specific question q as the number of samples N varies. All probability quantities in this section are conditioned on g, but this condition is omitted avoid clutter. Let Y be a random variable representing the final answer extracted from a sampled solution to a question generated by model. Importantly, Y refers specifically to the final answer, not the reasoning process or steps leading to it. In practice, the model is trained to enclose Y in a LaTeX box to The
make extraction easier. We suppose that the support set of Y is finite: y = (y1 ,e,.,,,....,.2. stochastic generation process of a model specified by the tuple (LM, ReStrat, Config) produces probability distribution over Y:
K Σ
Pr(Y yk) Pk: where
= and pk
Vk.
Without any loss of generality, we denote y1 3 the only correct answer to the question g. After sampling N independent samples following the above distribution, an aggregation method Agg is applied to obtain the output answer. We focus characterizing the probability that the output answer is y1, which means that the output answer is a correct solution to question
3.1 Majority Voting
Given N samples generated by the language model, majority voting counts the frequency of each unique answer among candidate answers. Then it outputs the answer with the highest count as the output answer. We refer to this aggregation method as Majority_Von Vote. We have the following result.
Theorem 3.1 (Majority voting). pf > ps for all ks .,..., K, then
K
Pr(Majority_Vor Vote picks yn) 2 1-
Σe k=
N NNPPi vpx))
(1a)
If Pt < for some l = 2.... K, then
Pr(Majority_Vors picks v1) S
(lb)
3

---

Note that the bound aaa approaches 1 as N + co, which implies perfect accuracy. In contrast, the bound (bb) approaches N + oc, implying complete failure.
3.2 Aggregation using Summation of Scores
Given N samples generated by a model, we first pass them through a reward model to obtain a reward score for each sample. Two popular types of reward models are Outcome Reward Models (ORM) [Cobbe et al. |021 Yu et al. 2023), which provide a single scalar reward for each complete solution trajectory, and Process Reward Models (PRM) [Luo et al. 2024] Lightman et al. 2023a]] which provide step-by-step feedback and aggregate it, typically by summing or taking the minimum, obtain a final score for the sample. While PRM is used throughout this work, our method is flexible and can be applied with any reward models.
For each unique answer, final outcome is selected as the answer with the highest total (summed) reward score across all samples [Wang al., 2024a Li et al. 2022]. We suppose that PRM returns a score for answer yu following a Gaussian distribution N (PR, of) for all k. We call this aggregation method PRM_ _ote. We have the following result.
the PRM scores of all samples that generate that answer. The
for all k = ....., K, then Theorem 3.2 (Voting with score sum). IfPI > for
Pr(PRM Vote picks vn) 2 1- Σ inf exp (NDi (e-0eunt + Nps (cumettlioz -1)
1200
k=2
(2a)
pij < PaPkk for some k 2,..., K, then
Pr(PRM Vote picks v1) < inf exp (NDR (e-maet(PoR + Np (clmnt)foi-1) (2b)
120
All infimum problems in 2 are convex optimization problems. While no analytical expression for
the optimal value & available, we could tractably find 4 for each term using Newton's method. Moreover, we could observe a similar conclusion as N tends to infinity: the bound (2a) approaches 1 while the bound (2b) approaches O.
3.3 Aggregation using Maximum of Scores
given N samples, we use the PRM to assign reward score to each sample. For each unique answer, take the maximum PRM score among all samples that produce that answer. The final prediction is the answer with the highest such maximum. We suppose that PRM returns score for answer y/s following Gaussian distribution A (MU,, ( for all k. We call this aggregation method PRM Max. We have the following result.
This aggregation method follows the same setup as in Section 3.2
Theorem 3.3 (Voting with score maximum). Let
(x) = +
p4k
3: 11 1, K,
ok
where t is the cumulative distribution function of the standard normal distribution. If ok for all k ......K K, then
Pr(PRM_Max picks v1) 2 Σ inf
{1- 11 (1)) +1 -1- Pa (1))
(3a)
ER
If or > for some k = 2,... K, then
Pr(PRM Max picks vn) < inf EER (1 (x,))))) +1 - (1- (11-20
- (1 (01)
(3b)
All infimum problems in 3 are one-dimensional and can be efficiently solved using standard numerical methods. Moreover, we observe similar asymptotic behavior as N increases: if a > on for all l: 2...., K, the bound in (3a) approaches 1 as N - co, while if os > on for some k, the bound in 3b approaches 0 as N oo.

---

Ensemble Planning with Contrastive Learning
Given a universe of methods M = {1, .I..... M) consisting of M reasoning methods in total, EPIC aims to create an ensemble model on M that assigns to any imput question from the test environment an appropriate method E .M that could deliver desirable accuracy-cost trade-off. We first discuss our modeling of the accuracy-cost trade-off in Section 4.1 then we describe the training phase and inference phase Sections and
We conclude this section by discussing our design choices.
4.1 Accuracy-Cost and Utility
We possess training dataset of question-anss pairs denoted as D {2z, V/ fj=1+ where zj a question statement, and yj is the corresponding true answer. In the training phase, we deploy reasoning model 4,, E M, to each question zj. The generated solution is Px We record whether (x(z)) is accurate by comparing it to the ground-truth answer V/j, and obtain the accuracy signal
aijj Accuracy(P,(zx)).u My) E 10.
(4)
If D is deterministic method. then is simple binary indicator Accuracy(4:.) (z,), y3) (z,, = yj). When & is a stochastic method, then we average the accuracy over five seed numbers to get percentage accuracy. The value aij indicates whether method i succeeds in answering question j. Moreover, we also record how many tokens the method i E .M costs to generate the answer. This token count is denoted by č.j > O. Because di [0, 11, normalize the token count by passing č. through a non- non-decreasing function Φ R2 + R_, then dividing by the maximum transformed cost to ensure that the cost CLj E [0, 1] has the same scale with dz
((C,))
Cr max
To balance cost and success rate, we establish the utility function that is the convex combination of accuracy 0z,j and normalized cost Ci.j as
u(ar,, c..)) = Aa., + - (1-1) () - c..),
(5)
where A E [0, 1] is a trade-off parameter, and the utility admits a value between 0 and 1. If À = 0, then u(ai,j, Ct,5) = 1 1-c,, which implies that the utility depends only the generation cost. In this way, we tend to favor the cheapest reasoning method, regardless of how effective it is at generating accurate answers. On the other end of the spectrum, when À u(ai,j, G.j) aijz implying that the utility is entirely derived from the accuracy. In this way, we tend to favor the most powerful reasoning method, regardless of its cost. To simplify the notation, omit the parameter X, and use the shorthand u.j = u(ai.j Ci.)).
The product of the data preparation process is processed dataset {æj, (ur.)) ia containing the training question and the corresponding utility of each reasoning method for that question. This dataset will be used in the subsequent contrastive learning process.
4.2 Contrastive Representation Learning with Probability Regularization
We now describe the core component of our framework that matches the input question with the appropriate reasoning method. We represent each question zj in the training dataset by its features fj DRD A lightweight neural network go R Rd maps each question feature vector fj to produce a dense embedding Go(f)) ) in a d-dimensional vector space. EPIC aims for an information compression with d < D. Moreover, each reasoning method i .M is assigned a trainable embedding vector i E Rd, which is the same dimension as the question embeddings. EPIC uses simple multi-layer perceptron for 0.
We now train the question embedding network parameter and the method embedding vectors V; jointly. One component in the training loss is the popular contrastive loss function, InfoNCE loss [Oord et al. 2018). We identify a positive method for each question zy. denoted as m (2₂). Given the utility values defined in can identify the method with the highest utility for question zy:
ml (Iz) arg max u(ai.j Ci,5), iEM
5

---

Evellate Jops
(e(-)
Easy
Medium Problem
Hard Problem
ge
two
Find constant larm the reparssoon
cirdles intersect?
gn(-)
10
2x
Reasoning Embeddings
1/1
172
VM
Reasoning
Reason method
method
Figure 1: Our method employs the regularized representation learning loss ( to learn both the reasoning method representation vectors, denoted as 01 VM and the question embedding network parameters 0. During inference, route suitable math questions to the appropriate reasoning method by computing the similarity between the input questions and the learned method representations. Color codes on problem difficulty levels are provided for illustration purposes only.
This leads to the contrastive loss component:
exp(s(go({3) Um,) Σiem exp(s(eo (fz), u;))
Lcontrastive (6, U1, PM) =
log
(6)
j=1
Above, is a similarity score function that measures the similarity of a question embedding ge (fz) with the method embedding v;. Standard choices for are the dot product similarity measure the negative 2-norm. The contrastive component (6) aims to pull go close to the positive method (m,)) ). and push go (Jş) far away from the negative methods * m4(x)) The loss in (6) is the categorical cross-entropy loss of classifying the positive method, with the fraction inside the logarithm being the model's prediction.
The second component of the loss function regularization term:
Two methods that share the same tuple (base model, reasoning strategy, aggregation technique and configuration) but differ only by the compute budget N should conform to a relative performance metric because they both inherit the same stochastic generator. We postulate the following regularization term:
target 2 target,,
((g((f))) s(go (fj),
freg (0, Ui...., UM)
Σ
(K.')MM (6,2") differ only
This regularizer promotes the fraction of the similarities to be close to the fraction of the target quantities. Ideally, we should use target): = Pr(mmthood picks the correct answer for question æ;). which is the intrinsic characteristic of the stochastic generator. However, this probability value is not readily available, therefore we leverage the bounds Section 3 as target values, and empirically compute these target values follows: For each question j and core configuration (generation method, temperature, aggregation method, etc.), we generate 80 solutions (5 independent runs of N 16 with different seed numbers) to obtain set of distinet solutions y1,--.; ....----KK. VKK We then estimate the parameters Pt: A & from these 80 solutions. We can then empirically identify whether the lower upper bound of the probability is active and assign the target value as either the lower or upper bound with the corresponding size N. The bound provided in Theorem 3.3 is valid when N is large enough. For smaller N. we use the empirical accuracy as alternative.
Combining two loss terms 6 and 7 we obtain the training problem
min 8
min UM....
fooutrastive (0, (0,01... U1...., vM) + Threg (0, 01, UM),
(8)
where > 0 is hyperparameter aiming to promote the sample efficiency of the training procedure.

---

4.3 Inference Time Matching
At inference time, we pass any new question Knew, or equivalently its feature vector fnew through the trained network go to obtain the question embedding go (.new). We then find the top-1 reasoning method by m arg maxie. s[go(nnew); vi), that maximizes the similarity score between the question and the trained representation vector 01 UM of the reasoning models. We then deploy method m to answer this question.
4.4 Discussions
We now discuss the necessity and importance of the design choices of our EPIC method.
Discussion (Questions' feature vector). There are multiple ways to obtain the feature vector f for each question I,, For example, we can take fy the activation of the last token of 2, extracted from one of the layers (potentially the last layer) of the language model. This approach does not incur any additional memory requirement because we do not need to load any auxiliary models onto the device. However, the activation dimension of the language models is usually high: for example, in Qwen2.5- ---.55--M----B-hhth IB-Isrruttt (Yang et al.] 2024a], , D 3584. This high dimensionality could prohibit efficient training of the representation parameters 9. Alternatively, we can use lightweight model to map xi to fz- This could incur additional memory overhead but generate lower D input the will use lightweight sentence embedding all-MiniLM-L6 -v that has only 22.6 million parameters and incurs only 80MB of additional VRAM. The corresponding feature dimension is D 384.
network go- In the experiment,
Discussion 2 (Importance of embedding network ga). Given the question features fi...., Sn E R.,, one could simplify the representation learning problem (8) by optimizing the method embedding vectors V1,--- VM directly on the space of R This equivalent to setting d = D, and letting ge collapse into an identity mapping. However, the proximity between two question features fy and fy does not convey enough information about the similarity regarding hardness, resource utilization, and suitability with methods. Moreover, learning the method embedding vy on Rº is more difficult than the smaller dimension space IR.. Hence, learning in I is inefficient. This observation necessitates the use of lightweight question map go-
Discussion 3 (Adaptive method insertion). Given a universe of models M. problem ( optimizes one vector 0; for a reasoning model i E M. Alternatively, we could use another network he that could take (text) description of reasoning method and output the respective embedding vector in the representation space T.d. Having the second network ha could unlock several new capabilities: (i) for new reasoning method that is not in M, we could quickly obtain its embedding and predict its performance on the questions, (ii) we could inverse engineer to design a better reasoning method. Unfortunately, training ha requires a meaningful textual description of the reasoning methods. This is currently outside the scope of this paper, and we leave it for subsequent work.
5 Numerical Experiments
In this section, we present numerical experiments showcasing the performance of EPIC on the math answering task. Experiments for the code generation task are relegated to Appendix [D.]
Dataset. We use the MATH dataset [Hendrycks et al. 2021 as a training set, utilizing its training
split of 7,500 math problems with solutions, as defined in Hendrycks et al. [202T]. For the code
generation experiment, we use the LiveCodeBench dataset [Jain et al. 2025 More details are in Appendix D.4, For evaluation, we test the MATH500 test split, which contains 500 samples, as defined in Lightman et al. [0023b]. We also use the test set of the GSM8K (Cobbe et al. 202 dataset to evaluate the transferability of the method embedding vectors learned in Section
Base models. We employ Owen2.5 BBath-7B-Instrus as our generation model, and math- shepherd- mistral-7b-pe [Wang et al. 2024b as our reward model in the PRM framework. These models are fixed throughout our main experiments. For transferability experiments, we augment our universe of

---

10000
Best-of-16 PVL
Other Methods
8000
UB
Average Cost (Tolenss
A 1.0
6000
Best-ar- PVL
boo
MCTS
4000
Offine Bok
Best-of-4
PVL
0=75
Beam Search
0.5.5
2000
Best-of-2
0.25
0.0
0.84
0.86
0.88
0.90
Awerage Success Rate (Accuracy)
Figure 2: Average success rate and token counts the test set with embedding dimension d = 64. Our ensemble planner performances with varying À E {0, 0.25, 0.5, 0.75, 1} are highlighted in red, and individual reasoning models in M plotted in blue. The boundary of the ensemble planners covers the individual models in the universe M. The Upper Bound (UB) under M is the proportion of questions that at least one method in M could successfully solve.
methods to include Qwen2.5-Math-L..B For code generation experiments, 3B-Instruct and -eon:-5--Codrrr---nstru
Swen2.5-Codee.
Performance metrics. We use the accuracy to measure the quality of generation and average token counts evaluate the efficiency of each method. For accuracy, we use the automatic grading provided by previous work Lightman et al.] [2023b] to evaluate the accuracy of generated solution (4). To measure average token counts, we set the hyperparameter 'max new token' 2048 for all methods and compute the average number of tokens generated.
Universe of methods M. We generate M consisting of 81 distinct methods, spanning variety of reasoning strategies, aggregation techniques, and parameter contigurations. A complete description is provided in Appendix B
Dataset generation for contrastive learning. For a deterministic method in M, we run inference on each question žy and record the accuracy and number of generated tokens. For the sampling method, we run the inference on each question five times to obtain mean estimate of az,j and C.j.
Baselines. We compare EPIC against three categories of baselines: (i) individual reasoning methods from the universe M, (ii) strong large-model references including DeepSeek- V3 and OpenA mini, and (iii) alternative reasoning selection methods RA, Offline Ada-BoK (Damani et al., 2025). DRA-À, and CL-À. All ensemble baselines and EPIC are trained and evaluated on the same M for fair comparison. Further details are provided in Appendix C
Reproducibility, All experiments are conducted on single machine with 8x NVIDIA RTX A5000 GPU and Intel(R) Xeon(R) Gold 6148 CPU @ 2.40GHz.
5.1 Comparison between EPIC and Baselines
In the first experiment, we benchmark how EPIC, an ensemble model, outperforms individual reasoning models in the universe M. Tableppreeente the test performance comparison between our ensemble planner and individual reasoning methods, computed based on average accuracy and the number of generated tokens. We apply the regularization parameter = 10-3 based on its better numerical results than other values shown in Appendix D Our method with À 0.25 achieves accuracy of 86.4%, matching the best-of- 16 approach while using significantly fewer tokens: EPIC generates 1859.2 tokens while best-of- 16 generates 10036.2 tokens. On relative scale, this is 5x reduction in the token counts at the same accuracy level. Compared to beam search, our approach

---

at À = 0.25 achieves better accuracy with 29.5% reduction in token usage. With À = 1.00, our method achieves the highest accuracy (89.4%) at significandy lower token count (6,921.7).
To better visualize the performance of EPIC, we plot in Figure 2 scatter plot locating the accuracy- cost trade-off of EPIC instances and representative reasoning models from M. Our EPIC instances (red) all lie on the frontier, thereby boosting the performance of the inference phase.
Since EPIC is ensemble method constructed from the universe of reasoning methods M, the performance of EPIC constrained by the capacity the universe M itself. We could compute the best possible accuracy of the whole universe MV on the test set: M could solve question if there at least one method from M that could generate a correct answer. Computing this value yields an upper bound of approximately 91.2%. Figure highlights that our method provides a flexible trade-off between efficiency and accuracy, approaching the upper bound (dashed vertical line) while maintaining computational efficiency.
Table Average accuracy and number of generated tokens on MATH500 using different reasoning methods and language models (full results in Appendix 4. Methods above the blue line are either upper bounds under M or are not included M. Methods above the green line correspond to individual single-reasoning configurations (without selection modules). Methods above the brown line do not support accuracy- trade-offs. We compare EPIC-X, DRA-À, and CL-A at various trade-off settings (groups separated by gray lines). Best results in each section are highlighted.
Method
Accuracy
Average Token Count +
OpenAl-ol-mini
Jaech et al. 2024
90.0
Deepseck-V3* [Liu et 2024
90.2
Upper Bound under M
91.2
CoT-G
83.2
620.4
Best-of-2
84.8
1242.5
Best-of-4
86.2
2499.4
Best-of-8
86.6
4986.8
Best-of-16
86.8
10036.2
MCTS
85.4
4338.1
Ream-search
85.2
2638.1
RA
84.4
1752.4
Offline Ada-BoK
87.0
4095.2
DRA-0.25
86.2
2453.6
CI.-0.25
86.0
2275.6
EPIC-00,5
86.4
1859.2
DRA-0.5
86.4
5719.3
CL.-0.5
86.6
5320.2
EPIC-0.5
86.8
2482.6
DRA-0.75
86.4
7523.2
CL-0.75
87.0
7524.6
EPIC-0.75
87.6
3192.9
DRA-1.0
87.0
T0542.5
CL-1.0
87.8
10923.4
EPIC-1.0
89.4
6921.7
Mothod not in M. We obtained results from et al. 2024
5.2 Transferability
We now examine the transferability of EPIC across both model scales and datasets. Table sum- marizes results for two complementary settings: (a) transferring from the MATH to the GSM8K dataset, and (b) applying EPIC in a cost-aware multi-mode environment with Qwen2.5-Math-11 and Qwwnn.5-Math-
5.2.1 Evaluating EPIC with Cost-Aware Multi-Model Reasoning
EPIC remains effective even when reasoning methods use heterogeneous base models. Previously, our universe M contained 81 methods built solely on -aahe-.---aat--B--Innnee We now augment this space with additional 81 methods using Qwen2.5-Math Instrutt Because larger models

---

Table 2: EPIC performance comparison across datasets and model sizes. (a) GSM8K results where EPIC is trained on the MATH dataset using d = 64 and = 0.25. (b) Performance and cost comparison for Qwen2.5-Math-1133 1.5B and 7B models. Best results in each column are highlighted.
(b) Performance and cost with Qwen2.5 models
(a) GSM8K results
Method
Accuracy t
Cost
Method
Accuracy f
Tokens
1.5B-CoT-G
76.0
856.5
CoT-G
93.5
297.0
1.5Best-of-4
78.6
3454.5
Best-of-2
94.0
594.6
7B-CoT-G
83.2
4342.8
Best-of-4
94.0
1195.6
7B-Best-of-4
86.2
17495.8
Best-of-8
94.3
2412.3
1.5Best-of-le
79.4
62663.2 .2
Best-of-16
94.2
5019.2
7B-Best 33-Best-of-16
86.8
70253.4 4
EPIC
95.0
2085.5
EPIC (A 0.25)
86.2
8047.8
EPIC (A 1.0)
89.0
35705.4
computationally more expensive, approximate the cost of each method as the product of its parameter count (in billions) and the number of generated tokens. This proxy aligns with real-world inference costs; alternatively, FLOPs or API pricing could be used.
Table 2b shows that EPIC adapts effectively across cost regimes. At À 0.25, EPIC achieves an accuracy of 86.2, matching the 7B-Best-of-4 method, while reducing the cost by over 50% (8047.8 1.0, EPIC attains the highest overall accuracy (89.0), outperforming 7B-Best- of-16 while maintaining roughly half the computational cost (35705,4 vs. 70253.4) ()....4. These results demonstrate EPIC's capacity to balance accuracy and cost by dynamically leveraging reasoning methods from different model sizes.
vs. 17495.8) 8). At
5.2.2 Transfer Another In-Domain Dataset
To further assess generalization, we evaluate EPIC trained on MATH and test on GSM8K [Cobbe et al.] [222], another widely used arithmetic reasoning benchmark. As shown in Table 2a GSM8K is a simpler dataset; hence, absolute gains are smaller. Nonetheless, EPIC achieves the best accuracy (95.0%) while requiring fewer tokens than high-cost baselines such as Best-of-8 or Best-of-16. This indicates that EPIC's learned representations transfer across related reasoning distributions and continue yield efficient anference-time behavior.
5.3 Additional Experiments
We perform ablation studies to better understand the impact of the representation dimension, the regularization parameter т, and the tradeoff parameter X, the utility function, the transferability. and the generalization
the code generation task. In the experiment of representation dimension, we fix À = 0.5 and vary dimension d E {16, 32, 64, 128}. Our results show a general trend: as increases, accuracy improves, while average token count stabilizes or slightly decreases. This result empirically confirms the expectation that increasing the embedding dimension could boost the performance of our method. In the experiment on the impact of X, we observe clear cost-accuracy trade-off as we fix d 64 and varies À (0.00, 0.25, 0.50, 0. 0.75. 1.00]. In the ablation study of the utility function, we switch to an alternative functional form, shown in equation (5), and observe decrease in performance, indicating that our design choice is superior. Due to space constraints, further experimental details are provided in Appendix
Conclusion
We introduced EPIC, the Ensemble PlannIng with Contrastive learning framework, contrastive learning framework that plans optimal reasoning strategies for language models by matching ques- tions to suitable methods. Our analysis established new accuracy bounds for common aggregation techniques, which directly inform a regularization term to guide more sample-efficient learning. periments mathematical reasoning benchmarks demonstrate that EPIC leverages these theoretical insights to achieve strong improvements in both accuracy and inference cost, showcasing the value of principled modeling for reasoning method selection.
10

---

Acknowledgments. Viet Anh Nguyen gratefully acknowledges the support from the CUHK's Improvement on Competitiveness in Hiring New Faculties Funding Scheme, UGC ECS Grant 24210924, and UGC GRF Grant 14208625.
References
Edward Beeching, Lewis Tunstall, and Sasha Rush. Scaling test-time compute with open models, 2024. URL //
scogpost-scaling toag-tessttingg wwweemmmm---mmeeeeeeeee
Zhenni Bi, Kai Han, Chuanjian Liu, Yehui Tang, and Yunhe Wang. Forest-of-thought Scaling test-time compute for enhancing LLM reasoning. In Forty-second International Conference on Machine Learning, 2025.
Tom Brown, Benjamin Mann, Nick Ryder, Melanic Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in Neural Information Processing Systems, volume 33, pages 1877-1901, 2020.
Ngoc Bui, Hieu Trung Nguyen, Shantanu Kumar, Julian Theodore, Weikang Qiu, Viet Anh Nguyen, and Rex Ying. Mixture-of - personas language models for population simulation. In Findings of the Association for Computational Linguisties: ACL 2025, pages 24761-24778, Vienna, Austria, July 2025, Association for Computational Linguistics. ISBN 979-8 109--889976-22--- doi: 10. 1865/133vv 2025. 1122.iinings--cl1177
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Rejichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021.
Mchul Damani, Idan Shenfeld, Andi Peng, Andreca Bobu, and Jacob Andreas. Learning how hard to think: Input-adaptive allocation of LM computation. In The Thirteenth International Conference on Learming Representations, 2025.
Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. Flashattention: Fast and memory- efficient exact attention with 10-awareness. In Advances in Neural Information Processing Systems, volume 35, pages (6344-16359) 2022.
Xinyu Guan, Li Lyna Zhang, Yifei Liu, Ning Shang, Youran Sun, Yi Zhu, Fan Yang, and Mao Yang. MSthr-Math: Small LLMs can master math reasoning with self-evolved deep thinking. arXiv preprint arXiv:2501. NiX::2501..0519, 2025.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the MATH dataset. In Advances in Neural Information Processing Systems, 2021.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai ol system card, arXiv preprint arXiv:2412.16720. 2024.
Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. In The Thirteenth International Conference on Learning Representations, 2025.
Juyong Jiang, Fan Wang, Jiasi Shen, Sungju Kim, and Sunghun Kim. A survey on large language models for code generation. arXiv preprint M.Xi::2406.005555. 2024.
Hyoungwook Jin, Seonghee Lee, Hyungyu Shin, and Juho Kim. Teach ai how to code: Using large language models as teachable agents for programming education. In Proceedings of the 2024 CHI Conference Human Factors in Computing Systems, pages 1-28, 2024.
11

---

Dacheng Li, Shiyi Cao, Chengkun Cao, Xiuyu Li, Shangyin Tan, Kurt Keutzer, Jiarong Xing, Joseph Gonzalez, and Ion Stoica. $ Test time scaling for code generation. arXiv preprint arXiv:2502. 14382, 2025a.
Dacheng Li, Shiyi Cao, Chengkun Cao, Xiuyu Li, Shangyin Tan. Kurt Keutzer, Jiarong Xing, Joseph E Gonzalez, and Ion Stoica. S Test time scaling for code generation. arXiv preprint arXiv:2502. /4382, 2025b.
Kenneth Li, Oam Patel, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference-time Eliciting truthful answers from a language model. In Advances in Neural Information Processing Systems, volume 36, pages 41451-41530, 2023.
intervention:
Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen. Making large language models better reasoners with step-aware verifier. arXiv preprint arXiv:2206. 02336, 2022.
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2023a.
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2023b.
Lin, Jiaming Tang, Haotian Tang, Shang Yang. Wei-Ming Chen, Wei-Chen Wang, Guangxuan Xiao, Xingyu Dang, Chuang Gan, and Song Han. AWQ: Activation-aware weight quantization for on-device LLM compression and acceleration. Proceedings of Machine Learning and Systems, 87-100, 2024.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:24/2.19437, 2024.
Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Meiqi Guo, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, et al. Improve mathematical reasoning in language models by automated process supervision. arXiv preprint arXiv:2406 2rX6522406.06592, 2024.
Rohin Manvi, Anikait Singh, and Stefano Ermon. Adaptive inference-time compute: LLMs can predict if they can do better, mid-generation. arXiv preprint arXiv:2410.02222 2024.
Hicu Trung Nguyen, Bao Nguyen, Binh Nguyen, and Viet Anh Nguyen. Task-driven layerwise additive activation intervention. In Proceedings of the 2025 Conference of the Nations of the Amer- icas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 2: Short Papers), pages 506-513, Albuquerque, New Mexico, April 2025a. Association for Computational Linguistics. ISBN 979-8- 1---8-9977---- doi: 10.18653/v 1/2025 1are535naacl-shhs
Hieu Trung Nguyen, Bao Nguyen, and Viet Anh Nguyen. Structured pruning for diverse best-of-n reasoning optimization. In Findings of the Association for Computational Linguistics: ACL 2025, pages 23911 100111223922 Vienna, Austria, July 2025b. Association for Computational Linguistics. ISBN 979-8-891177 8-6-1981-76665.5 doi: 10.18653/v1/2025 findings- aclnggs-all1125.....
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predictive coding. arXiv preprint arXiv: 2018
Joon Sung Park, Joseph O' Brieen, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1-22, 2023.
Bernardino Panere- Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M Pawan Kumar, Emilien Dupont, Francisco JR Ruiz, Jordan S Ellenberg, Pengming Wang, Omar Fawzi, et al. Mathematical discoveries from program search with large language models. Nature, 465 4756,84-77700 2024.
12

---

Charlie Victor Snell, Jachoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test- time compute opti- mally can be more effective than scaling parameters for reasoning. In The Thirteenth International Conference on Learning Representations, 2025.
Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei. and Paul F Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008-300 10000000001,, 2020.
Trieu H Trinh, Yubuai Wu, Quoe V Le, He He, and Thang Luong. Solving olympiad geometry without human demonstrations. Nature, 625(7995):476 482, 2024.
Ziyu Wan, Xidong Feng, Muning Wen, Stephen Marcus McAleer, Ying Wen, Weinan Zhang, and Jun Wang. Alphazeno-like tree-search can guide large language model decoding and training. In Forty-first International Conference on Machine Learning, 2024.
Jun Wang, Meng Fang. Ziyu Wan, Muning Wen, Jiachen Zhu, Anjie Liu, Ziqin Gong, Yan Song, Lei Chen, Lionel M. Ni, Linyi Yang, Ying Wen, and Weinan Zhang. OpenR: An open source framework for advanced reasoning with large language models. arXiv preprint arXiv:2410.09671 2024a.
Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-Shepherd: Verify and reinforce L.MSs step-by-step without human annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 9426-9439, Bangkok, Thailand, 2024b. Association for Computational Linguistics.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- ery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203. 11171, 2022.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in Large Language Models. In Advances in Neural Information Processing Systems, volume 35, pages 24824-24837, 2022.
Yuhuai Wu, Albert Qiaochu Jiang, Wenda Li, Markus Rabe, Charles Staats, Mateja Jamnik, and Christian Szegedy. Autoformalization with Large Language Models. In Advances in Neural Information Processing Systems, volume 35, pages 32353-32368, 2022.
Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. TravelPlanner: benchmark for real-world planning with language agents. In Forty-first International Conference Machine Learning, 2024a.
Yuxi Xie, Anirudh Goyal, Wenyue Zheng, Min- Yen Kan, Timothy P Lillicrap, Kenji Kawaguchi, and Michael Shieh. Monte Carlo Tree Search Boosts Reasoning via Iterative Preference Learning. In The First Workshop on System-2 Reasoning at Scale, NeurIPS, 2024b.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2.5 technical report. arXiv preprint arXiv:2412.15115. 2024a.
An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jianhong Tu, Jingren Zhou, Junyang Lin, Keming Lu, Mingfeng Xue, Runji Lin, Tianyu Liu, Xingzhang Ren, and Zhenru Zhang. Qwen2.5-Ma technical report: Toward mathematical expert model via self-impprvemment. arXiv preprint 2rXhv22409.12122. 2024b.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with Large Language Models. In Advances in Neural Information Processing Systems, volume 36, pages 11809-11822, 2023.
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36, 2024.
13

---

Fei Yu, Anningzhe Gao, and Benyou Wang. Ovm, superie--uperrvseed value models for planning in mathematical reasoning. arXiv preprint arXtv:2311.09724. 2023.
Dan Zhang, Sining Zhoubian, Ziniu Hu, Yisong Yue, Yuxiao Dong, and Jie Tang. Rest-MCTS": LLM self-training via process reward guided tree search. In Advances in Neural Information Processing Systems, volume 37, pages 64735-64772, 2024.
Xiang Zhang. Tianze Ling, Zhi Jin, Sheng Xu, Zhiqiang Gao, Boyan Sun, Zijie Qiu, Jiaqi Wei, Nan- qing Dong, Guangshuai Wang, al. n-PrimeNovo: accurate and efficient non-autoregressive deep learning model for de novo peptide sequencing. Nature Communications, 16(1):267, 2025.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The claims in the abstract and introduction clearly stated and fully supported by the theoretical analysis and experimental results presented the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We provide the limitations in the impact statement at the end of the main paper.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification: We provide a full set of assumptions and complete proof.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide all necessary information
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification: Yes. include an anonymous link the implementation code for this paper.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e-g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide information about our experiments in the experimental section.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Due limited computational resources, we were unable run multiple experimental trials required to compute error bars or statistical significance.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: We provide information about machine in experimental section.
9. Code of ethics
15

---

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics http://pgggg.p.oo......./ooo//aa
Answer: [Yes]
Justification: We make sure preserve anonymity.
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We have an impact statement at the end of the main paper.
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: the paper poses such risks.
12. Licenses for existing assets
Question:
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: we cite all existing assets.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: the paper does not release new assets.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: the paper does not involve crowdsourcing nor research with human subjects. 15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: the paper does not involve crowdsourcing nor research with human subjects.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs as any important, original, non-standard components.
16

---

A Proofs of Section 3
After sampling independent candidate solutions from the distribution, aggregate them obtain the output answer. We focus on characterizing the probability that the output answer is y1, meaning that the output answer is correct solution to question q.
Let ((() (C1 ..... .(k) denote the counts of each solutions after N samples, that
Σ Ck N. The count random vector
(.)) follows the multinomial distribution:
Pr(C (N) = = c1.
N
K
PE
(9)
CK
for any vector c (C.,...,CK' of natural numbers summing up N.
A.1 Majority Vote Analysis
The majority vote selects the correct answer, with probability one if ei > c for all l: & 1, or with some probability w 1 if 0₁ 2 o& for all & * with strict equality e1 cg for some k # 1. The value w: represents the probability of choosing y1 in cases of ties.
Thus, the exact probability that the majority vote selects y/1 is bounded by
Σ tL G126N Vkz1
N!
PM 6V' W=1
II
I Pr(mmaority vote picks y1) I
Σ
N!
Il K'1
PK
C1
CK!
c1
ck!
61 G1 VRz1
However, these bounds are computationally intractable for large N due to the combinatorial explosion of possible vote count configurations. Moreover, they do not provide clear insight into how the selection probability changes as varies, limiting their practical utility analytical understanding approximation.
We now state the theorem that bounds the probability that the count of one solution less than or equal to another:
Proposition A.1 (Count boper-,ound), Assume Pa > ps for any pair of distinct indices a, b e
{1...., K}. + b. Then, we have
Pr(C(() S () exp (-N (Vpa vpb)²).
(10)
Proof of Proposition A.7 For each draw u = 1,... N, let C. denote the selected category with Pr(Cu- k)) - Pk. Define
N
((M)
Σ
[(C,=)),
k= K.
Then the difference of counts between bins and can be written
((N) C(N)
X
X. = L[c,=a) L(C.=b).
The variables (X,) are i.i.d., therefore for any > 0, Markov's inequality implies
Pr(C(M) < c ((N) Pr -10 -ON 2 1) - ) = (Ele (x N
Conditioning C yields
Eie] Pac + pbc + +1 Pa pb).
Substituting this expression into the previous bound gives
Pr(C < (.N)) s exp( N log(pat + + pre + Pa - pbb -Pt))
17

---

To optimize the bound, define h(t) P +pbe' + +1-11ppp Pa Pb- Minimizing h(t) over t >> 0 leads the first-order optimality condition
(t) = P + + pbc' =
e2t
Pa
log
Pu
pb
pb
Since Pa > Pb. we have ta > O, which is valid. Plugging t* back into htt), we obtain
h((*) Pa pb 2//PuPp (vPo vPb)²
Therefore, we obtain
Pr(C(N) S 26 S (1-(V/a - vPb))) Finally, using log(1 - x) S - for x E (0, 1). we obtain the exponential form
Pr(((() < ((N)) S Hop(-N(cp N(vTa exp(-N
//h)²))
The proof is complete.
We are now ready prove Theorem 3.1
Proof of Theorem 3.7 Recall that we assume is the correct solution. P1 Pk for all k 2....,K K. We obtain by applying Proposition A.I
Consider the case where
K n
{ (cP > km
Pr(majority vote picks vn) N Pr
k=
{ c(
=1 -r
K
Pr (((N) S xx(
21
k=2
K
N
2 1 -
k=2
Now consider the case where there exists k E (2..... K} such that pk Pı:
K
((N) 2 C(MI) < Pr (c/N) 2 c((e))
Pr(majority vote picks vI) < Pr
n
<e- -e-N(v -NVPp---vvi v
where the last inequality follows from Proposition A.I
A.2 Aggregation using Summation of Scores
Let (((N) be the count of the solution yk, and let each of the A(N) PRM scores be independently GIN)
distributed with the same distribution Sk., NN of) for all u. Define ū(()) now state the theorem that bounds the probability that the total score for one solution is less than or equal that of another.
> y=
Sen. We
Theorem A.2 (Sum upper-bound). Suppose that Palla Potts for some distinct a, b, (1....,K} K}. Then
Pr (Ú!N) S9>>000 <0
(N < inf inf exp (Npa (e-tent t0002 -1 1) + Npb (clun -00
(11)
150
Moreover, under this condition, the right-hand side decays exponentially in N.
18

---

Proof. For each draw = l..... Let Sku (uR, of) be independent across both & and u, and independent of (Ca ( uu=1 Then
. N, let Cu denote the category selected. with Pr(Cu = k) = Pk-
U Σ Sku (CC==))
k I....,,,
so that
( (( Σ Xus where X,, Sau (Cul(C_=o) (C_=a) Sou (C. =b)-
The random variables (Xu)== are i.i.d. Hence, for any >> 0, Markov's inequality yields
Pr (i S m - B
-0 -
= (Ble (B(e-cXij)
Conditioning on C1 gives
Ee [x = Pa Ele (5.11 pb 55 + Pa pb).
Using the moment generating function of normal random variable gives
Ele [ = t
Finally, we obtain
e 100- Pae -0atttea pse phe don f6 + Pa ph)
To simplify the expression, we use log(1 + x)) S x for x > -1 We obtain
Pr ( < 0 z exp (N tea 1) -
Optimizing over t > 0, we obtain
Pr ((()) < ) < inf exp Npa Npa(e-Wattfed + Npblctnot3300 11) t>0
To verify the exponential decay, we define
F(t) Pa (e tu + g - -1) +
1).
We have F(0) = 0 and
F'(0) -aala Poltb < <0,
for small 0, F()) < 0. Therefore, the exponent negative and the bound decays exponentially N.
We now prove Theorem 3.2
Consider the case where Р1d1 > PaAk for all ts = 2.... R...,K K. Following an analogous argument in the proof of Theorem 3.1 we have
Proof of Theorem 3.2
Pr(PRM Vote picks y1) 2
inf exp (Np) ( (e-nmrt4400 4x50
16099 -1) + Npx -1)
Consider the where there exists i such that PkA4 P1/11 We have
Pr(PRM__ord Vote picks vı) < Pr(ūt(M) < ((())
< inf exp (NDx (N (c-nettffo- -1) NPi (cun+(fod-1)) t00
This completes the proof.
19

---

A.3 Aggregation using Maximum of Scores
To the readability, recall the setup for this section. Let ((N) be the count of the solution yk, and let each of the ((()) PRM scores be independently distributed with the same distribution Skn ~ [uu, o for all u. We define M max 1gu5ōh Sku.
Proposition A.3 (Max PRM upper bound). Assume da Tb for some distinct pair a, (L....,K)) # b. Then, for any N 2 L,
Pr(M S M, (
inf tE
(1 (1-Put Pall
1 Ma |) +1 (1 poll
Φ 1 )
Da
Ob
Moreover, the right-hand side converges to zero as 00.
Proof of Proposition A.3
We start by characterizing the distribution of M,
Conditional on
Ci m, we have
Pr(M st| C/ = m) = [t)"")
where (1) = (((1-u)) - mw)//w) is the CDF of N(us, of) Substituting this into the law of total probability yields
N Pr(M =t)= Σ
Pr(M m) Pr(M) (M) St M = m)
m==
N
pb (1 (1-pb) (t)
m
m=
N
Σ
(po (p)) (1-6))
Po)
m
= (1 - ps + pb (t))
= (1 p- (()))
Taking the complement gives
Pr(M (N)
> 1) = 1 - (1 pb -
An identical argument gives
2a a(t-1) Ma
Pr(M(N)
Pol -1)))))
0a
Next, we have that
Pr(M < ( Pr(j < f or M (N) for all t)
S P < l or Ml (N) for any L)
S (P(M)()) () + Pr(M > f),
where the last inequality follows from the union bound. Hence we have the following for every t E R:
Mr(M < M (1 pal1 +
(يسي o. ) +1 (1 Pb 11 + مسسا σb
)
Since this inequality holds for I, we may optimize the bound by taking the infimum over E
( < M < inf teR (1
Pall + (+=Ma) )D + (1 pb[1 + ( 1)
We now analyze the asymptotic decay of the bound N + co. Let IN 1bb + σb for some fixed E (0, 1). We define
2(1 2
E) log N
iN
14b
/2 + +c) log N, Za,N
1N No 9b 2 + log N P/ - 14b
26.N
σb
de
20

---

Then, the bound evaluated at t becomes:
Pr(M S ) inf {(1 Pa[l & (za,N)))
+1 (1 (1-ph(] pb[1
& (26,N ())
EER
We use the standard Mills inequalities, valid for all x > 0:
x² + p(x) = ((x)) = (xx)
p(x) (2T) -1/2)
1/2 e-212- =2//
We first study 1 1- (1 po[l 1 (2) For any 0 S 2 it holds that 1 - (1 - z) s No.
We use it to have
1- (1-Po(l- (2b,N))) N Np (z6,N))
Using the Mills upper bound, we obtain
1 (26,N) v p(ZL,N)
N (1+e)
2b,N
2 Eb,N
Hence, we have
Npb ((26,N)] S
pb
-
as N - 0C.
2tT 26,N
Next we study (1 Pal - 2 (2), We start with the inequality
0 < (1 P. [1 ((2,,))))* I <exp(-Npall ((E,,N)))
which holds because - x v - for any E [0, 1].
If N for sufficiently large N. then 1 ((Z,,N) 1/2, that
- (1-Pa( ((E,,)))) < - pa/2) +0 as N - oc.
Otherwise, if Za,N > 0, the lower Mills bound gives
C - x/12
- (2a.N) 2
Na.N
for some constant > 0. Hence,
Np. 11 - (22,,N))
O² N
e 22.N12
Za,N
Using the definition of N, we obtain
2 o/ 2(1 +) log N + o(log N).
N o2
Substituting into the inequality yields us
Np. ((a,NN)] 2 C" NI- co N)
for some constant C" > 0. Since Th. we can choose
0 sufficiently small so that
1 - (1
0,
implying that Np_[1 ((2a,N)] + co, and hence
(1 Pal ((a.N)) O.
Combining with the analysis for b, we finally obtain
Pr(M(N) < (( +0 (N + 20).
This completes the proof.
21

---

We are now ready to prove Theorem 3.3
Proof of Theorem 3.3 Consider the case where 01 > ok for all k 2...., K. Following an we use Proposition A.3 have
analogous argument in the proof of Theorem 3.1
$( t M1 1))
Pr(PRM Max picks y1) 2
inf EE
(1 pı(1
01
(1 - Pk
D
where is the cumulative distribution function of the standard normal distribution. By Proposi-
tion A.3 each term in the summation tends to zero as N - oc. Therefore the probability that PRM Max correctly selects y/ tends
In the alternative case, suppose there exists some such that on Following an analogous argument in the proof of Theorem 3.1 we use Proposition A.3 to have
(1 pMl1 $1 00--1i0 ))
Pr(PRM_Ma) picks v1) S inf
IER
D "(ー ))
+1
Pi
Furthermore, by Proposition A.3 the bound tends to when N
This completes the proof.
B Universe of Methods
In this section, we provide systematic description of universe of methods M mentioned in Section [5] Each method is represented by a tuple (L.M, ReStrat, Conf, N), where each component defined as follows.
Language Model (LM), For our main experiments, we consider single base model: In Appendix 5.2. we extend our pool of methods by addition- ally using Qwen2. 5-Math-11..... Instruct.
Reasoning Strategies (ReStrat). We consider four primary groups of reasoning methods:
The deterministic strategy denoted as COT-G that always selects the highest- probability next token.
Greedy Search:
The model samples N complete responses for each question according to the output token distribution, controlled by a temperature hyperparameter. The temperature adjusts the randomness of sampling: lower temperatures make the output more deterministic, while higher temperatures increase diversity. If N 1, this produces single, randomly sampled response. When N > 1, aggregation strategies (described below) are applied to select a final answer from the N candidates.
Best-of-N:
The model first generates N distinct first steps, each evaluated by a PRM. The top N/m steps with the highest PRM scores are kept, where N/m E Z. For each retained first step, the model generates m second steps, forming N partial solutions. This process repeats until N complete solutions are produced or reach the maximum number of beam expansions (50 in our case).
Beam Search:
Monte Carlo Tree Search: This method, denoted as MCTS, formulates response generation as tree search problem, iteratively exploring possible answers by balancing exploration and exploitation. At each step, MCTS selects the most promising node based on selection policy (e.g., Upper Confidence Bound), expands new response candidates using the base model, evaluates them through rollouts or verifier models, and back-propagates the scores refine future selections. The search continues until obtain N complete solutions.
22

---

Configuration (Conf). configuration presents all the hyperparameters of a reasoning strategy. While almost all of them are fixed as default values suggested by the language model report and repository OpenR [Wang et al. 2024a]. we vary the temperature of decoding in the set (0.4, 1.0} to explore different levels diversity.
Aggregation methods (Agg). For reasoning strategies that produce multiple candidate responses, we employ aggregation methods select the final answer. These methods include:
Majority-Vee -Voe (MV): Select the most frequently occurring answer from multiple samples.
PRM_ HOM__ote_Minn (PVM): For each generation, the PRM to each step, select the mini- mum score within the generation, and ultimately choose the generation with the highest sum of minimum score across all samples.
PRM. Vote_ooe__ass (PVL): For each generation, use the PRM score each step, select the score associated with generating the last step, and ultimately choose the generation with the highest sum of scores across all samples
PRM Min (PMM); For each generation, use the PRM to score each step, select the minimum score within the generation, and ultimately choose the generation with the highest minimum score across samples.
PRM Max Last (PML): For each generation, use the PRM to score each step, select the score associated with generating the last step, and ultimately choose the generation with the highest score across all samples.
PRM_Max
Notably, except for Majority_ Vote, other methods are required call the reward model.
Candidate solution size (N). We vary the number of candidate solutions in N E {1, 41,2,4,8,16)
Overall, the universe MM consists of 81 reasoning models that span different reasoning strategies and configurations. The detailed composition of the universe M is summarized Table 3 Specifically, construct M by combining reasoning strategies, decoding settings, and aggregation methods. The full enumeration yields 81 unique configurations: 60 Best-of-N variants, 10 Beam Search variants, 10 MCTS variants, and one deterministic CoT-G configuration.
Table 3: Construction of the universe M (81 methods) by strategy, search budget N, decoding temperature temp, and aggregation Agg E (MV, PVM, PVL, PMM, PML].
Strategy
N values
temp values
#Agg choices
Count
Best-of-N
(2, 4, 8, 16] {2, 4]
(0.4, 0.7, 1.0) 0.5
4x3 5 60
Beam Search
5
2 5 = 10
MCTS
{2,
0.5
2 = 10 1
CoT-G (greedy)
(greedy)
N/A
Total
81
C Baseline Descriptions and Complete Experimental Results
We present an additional ablation study focusing another critical component of our framework: the design of the reasoning selection module. In particular, we replace our contrastive-leamit and wrodeo reasoning selection model with four baselines. We first consider two baselines that do not support an accuracy-cost trade-off:
(025 which originally operates on batches of questions and manages resources at the batch level. For fair comparison, we use their 'Offline allocation' strategy, modified operate the individual question level without requiring access to the entire batch at test time. Here, each allocation corresponds to choosing a specific reasoning configuration.
Offtine Ada-BoK adapts the approach from Damani
Random Allocation (RA): At inference time, each question randomly selects reasoning method from the available reasoning configurations.
23

---

The next two baselines, along with our method, support explicit accuracy-cos. trade-off, controlled by the hyperparameter which we vary over {0.0, 0.25, 0.5, 0.75, 1.0).
Multi-class classifier (CL-X), In this tlassifierbbased version, the best reasoning method for each question still determined using the utility-driven labeling (based on the parameter A), exactly as in our original approach. However, instead of using contrastive loss with a two-tower embedding structure, we adopt a two-layer classifier placed on top of the pretrained sentence transformer network and train with standard coss-eytropy loss, commonly used in classification scenarios. We denote this caasificationnbbsee ablation as CL-À.
Distributional Random Allocation (DRA-X): Parameter À matches the trade-off parameter used in our approach. During training, each question is labeled with the reasoning method having the highest X-adjusted score prediction. At inference, reasoning methods are randomly drawn from the observed training distribution.
Table Average accuracy and the number of generated tokens on MATH500 for different methods and models. Methods above the blue line are either Upper Bound under M not in M. Methods above the green line are individual single-reasoning configurations (no selection module involved). Methods above the brown line do not support accuracy- cost trade-off. Below, compare EPIC-X, DRA-A, and CL-À at different trade-off settings (groups separated by gray lines). Best results each section in bold.
Base Model
Method
Accuracy f
Average Token Count
Qwen2.5 72B Basc
CoT* Yang al. Yang
2024b
80.0
QwQ 32B
CoT
2024b
83.2
OpenAl-ol-mini
CoT
Jaech el al.
2024
90.0
Deepseek-V3
CoT* Liu ct al.
2024
90.2
-wen2.5-Math-7B--o
Upper Bound under M
91.2
Qwenn.55-Math-7BBInnnttt
CoT-G
83.2
620.4
Wwen2.55-auth7BB-nnniie
Best-of-2
84.8
1242.5
Qwen2.5-Math-77 7B-Instruct
Best-of-4
86.2
2499.4
Qwen2.5-Math--7 Ins-rust
Best-of-8
86.6
4986.8
Qwen2.5-Math-778 7B-Instruct
Best-of-16
10036.2
Qwen2.5-Math-77 -r--aatth-----trructt
MCTS
4338.1
Owen2.55Maut-77-Inntrr
Beam-search
85.2
2638.1
Owen2.55-Math-77---nne
RA
84.4
1752.4
Swen2.55-aath77--nnttn
Offline Ada-BoK
87.0
4095.2
DRA-660
85.6
1248.4
Owen2.55-Math-77--nnsee
CL-0.0
85.2
606.7
EPIC-0.0
85.8
892.9
DRA-0.25
86.2
2453.6
Owen2.5-Maath-7T-Innte
CL-0.25
86.0
2275.6
Swen2.55-Math-78-nnntrr
EPIC-0.25
86.4
1859.2
Owen225-Math--7B--nnae
DRA-0.5
86.4
5719.3
Owen2.55-Math-7TB-nnee
CL-0.5
86.6
5320.2
Dwenn.5--Math-7B-Iimn
EPIC-0.5
86.8
2482.6
9wen2.55Mutt-778---
DRA-0.75
86.4
7523.2
Owen2.55-Muth-7B-IIme
CL-0.75
87.0
7524.6
Dwen2.5-Math-777
EPIC-0075
87.6
3192.9
S0en2.5--uthh77B--in
DRA-T.0
87.0
T0542.5
Swwn2.55MMthh-TB-nnnae
CL-1.0
87.8
10923.4
Qwen2,5-Math-77
EPIC-1.0
89.4
6921.7
"Methed M.
We observe from Table that our EPIC-À strongly outperforms the simpler classification version (CL-A) and DRA-À at almost every level of the parameter X. Still, the CL-À method remains superior to individual single reasoning configurations, confirming that our labeling strategy based on proposed utility function is indeed effective. Moreover, our original contrastive-learning approach with two-tower embedding structure significantly enhances scalability: introducing new reasoning methods simply involves adding new embedding vector without retraining the entire selection

---

module. This two-tower model and contrastive loss combination have proven highly advantageous over classification-basee methods, both in terms of scalability and overall predictive performance.
D Ablation Studies and Additional Experiments
This section presents additional ablation studies to investigate the impact of various design choices in our framework. First, examine the effect of representation embeddings and the cost-accuracy trade-off parameter A. Second, we assess the robustness of our approach by substituting the utility function (5) with alternative formulation. Third, evaluate the transferability of our framework across different language models and other in-domain datasets. Finally, we present additional experiments for the code generation task.
D.1 The Impact of the Representation Dimension
In this experiment, we study the impact of the dimension of the representation space d the performance of EPIC. One could expect that larger dimensions will give a higher representation power and thus EPIC could perform better. For simplicity, we conduct the experiments only for = 0.25. The average test accuracy and token counts are reported in Table 5 We could identify global trend that, d increases, the accuracy increases, while the average token count tends to go flat or decrease. This result empirically confirms the expectation that increasing the embedding dimension could increase the ensemble's 's performance.
Table 5: Impact of on test accuracy and average token counts with À 0.25.
16
32
64
128
Accuracy t Average token counts
85.6 1828.3
85.4
86.4
86.2
2271.4
1859.2
2004.5
D.2 The Impact of the Trade-off Parameter À
Table taleooolllutttee the impact of À on accuracy and the number of generated tokens. As À increases from 0.00 to 1.00, observe consistent rise in accuracy from 85.8% 89.4% and in average token counts from 892.9 to 6921.7. This trend indicates a clear cost-accuracy trade-off and can be identify an upward trend of the red circles.
visualized in Figure where
Table 6: Impact of À test accuracy and average token counts with embedding dimension d = 64.
X
0.00
0.25
0.50
0.75
1.00
Accuracy t
85.8
86.4
86.8
87.6
89.4
Average token counts |
892.9
1859.2
2482.6
3192.9
6921.7
D.3 Ablation Utility Function
We previously presented two ablation analyses on the embedding dimensionality in Section D. and D.2 To further justify our choice of utility function, conduct an additional ablation study using an alternative utility formulation:
the trade-off parameter À in Section D.2
u(as.j- Ci,) at, (1 - ==,,))~>,
where at,j denotes accuracy and cı denotes cost for the j-th method on the i-th instance. We vary the trade-off parameter À over the (0.0, 0.25, 0.5, 0.75, 1.0).
Notably, for = 0.0 and À = 1.0, this alternative function reduces to our original utility formulation. Therefore, we focus comparison the intermediate values À E (0.25, 0.5, 0.75). For better presentation, we denote our framework with this alternative utility function as PMU (Power Mean Utility).
As shown Table 7 EPIC consistently outperforms the DRA baseline across evaluated PMU configurations, demonstrating its robustness under different accuracy-cost trade-offs.
25

---

Table Comparison of PMU and EPIC methods different À settings. Best results each section are in bold.
Method
Accuracy
Average Token Count +
PMU-0.25
86.0
2334.1
EPIC-0.25
86.4
1859.2
PMU-0.5
86.4
3035.4
EPIC-0.5
86.8
2482.6
PMU-0.75
87.2
4724.5
EPIC-0.75
87.6
3192.9
D.4 Results Code Benchmark
To assess the generality of our proposed method beyond the math domain, we evaluate it Live- CodeBench (Jain et al. 2025]. The universe of methods includes both the Chain of Thought Greedy (CoT-G) and Best-of-N sampling strategies. the Best-of-N approach, the base model generates N candidate responses per question, from which the best is selected, with N E (2, 4, 8, 16} and the decoding temperature chosen from 0.2, 0.6. CoT-G produces step-by-step solutions using greedy decoding (temperature set to 0). For evaluation, we adopt the pass@k k metric, as described in [i] jet al. 2025b), and test two LLM base models of differing capacities: Qwen2.5-Coder- -----ooddr-----nstrreee and O Coder-7B-Instruc" (Yang et al. 2024b). In the code benchmark, we do not consider aggregation methods, so the regularization parameter is set to this experiment.
Table reports the accuracy (pass@ @ k) and average token counts for each method. We observe that our method, EPIC, achieves competitive or superior accuracy to all baselines at both 0.25 and À = 1.0. For the 7B model, EPIC (A 1.0) achieves the highest overall accuracy (61 (8%). outperforming Best-of- 16, while also consuming fewer tokens. Similarly, on the 3B model, EPIC (A 1.0) achieves the best accuracy (48.01% %), exceeding the Best-of- 16 baseline.
Trade-off Control. EPIC with À 0.25 achieves balanced performance, providing better accuracy than CoT-G and Best-of-2, but at a modest computational cost, highlighting the framework's flexibility in managing the trade-off of accuracy and cost.
Efficiency at Lower Cost. Notably, CoT-G remains the most computationally efficient method, but at the expense of lower accuracy. EPIC offers favorable balance, substantially improving accuracy while keeping generation costs well below those of aggressive sampling strategies like Best-of-16.
Table 8: Performance comparison of reasoning methods on Qwen2.5-Coder-33 1nnp:---Cooerr-B-Isstteee and Qwen2.5- Coder-7B-Instrun The best value in each column is in bold.
Owen2.5-Codeee 3-r----o-d---n-ineeeee BB-nttruuct
Qwen2.5-Coder- BB-nnstruet
Method
Accuracy 1
Avg. Token Count
Accuracy
Avg. Token Count
CoT' Greedy
24.85
580.80
35.81
505.15
Best-of-2 (with best temperature)
27.40
1144.73
41.68
1000.27
Best-of-4 (with best temperature)
32.88
2299.65
48.53
2011.99
Best-of-8 (with best temperature)
40.90
4672.83
53.82
4031.65
Best-of-16 (with best temperature)
46.38
9323.22
58.71
8034.65
EPIC 0.25
30.12
1025.43
43.44
813.42
EPIC 1.0
48.01
8349.12
61.88
7013.54
E Qualitative Results
E.1 A Specific Reasoning Method Favors in Certain Questions
This appendix provides curated example in Table for qualitative analysis. The goal is to present case where some individual methods in the universe M fail, but EPIC still produces accurate answers by selecting the most suitable one.
26

---

E.2
Visualization and Analysis of Learned Embedding Space
VSAVsullbttinn Dasston Reasoning Emoeddings
PCA Vauulbation of Dusation Taaumiigg Dribadd ags
4414
23:18
18:18
-04
0.11
2.50
6.1%
640
-.11
0.33
(a) À 0.25
(b) A== = 0.5
VcAViaaliatioo Gonation and Reaaz Tmazadingp
dit
80:1
805.0
(c) 1.0
Figure 3: PCA visualization of question (grey) and reasoning method (colored crosses) embeddings for three different settings of the utility trade-off A. Each method is labeled by index: num, where num is the number of generated answers.
To further understand how EPIC organizes and exploits the structure of mathematical reasoning methods and questions, we visualize the learned embedding space using Principal Component Analysis (PCA). Figure presents three PCA plots of the question and method embeddings for different utility trade-off values: = 0.25, À 0.5, and X = 1.0. In these plots, the grey dots represent the embedded math questions, while each colored cross denotes reasoning method. Each method is annotated with the format index: num, where index is the method identifier and num is the number of generated answers for that method.
Plot illustrate how the utility trade-off parameter shapes the structure of the learned embedding space:
When 0.25 (cost prioritized), method embeddings are widely scattered and tend avoid regions dense with question embeddings. Only low-cost methods are positioned near clusters of questions.
When À 0.5 (equal weight to accuracy and cost), higher-cost but occasionally effective methods move closer to question clusters. This reflects a balanced trade-off, where moderately accurate and moderately costly methods are preferred.
When (accuracy prioritized), most methods form distinct clusters around their optimal question types. This demonstrates EPIC's ability match each question with the most suitable method.
27

---

These PCA visualizations confirm that EPIC organizes method and question representations according the chosen trade-off. This structured alignment helps explain EPIC's strong performance discussed earlier in the paper.
28

---

Reference Q&A
Q: Evaluate
sarnsnn 0.4 + aresin 0.5) sin(arcsin 0.5 - arcsin 0.4)
A: From the angle addition and subtraction formulas,
sin(x y) sin cos + cos x sin y, sin(x y) sin x cos cos sin y/,
we get sin(x + y) sin(x y) (sin x cos y + cos a sin ()in cos y cos x sin y) = sin x
sin y. For x aresin 0.5 and y aresin 0.4,
9 100
sarcsin 0.5 + aresin 0.4) sin 0.5 aresin 0.4) = 0.5² 0..² = 0.09
Best-of-16 MV
To evaluate sin aarcsin 0.4 + arcsin 0.5) sin(arcsin aresin 0.4), let = arcsin 0.4 and
aresin 0.5, sin 0 0.4 and sin ø 0.5. We need find sin(0 + Φ) sin( 0).
Using the sine addition and subtraction formulas:
sin(0 + x) = sin cos + cos 0 sin Φ, sin(ø 0) sin @ cos 0 - cos sin o
get:
si + Φ) si 0) = (sin 0 cos x)² (cos 0 sin x) Now, using cos / = - sin 0 0.84 and cos /b = 1 sin 2 0.75, the expression becomes:
(0.4- v0.75) 0.84 0.5) 0.16 0.75 0.25 0.84 0.12 0.21 -0.09.
Thus, the value of the expression is:
0..9
EPIC selects Best-of-16 PVI
To evaluate sin (aresin 0.4 + aresin 0.5) sin (arcasssi 0.5 arcsin 0.4), let aresin 0.4 and = arcsin 0.5, so sin a 0.4 and sin 0.5. The expression becomes sin(a + () sinn - a).
Using the product-to-sum identity for sines:
sin x sin y = |cos(x x - v) - cos(x + y)],
we apply it to x = a 3 and y = 3 - a. yielding:
sin(a + sin (B ai) ícos 2a cos 2,3..
Using the double-angle identity cos 20 = sin 0, we get: cos 2a = 2 sin
and cos 23 sin' 2 3. Substituting these, the expression becomes:
sin(a + 8) sin - a) / [1 2sin a) - (1 - 2sin 3)
Simplifying:
2(sin 2 B sin 2 a) = sin" 3 sin
Substituting sin a = 0.4 and sin 3 = 0.5: sin B sin² a = -0.4)) 0.25-0...0..... 0.16 = 0.09.
Therefore, the value of the expression is: 0.09
Table 9; A curated sample from the MATH benchmark. This is example where individual method fails, but EPIC could deliver an accurate answer by selecting the most suitable one.
29


Reviews:
[{"review_id": "kSa03lJUW8", "reviewer": "Reviewer_v8TG", "summary": "This paper presents EPIC, a framework that helps LLMs choose the best reasoning strategy for each math problem. Instead of applying one fixed method, EPIC learns to match questions with suitable strategies using contrastive learning and a regularization term based on probabilistic aggregation bounds. It offers a theoretical analysis, a learning setup using these bounds, and experiments showing either reduced cost or improved accuracy.", "questions": "- Can you provide empirical evidence that the theoretical bounds hold in practice for LLM reasoning tasks?", "limitations": "Discussed", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n\n- The integration of probabilistic bounds with contrastive learning for reasoning method selection is well-executed and practical.\n- The derivation of accuracy bounds for majority voting, PRM voting, and PRM max aggregation provides useful theoretical grounding.\n\nWeaknesses:\n\n- there is no comparison with various strategy selection techniques such as MRP, RTR, or TATA frameworks\n- Insufficient discussion of how EPIC differs from existing routing approaches\n- the aggregation bound is a known results from probability theory\n\n\nreferences: \n\n-  “Meta Reasoning for Large Language Models” (Meta-Reasoning Prompting, MRP). arXiv:2406.11698 .\n-  Zhihong Pan, Ruohan Zhang, et al. “Route to Reason: Adaptive Routing for LLM and Reasoning Strategy Selection” (RTR). arXiv:2505.19435 .\n-  Xin Xu, Yan Xu, et al. “Teaching LLMs According to Their Aptitude: Adaptive Reasoning for Mathematical Problem Solving” (TATA). arXiv:2502.12022 .\n- Michael Mitzenmacher and Eli Upfal. “Probability and Computing: Randomization and Probabilistic Techniques in Algorithms and Data Analysis.” Cambridge University Press, 2017. \nOlivier Germain, Amaury Habrard, and Édouard Marchand. “PAC-Bayesian C-bounds for Majority Vote.” In Proceedings of ICML, 2009.", "was_revised": true, "final_justification": "After reviewing the rebuttal discussion I found most of the issues were adequately addressed.", "rebuttal": "We appreciate the reviewer’s comments on the strengths of our work. However, we would like to raise two concerns:\n#### 1. Citation to a Work Posted After Submission\nThe reviewer requests comparison with *Route to Reason (RTR)* \\[arXiv:2505.19435], which was posted on **26 May 2025**, **after** the NeurIPS submission deadline of **15 May 2025**. As such, this work was not accessible to us at the time of submission and should not be used as a basis for criticism, following NeurIPS reviewer guidelines.\n#### 2. Possible Fabricated Citation\nThe reviewer cites the following paper:\n> Olivier Germain, Amaury Habrard, and Édouard Marchand. “PAC-Bayesian C-bounds for Majority Vote.” In Proceedings of ICML, 2009.\n\nWe have thoroughly searched online repositories and Google Scholar, but could not locate any publication matching this citation. We postulate that this may be a fabricated or misattributed reference, possibly conflating the following two real works:\n* Pascal Germain et al., *PAC-Bayesian Learning of Linear Classifiers*, ICML 2009.\n* Pascal Germain et al., *Risk Bounds for the Majority Vote: From a PAC-Bayesian Analysis to a Learning Algorithm*, JMLR 2016.\n\nWe respectfully request that these issues be considered in the review evaluation.\n\n\nRegardless, we would like to address the reviewer’s concerns below:\n\n> Weakness 1: There is no comparison with various strategy selection techniques such as MRP, RTR, or TATA frameworks. Insufficient discussion of how EPIC differs from existing routing approaches.\n\n**Meta-Reasoning Prompting (MRP)** [1] relies on GPT-4 to select from 7 prompting strategies. At test time, MRP performs an additional call to GPT-4 using a *meta-reasoning-prompt* that describes the available prompting techniques and determines which one to apply. A key limitation of MRP is its reliance on an advanced and proprietary model (e.g., GPT-4), whereas our **EPIC** operates without such constraints. \n\n**Teaching LLMs According to Their Aptitude (TATA)** [2] adopts a more complicated three-stage framework: (1) *data preparation*: For each question in the dataset, TATA generates both a Chain-of-Thought (CoT) and a ToRA \\[3] response. (2) *data selection*: TATA selects either the CoT or ToRA response for each question by evaluating its effectiveness. To do so, TATA inserts the question-response pair as a one-shot example and measures LLM performance on a validation set of 100 questions. (3) *Instruction Finetuning*: TATA then finetunes LLM on this selected dataset, aiming to internalize the decision of when to apply CoT or ToRA reasoning. Despite this complexity, TATA only supports two reasoning methods and appears to be more challenging to scale up the number of reasoning methods due to the complicated framework. Meanwhile, EPIC provides much wider coverage of up to 81 methods and is easily extensible without modifying any cost function. \n\nWe empirically compare EPIC with MRP and TATA on MATH500 as follows: \n\n**EPIC vs MRP**\n\n| Methods & LLM                    | Accuracy on MATH500 |\n| -------------------------------- | ------------------- |\n| gpt-4o-mini   - CoT                       | 0.774               |\n| gpt-4o-mini-stepback-prompting   | 0.742               |\n| gpt-4o-mini-refine-prompting        | 0.752               |\n| gpt-4o-mini-analogical-prompting | 0.694               |\n| gpt-4o-mini-MRP                  | 0.758               |\n| EPIC                             |          0.894          |\n\n**EPIC vs TATA**:\n| Methods & LLM                    | Accuracy on MATH500 |\n| -------------------------------- | ------------------- |\n|Qwen2.5-Math-1.5B | 0.396  |\n| TATA - QwenMath2.5 - 1.5B        |    0.534        |\n|Qwen2.5-Math-1.5B-Instruct | 0.75  |\n| TATA - QwenMath2.5 - 1.5B - Instruct        |    0.65        |\n| EPIC-QwenMath2.5 - 1.5B - Instruct                       |          0.808           |\n\nWe conducted best-effort reproductions of both MRP and TATA baselines despite limited access to their full implementations (MRP lacks released code and provides only partial prompts, while TATA publishes part of the code only for its second stage). \nFor MRP, we use GPT-4o-mini as the backbone model for both prompt selection and answer generation. To ensure a fair comparison between TATA and EPIC, we use the same generation model (i.e., QwenMath2.5-1.5B-Instruct) for both methods. EPIC outperforms both MRP and TATA by a wide margin, achieving up to **13.6% absolute improvement** over the best baseline. \n\n**Baseline reproducibility notes**:\nOur reproduction of MRP aligns with the original paper, which reports only **marginal improvements** of 0.05% gain in macro average (Table 1). When evaluated with GPT-3.5 (Table 2), MRP even underperforms the basic CoT baseline. This shows that MRP is fragile and lacks robustness.\n\nFor TATA, we instruction-finetuned (using the code in [5]) two models (QwenMath2.5 - 1.5B and QwenMath2.5 - 1.5B-Instruct) on their partially published dataset [2], observing an increase in performance in the Base model but not the Instruct model.\n\n> the aggregation bound is a known results from probability theory\n\nRegarding the work “Olivier Germain, Amaury Habrard, and Édouard Marchand. “PAC-Bayesian C-bounds for Majority Vote.” In Proceedings of ICML, 2009.“, the issue with this citation is mentioned above.\nAs for the [4], we do indeed cite one of their result from Theorem 5.10 in our Lemma A.2. This is used to establish a Poisson approximation that supports a small part of our broader theoretical development.\nHowever, the main aggregation theorems in our work, Theorems 3.1 to 3.3, go significantly beyond these results. Deriving these bounds required non-trivial extensions, including carefully analyzing majority vote, PRM sum, and PRM max aggregation under Gaussian assumptions with detailed probabilistic and optimization-based arguments.\nWe hope this clarifies our theoretical novelty and the specific role of previously known results in our overall derivation.\n\n**References**: \n\n[1] Gao, Peizhong, et al. \"Meta reasoning for large language models.\" arXiv preprint arXiv:2406.11698 (2024).\n\n[2] Xu, Xin, et al. \"Teaching llms according to their aptitude: Adaptive reasoning for mathematical problem solving.\" arXiv preprint arXiv:2502.12022 (2025).\n\n[3] Gou, Zhibin, et al. \"Tora: A tool-integrated reasoning agent for mathematical problem solving.\" arXiv preprint arXiv:2309.17452 (2023).\n\n[4] Michael Mitzenmacher and Eli Upfal. “Probability and Computing: Randomization and Probabilistic Techniques in Algorithms and Data Analysis.” Cambridge University Press, 2017\n\n[5] Tong, Yuxuan, et al. \"Dart-math: Difficulty-aware rejection tuning for mathematical problem-solving.\" Advances in Neural Information Processing Systems 37 (2024): 7821-7846."}, {"review_id": "mqBzYbCOes", "reviewer": "Reviewer_okVb", "summary": "This paper presents EPIC, an “Ensemble PlannIng with Contrastive learning” framework that dynamically selects the most suitable reasoning method for each input query in large language models, balancing accuracy and inference cost. The authors first derive probabilistic accuracy bounds for three common aggregation strategies i.e majority voting, score-sum voting, and score max voting under varying numbers of candidate samples (Sec. 3) . They then introduce a contrastive representation learning approach (Sec. 4) that embeds both questions and reasoning methods into a shared space, using a contrastive loss to pull questions toward their highest-utility method and a regularizer informed by the theoretical bounds. At inference time, EPIC routes each query to the reasoning method with maximal embedding similarity. Experiments on the MATH500 benchmark demonstrate that EPIC reduces token usage by up to 5x for the same accuracy and increases accuracy by ~1 % at fixed cost compared to other baselines. Additional ablations confirm the impact of embedding dimension, trade-off parameters, and transferability to code generation tasks.", "questions": "1. How does EPIC perform if the probability bounds used in the regularizer are noisy? Could the authors report performance when using fewer samples (e.g 40 vs. 80) to estimate\n2. Can the authors comment on wall‐clock training time and GPU/memory usage? Are there ways to amortize the cost (e.g caching scores)?\n3. Beyond reasoning tasks, can EPIC be applied to multi‐modal or retrieval‐augmented generation scenarios? A brief discussion would broaden the paper’s appeal.", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 4, "strengths_and_weaknesses": "Strengths: \n\n1. Rigorous theoretical foundation especially the derivation of explicit probability bounds for the three aggregation schemes is a clear advance, offering practitioners guidance on how sample size affects accuracy.\n2. Incorporating these bounds as a regularizer in the contrastive learning framework is novel and grounds the embedding learning in quantitative performance metrics.\n3. EPIC consistently lies on the Pareto frontier of accuracy versus token cost, achieving the same accuracy as best‐of‐16 with 5× fewer tokens (1859 vs. 10036) and improving max accuracy to 89.4% at lower cost than beam search \n\n\nWeaknesses: \n\n1. The regularizer relies on empirical bounds estimated from 80 samples per question; it is unclear how sensitive EPIC’s performance is to estimation error, especially when N is small.\n2. Experiments are confined to only 2 basic mathematical reasoning (MATH500, GSM8K) and 1 code benchmark. Broader testing on open‐ended QA or real‐world dialog tasks would strengthen claims of generality.\n3. While EPIC reduces inference tokens, the training phase requires generating and scoring many samples per question, which may be prohibitive for very large method universes. \n4. And also The paper fixes the regularization weight of 10^-3 based on an appendix but provides limited intuition on tuning so more guidance or sensitivity plots would aid reproducibility.", "was_revised": true, "final_justification": "I think most of my concerns are resolved and hence i am increasing the scores.", "rebuttal": "We thank the reviewer for constructive and insightful comments. We address your remaining concerns about our work as follows:\n\n> The impact of the number of samples used to estimate bounds for training\n\nThank you for the opportunity to clarify this point in our paper. \n\n- We conducted an ablation study to address your question by varying the number of samples used to estimate the bounds, selecting from the set of 20, 40, and 80. In this experiment, we fixed the trade-off parameter to 1.0 and the regularization weight to $10^{-3}$, while keeping all other settings consistent with the main experiment. The results are summarized in the table below:\n\n| Method | Accuracy | Cost |\n| --------------------- | -------- | ------- |\n| Best-of-8 | 86.6 | 4986.8 |\n| Offline Ada-BoK | 87.0 | 4095.2 |\n| Best-of-16 | 86.8 | 10036.2 |\n| EPIC-1.0 (20 samples) | 87.6 | 6499.4 |\n| EPIC-1.0 (40 samples) | 88.8 | 6248.4 |\n| EPIC-1.0 (80 samples) | 89.4 | 6921.7 |\n\nWe observe that reducing the number of samples used to estimate the bound slightly decreases EPIC's accuracy, which can be attributed to the increased noise in the estimation when using fewer samples. \n\n- To more deeply examine this effect, we conduct an additional analysis. Specifically, we consider the Best-of-N sampling configuration with temperature 0.2 on the MATH500 dataset. For each question, we generate $N$ solutions using the model and apply an aggregation method to derive the final answer. This process is repeated 10 times, and the accuracy is tracked. If increasing $N$ (e.g., from 2 to 80) causes the pass rate for a question to decrease, we categorize that question as belonging to the *failure branch* of the aggregation method.\n\nAlternatively, we can predict whether a sample lies in the failure branch of an aggregation method by using its bound condition, estimated from 20, 40, or 80 samples. The table below reports the accuracy of these bound-based predictions across three aggregation methods:\n\n| Aggregation Method | 20 Samples | 40 Samples | 80 Samples |\n| ------------------ | ---------- | ---------- | ---------- |\n| Majority\\_Vote | 89.8% | 92.6% | 98.0% |\n| PRM\\_Vote\\_Min | 52.4% | 76.6% | 91.4% |\n| PRM\\_Max\\_Min | 78.6% | 85.0% | 95.2% |\n\nWe observe that the bound conditions in Theorems 3.1 and 3.2 for aggregation methods Majority\\_Vote and PRM\\_Vote\\_Min, respectively, achieve high accuracy even with just 20 samples, whereas the bound condition in Theorem 3.3 for PRM\\_Max\\_Min requires more samples for accurate estimation. This analysis explains why EPIC maintains strong performance with fewer samples. It benefits from the fact that Majority\\_Vote and PRM\\_Vote bounds do not require many samples to yield accurate estimates.\n\n> The impact of the regularization weight\n\nWe address your concern by reporting the results of a grid search over the regularization weight, selecting from the set $\\{10^{-4}, 10^{-3}, 10^{-2}\\}$. The results in the table below for EPIC-0.25 and EPIC-1.0 are evaluated on 500 validation questions split from the training set.\n\n| Method | $10^{-4}$ | $10^{-3}$ | $10^{-2}$ |\n| --------- | ----------- | ----------- | ----------- |\n| EPIC-0.25 | 85.2% | 86.6% | 85.8% |\n| EPIC-1.0 | 87.4% | 88.0% | 88.0% |\n\nThe regularization term enforces consistency within a group of reasoning methods that share the same parameters, except for the number of generated answers. In contrast, the contrastive loss encourages a broader separation in routing across all reasoning methods. So the regularization weight regulates the trade-off between capturing local structure (within-group) and global structure (across all methods in the universe). When either component is underweighted, overall performance tends to degrade.\n\n> Application of EPIC to other tasks like multi‐modal,  retrieval‐augmented generation, open‐ended QA, or real‐world dialog\n\nEPIC can be extended to these tasks with suitable modifications. For multi-modal inputs, we can replace the question encoder with a multi-modal embedding model to process cross-modal information. In retrieval-augmented generation, EPIC applies naturally by treating different retrieval and synthesis configurations as reasoning methods. For open-ended QA and real-world dialog, EPIC can reduce computational cost by routing questions to different models or decoding strategies based on utility, while preserving output quality. Appendix D.4.1 includes an experiment demonstrating EPIC’s effectiveness in selecting reasoning methods associated with different generation models. In addition, EPIC can incorporate surrogate utility signals such as toxicity or truthfulness scores by treating them as PRM scores. This allows EPIC to encourage safer and more reliable outputs.\n\n> Computation resources for the training process. Are there ways to amortize the cost (e.g caching scores)?\n\nOur training process consists of two stages: data generation and model training. \n\n- In the data generation phase, we run 81 reasoning methods over 7,500 training questions, generating and scoring 80 answers per question. This step takes approximately 410 GPU hours on an A5000 and requires around 28GB of VRAM to load both the 7B generation model and the 7B score model. This procedure is expensive but essential to know the capabilities of reasoning methods. \n- Once the data is generated, the training phase is lightweight. We train a small transformer-based model with only 22.6 million parameters, requiring just 80MB of VRAM and completing in under 0.3 GPU hours on the same hardware.\n\nWe are considering two approaches to reduce the cost of the data generation step:\n\n- Score caching as you suggested: Since some generation patterns repeat across multiple samples, caching their PRM scores can reduce redundant computations. However, due to the inherent randomness of sampling, the cache hit rate might be relatively low.\n- Fewer samples for bound estimation: As discussed in our response to the question about sample efficiency, some of the probability bounds used for regularization can be accurately estimated with fewer samples. By selectively using fewer samples for these bounds during training, we can reduce the generation cost without significantly affecting performance."}, {"review_id": "t2TlGg2xYy", "reviewer": "Reviewer_okKY", "summary": "This paper proposes EPIC (ensemble planning with contrastive) learning framework. Creates a sample dataset of question answer pairs and the corresponding “utility” with respect to each reasoning method in domain M. Uses contrastive learning to map input questions to an embedding space (learned jointly with the vectors representing the reasoning methods) and aligning the embedding with the reasoning method that would answer it best. Key findings show that EPIC can achieve higher accuracy than large models with significantly lower token count.", "questions": "- What happens when you change the number of reasoning methods it has access to. As it increases, does the approach scale well with the increased number of methods? Qualitatively looking at the reasoning methods in use in the MATH500 experiment, is there an over-reliance on a few? If we remove those, how does the performance update compared with other selection approaches?\n- What are the current failure points that you have observed? Making those clear would be helpful.", "limitations": "There was no limitation section. Please add one to the paper so its clear. The discussion section motivating design choices does not count.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The approach is simple and well explained, with some rigorous bounds on aggregation accuracy.\n- Exciting to see that with smaller 7B models its able to reach performance thresholds of large models under standard use. It shows grounds for its application in the real-world, especially with the supplemental cost analysis.\n- Also a novel and intuitive approach, motivated by the cost limitations in the space.\n\nWeaknesses:\n- The results showed, especially in Table 1 are a bit confusing, especially with the line-based separation. Making the distinction between the methods within that table clearer would be helpful.\n- No limitation section", "was_revised": true, "final_justification": "I originally saw this paper as well-motivated, thoroughly tested, and impactful. I had some concerns on how it scaled and its over-reliance on a few methods, but the author's rebuttal addressed them properly. I have no unresolved issues and see this work as a strong contribution to the community. I maintain my original score of \"Accept\" for those reasons.", "rebuttal": "Thank you for valuing our contributions. Below, we address the remaining concerns you’ve raised:\n\n> The presentation of Table 1\n \nThanks for your comments on this point. We agree that the table needs better clarification, especially regarding the line separations. Although we provided some explanation in the table caption, we would like to elaborate further here.\n\n- The rows above the blue line show the performance of frontier models on the same test set, along with our upper bound, which is calculated as the proportion of questions that can be solved by any method in our reasoning pool.\n- The rows above the green line correspond to individual single-reasoning configurations. Since there are many such configurations, we report only the best accuracy achieved by tuning some parameters. For example, the \"Best-of-2\" row shows the highest accuracy across different temperature settings and aggregation methods.\n- The rows above the brown line include methods that do not support accuracy–cost trade-offs. RA is a simple baseline, while Offline Ada-BoK is a baseline taken from a different paper.\n- Finally, the last two groups present the results of our method and its ablated variants under two trade-off settings: $\\lambda = 0.25$ and $\\lambda = 1.0$.\n\n> No limitation section\n\nThank you for pointing that out. We will include a Limitations section in the revised manuscript.\n\n> The question “What are the current failure points that you have observed?”\n\n- We are unsure about the reviewer's question, but we assume it refers to the failure cases discussed in our three theoretical results in Section 3. We want to emphasize that many questions fall into the failure branch predicted by our theories. Failure branch means if we try to generate more answers from the base model for aggregation, the probability of having a correct final answer tends to 0. Recognizing and understanding these cases is key to the better performance of our method compared to other baselines.\n\n- To illustrate this further, we consider the Best-of-N sampling configuration with temperature 0.2 on the MATH500 dataset. We generate $N$ solutions for each question using the model and apply an aggregation method to derive the final answer. This process is repeated 10 times, and the accuracy is tracked. If increasing $N$ (e.g., from 2 to 80) causes the pass rate for a question to decrease, we categorize that question as belonging to the *failure branch* of the aggregation method.\nAlternatively, we can predict whether a question lies in the failure branch of an aggregation method by using its bound condition, estimated from 80 samples. The table below reports the percentage of questions that fall into the failure branch (measured empirically), along with the accuracy of our theoretical prediction (in parentheses) for two generators and five aggregation methods considered in our paper:\n\n| Aggregation Method | Qwen2.5-Math-7B-Instruct | Qwen2.5-Math-1.5B-Instruct |\n| ------------------ | ------------------------ | -------------------------- |\n| Majority\\_Vote | 16.4% (98.0\t%) | 25.4% (96.0%) |\n| PRM\\_Max\\_Min | 18.0% (91.4%) | 26.6% (95.2%) |\n| PRM\\_Vote\\_Min | 15.8% (97.8%) | 24.8% (96.0%) |\n| PRM\\_Max\\_Last | 17.0% (93.4%) | 27.0% (95.6%) |\n| PRM\\_Vote\\_Last | 16.0% (97.0%) | 24.8% (95.8%) |\n\nWe draw three key takeaways from this analysis:\n- Our theoretical predictions closely match the empirical results, confirming the correctness and reliability of our findings.\n- In Table 1 of our main paper, the best configuration, Best-of-16, with PRM_Vote_Min, achieves 86.8% accuracy. The empirical failure branch rate here is 15.8%, and further analysis shows that 92.4% of the failed questions fall into the failure branch. This confirms that most errors of aggregation methods are due to fundamental limitations captured by our theory. Increasing to Best-of-32 with the same aggregation method only slightly improves accuracy (87.0%), while the failure branch rate for the remaining errors grows to 96.8%. This suggests the model is already operating close to its theoretical upper bound.\n- The 1.5B model exhibits a higher failure rate than the 7B model, reflecting the superior reasoning ability and stability of the larger model. \n\n> Does the approach scale well with the increased number of methods as it increases? \n\nYes, our approach is specifically designed to scale efficiently as the number of reasoning methods grows. \n- In EPIC, each reasoning method is represented as an embedding vector in a shared low-dimensional space. Adding a new method during training simply introduces a small number of additional learnable parameters (e.g., $d = 64$), resulting in linear parameter growth with respect to the number of methods. This design ensures that scalability is maintained without significantly increasing model complexity. At inference time, selecting the best method for a given question involves a similarity search in this embedding space. This search remains fast and efficient even when the method pool is large.\n\n- To empirically test scalability, we conducted an experiment (Appendix D.4.1), expanding the method pool from 81 to 162 by including 81 additional methods generated from a smaller model (Qwen2.5-Math-1.5B), alongside the original techniques from the 7B model. Despite the pool doubling in size, EPIC’s overall accuracy remained nearly unchanged, confirming its ability to maintain performance. More importantly, the larger method pool allowed EPIC to assign simpler questions to the lower-cost 1.5B methods. This significantly reduced overall computational cost without compromising accuracy, demonstrating that EPIC scales effectively in both performance and efficiency as more methods are added.\n\n> Is there an over-reliance on a few? If we remove those, how does the performance update compared with other selection approaches?\n\n- We thank the reviewer for the insightful question. Indeed, our method tends to choose some reasoning methods more frequently. However, we argue that this behavior reflects a strength rather than a weakness. The goal of our approach is to leverage the diversity and complementary strengths of reasoning methods to optimize the trade-off between cost and accuracy. When a particular method aligns well with this objective, it is expected to be selected more frequently. For example, Table 1 shows that the cheapest method, CoT-G, achieves an accuracy of 83.2%. If minimizing cost is a priority, it is intuitive for our policy to prioritize this method. Only the remaining harder questions should be routed to more expensive reasoning strategies.\n\n- To further examine the impact of this potential over-reliance, we conducted a control experiment. We first ran EPIC-0.25 on the test set to record how frequently each reasoning method was selected. Then, we removed the top 5%, 10%, and 15% most frequently used methods (corresponding to the top 4, 8, and 12 methods, respectively), and re-ran the selection process. The results are shown below with the baseline results in the main paper, so that you can compare more easily. The notation EPIC-0.25 (-Top k%) refers to the variant where the top k% most frequently used reasoning methods were excluded from the selection pool.\n\n| Method | Accuracy | Cost |\n| -------------------- | -------- | ------ |\n| Best-of-4 | 86.2 | 2499.4 |\n| Best-of-8 | 86.6 | 4986.8 |\n| Best-of-16 | 86.8 | 10036.2 |\n| MCTS | 85.4 | 4338.1 |\n| Beam Search | 85.2 | 2638.1 |\n| RA | 84.4 | 1752.4 |\n| Offline Ada-BoK | 87.0 | 4095.2 |\n| EPIC-0.25 (Original) | 86.4 | 1859.2 |\n| EPIC-0.25 (-Top 5%) | 86.8 | 2923.5 |\n| EPIC-0.25 (-Top 10%) | 87.0 | 3494.5 |\n| EPIC-0.25 (-Top 15%) | 87.2 | 4582.5 |\n\nThese results show that our approach remains highly competitive even after removing the most frequently selected methods. In fact, the accuracy slightly improves, and the cost increases. This is because, under a trade-off parameter of 0.25, the policy still emphasizes cost more than accuracy. As a result, the most frequently used methods are those with lower cost. When they are removed, some questions are routed to higher-cost methods that offer better accuracy. Despite this shift, EPIC-0.25 continues to outperform all baselines in terms of both cost and accuracy. This result highlights the robustness of our learned selection policy and representation, rather than a mere dependence on a few reasoning methods."}, {"review_id": "1Z6CXbFr4Z", "reviewer": "Reviewer_Q4vo", "summary": "In this paper, the authors have proposed a theoretical framework for training a embedding model based on only the input question to select the best inference time scaling strategy (including base model, decoding strategy, computation budget, and aggregation methods). The extensive experimental results on code and math benchmarks have well verified the effectiveness of the proposed method in reducing token usage and improving the averaged success rate.", "questions": "What are the specific configurations for the best-performing approaches (i.e., EPIC-1.0) on MATH500 and LiveCodeBench?", "limitations": "Yes", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "### Strengths\n\n1. The idea is straightforward and effective.\n2. The writing is clear.\n3. The experiments are extensive and solid.\n\n### Weakness\n\nI think Section 3 has less connections with the following sections, especially Section 4, which serves as the direct intuition of the proposed  method. Towards the paper organization, I would suggest put more experimental results from the appendix in the main paper, and move Section 3 to Appendix to keep the consistency with the proof.", "was_revised": true, "final_justification": "I have no concern with this paper. I decided to keep my score.", "rebuttal": "We’re grateful for the reviewer’s positive feedback and appreciate the opportunity to address the remaining concerns:\n\n> Moving Section 3 to the Appendix and the experiments about code generation to the main paper\n\nThanks for your constructive comments on our paper presentation. \n- We agree that the code generation experiments deserve more visibility and have made adjustments accordingly. In the revised manuscript, we’ve moved most of the experimental setup details to the Appendix and shortened the description in the main text. Additionally, Table 1 has been resized to half-page width. These changes create space to bring the code generation results into the main paper, placing them alongside Table 1 for easier comparison and improved clarity.\n\n- As for Section 3, it contains the core theoretical results that are exploited for the regularization term used in our loss function (introduced in Section 4). Keeping these results in the main body is crucial for preserving the logical flow of the paper and highlighting our main contributions. To conserve space, we’ve kept only the key results in the main text, with all detailed proofs moved to the Appendix.\n\n\n> The specific configurations for the best-performing approaches (i.e., EPIC-1.0) on MATH500 and LiveCodeBench\n\nThank you for the question. The configurations of EPIC-1.0 differ between the MATH500 and LiveCodeBench benchmarks, as they are tailored to the unique characteristics of each task.\n\n- For MATH500, EPIC-1.0 is configured with an embedding dimension of 64, a regularization strength of \\$\\tau = 10^{-3}\\$, and a trade-off parameter \\$\\lambda = 1.0\\$, which prioritizes accuracy. It uses the Qwen2.5-Math-7B-Instruct model as the base generator and math-shepherd-mistral-7b-prm as the reward model. The method pool includes 81 reasoning strategies covering diverse decoding and aggregation configurations. \n\n- For LiveCodeBench, EPIC-1.0 is configured with an embedding dimension of 64 and no regularization ($\\tau = 0$), as aggregation methods are not considered in code-generation tasks. The trade-off parameter is again set to $\\lambda = 1.0$, favoring accuracy. The base models are Qwen2.5-Coder-3B-Instruct and Qwen2.5-Coder-7B-Instruct, which share the same set of hyperparameters as mentioned above. The method pool consists of a custom set of 11 decoding strategies designed for code generation, including Chain of Thought - Greedy (CoT-G) and Best-of-N sampling strategies."}]

Author rebuttal:

Meta-review: This paper proposes EPIC, a novel and well-executed framework for dynamically selecting reasoning methods to optimize the accuracy-cost trade-off in language models. The method is grounded in a rigorous theoretical analysis of aggregation methods, and these derived probability bounds are cleverly incorporated as a regularizer within a contrastive learning framework. The reviewers were unanimously positive, finding the work to be technically solid, original, and impactful. The authors provided a thorough rebuttal with additional experiments that successfully addressed all reviewer concerns, leading to a consensus for acceptance.

* Novelty and Technical Soundness: The reviewers praised the novel integration of theoretical probability bounds with a contrastive learning framework for method selection. They found the derivation of accuracy bounds for aggregation methods to be a "clear advance" (okVb) and a "useful theoretical grounding" (v8TG).

* Strong Empirical Results: The paper demonstrates significant improvements on the Pareto frontier of accuracy versus cost. Experiments show EPIC can achieve the same accuracy as strong baselines with up to 5x fewer tokens and can improve overall accuracy at a lower cost (okVb, okKY). The authors also added strong comparisons against recent baselines during the rebuttal, further validating EPIC's effectiveness (v8TG).

* Clarity and Scalability: The approach was commended for being "simple and well explained" (okKY) and "straightforward and effective" (Q4vo). The authors also addressed scalability concerns by showing that the framework scales efficiently and remains robust even when the pool of available methods is doubled or when the most-used methods are removed (okKY).

* Thorough Rebuttal: The authors provided comprehensive responses and new experiments to address all questions. This included ablations on the sensitivity to estimation samples (okVb), analysis of computational costs (okVb), comparisons to new baselines (v8TG), and robustness checks (okKY), which satisfied all reviewers.

This is a strong paper with clear contributions to an important problem. The combination of theoretical insight and strong empirical validation makes it an excellent fit for NeurIPS. All reviewers (v8TG, okVb, okKY, Q4vo) recommend acceptance.
Decision: Accept (spotlight)