Title: Reasoning Planning for Language Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: reasoning method selection

two-tower framework

accuracy-cost trade-off

inference time scaling
Abstract: Selecting an appropriate reasoning method for a given query remains a key challenge in language model generation. Existing approaches typically generate multiple candidate responses and use an aggregation strategy to select the output answer, often assuming that more candidate answers yield higher accuracy. We revisit this assumption through a rigorous theoretical analysis, deriving accuracy bounds for standard aggregation methods under fixed generation distributions and candidate sizes. Building on these insights, we introduce EPIC, an Ensemble Planning with Contrastive learning framework to learn a shared representation space that captures both model reasoning abilities and query-method compatibility. EPIC incorporates our probability bounds as a regularizer in a utility-driven optimization that balances accuracy and computational cost. Experiments on diverse mathematical reasoning tasks show that EPIC consistently selects optimal reasoning methods, improving accuracy while reducing computational overhead. Our code can be found at https://github.com/nguyenngocbaocmt02/EPIC.


Peer reviews:
comments: SUMMARY:
In this paper, the authors have proposed a theoretical framework for training a embedding model based on only the input question to select the best inference time scaling strategy (including base model, decoding strategy, computation budget, and aggregation methods). The extensive experimental results on code and math benchmarks have well verified the effectiveness of the proposed method in reducing token usage and improving the averaged success rate.

STRENGTHS AND WEAKNESSES:
### Strengths

1. The idea is straightforward and effective.
2. The writing is clear.
3. The experiments are extensive and solid.

### Weakness

I think Section 3 has less connections with the following sections, especially Section 4, which serves as the direct intuition of the proposed  method. Towards the paper organization, I would suggest put more experimental results from the appendix in the main paper, and move Section 3 to Appendix to keep the consistency with the proof.

QUESTIONS:
What are the specific configurations for the best-performing approaches (i.e., EPIC-1.0) on MATH500 and LiveCodeBench?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 4
ORIGINALITY: 3
DATE: 2025-07-02

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes EPIC (ensemble planning with contrastive) learning framework. Creates a sample dataset of question answer pairs and the corresponding “utility” with respect to each reasoning method in domain M. Uses contrastive learning to map input questions to an embedding space (learned jointly with the vectors representing the reasoning methods) and aligning the embedding with the reasoning method that would answer it best. Key findings show that EPIC can achieve higher accuracy than large models with significantly lower token count.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The approach is simple and well explained, with some rigorous bounds on aggregation accuracy.\n- Exciting to see that with smaller 7B models its able to reach performance thresholds of large models under standard use. It shows grounds for its application in the real-world, especially with the supplemental cost analysis.\n- Also a novel and intuitive approach, motivated by the cost limitations in the space.\n\nWeaknesses:\n- The results showed, especially in Table 1 are a bit confusing, especially with the line-based separation. Making the distinction between the methods within that table clearer would be helpful.\n- No limitation section\n\nQUESTIONS:\n- What happens when you change the number of reasoning methods it has access to. As it increases, does the approach scale well with the increased number of methods? Qualitatively looking at the reasoning methods in use in the MATH500 experiment, is there an over-reliance on a few? If we remove those, how does the performance update compared with other selection approaches?\n- What are the current failure points that you have observed? Making those clear would be helpful.\n\nLIMITATIONS:\nThere was no limitation section. Please add one to the paper so its clear. The discussion section motivating design choices does not count.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents EPIC, an “Ensemble PlannIng with Contrastive learning” framework that dynamically selects the most suitable reasoning method for each input query in large language models, balancing accuracy and inference cost. The authors first derive probabilistic accuracy bounds for three common aggregation strategies i.e majority voting, score-sum voting, and score max voting under varying numbers of candidate samples (Sec. 3) . They then introduce a contrastive representation learning approach (Sec. 4) that embeds both questions and reasoning methods into a shared space, using a contrastive loss to pull questions toward their highest-utility method and a regularizer informed by the theoretical bounds. At inference time, EPIC routes each query to the reasoning method with maximal embedding similarity. Experiments on the MATH500 benchmark demonstrate that EPIC reduces token usage by up to 5x for the same accuracy and increases accuracy by ~1 % at fixed cost compared to other baselines. Additional ablations confirm the impact of embedding dimension, trade-off parameters, and transferability to code generation tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n\n1. Rigorous theoretical foundation especially the derivation of explicit probability bounds for the three aggregation schemes is a clear advance, offering practitioners guidance on how sample size affects accuracy.\n2. Incorporating these bounds as a regularizer in the contrastive learning framework is novel and grounds the embedding learning in quantitative performance metrics.\n3. EPIC consistently lies on the Pareto frontier of accuracy versus token cost, achieving the same accuracy as best‐of‐16 with 5× fewer tokens (1859 vs. 10036) and improving max accuracy to 89.4% at lower cost than beam search \n\n\nWeaknesses: \n\n1. The regularizer relies on empirical bounds estimated from 80 samples per question; it is unclear how sensitive EPIC’s performance is to estimation error, especially when N is small.\n2. Experiments are confined to only 2 basic mathematical reasoning (MATH500, GSM8K) and 1 code benchmark. Broader testing on open‐ended QA or real‐world dialog tasks would strengthen claims of generality.\n3. While EPIC reduces inference tokens, the training phase requires generating and scoring many samples per question, which may be prohibitive for very large method universes. \n4. And also The paper fixes the regularization weight of 10^-3 based on an appendix but provides limited intuition on tuning so more guidance or sensitivity plots would aid reproducibility.\n\nQUESTIONS:\n1. How does EPIC perform if the probability bounds used in the regularizer are noisy? Could the authors report performance when using fewer samples (e.g 40 vs. 80) to estimate\n2. Can the authors comment on wall‐clock training time and GPU/memory usage? Are there ways to amortize the cost (e.g caching scores)?\n3. Beyond reasoning tasks, can EPIC be applied to multi‐modal or retrieval‐augmented generation scenarios? A brief discussion would broaden the paper’s appeal.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents EPIC, a framework that helps LLMs choose the best reasoning strategy for each math problem. Instead of applying one fixed method, EPIC learns to match questions with suitable strategies using contrastive learning and a regularization term based on probabilistic aggregation bounds. It offers a theoretical analysis, a learning setup using these bounds, and experiments showing either reduced cost or improved accuracy.\n\nSTRENGTHS AND WEAKNESSES:\nStrength:\n\n- The integration of probabilistic bounds with contrastive learning for reasoning method selection is well-executed and practical.\n- The derivation of accuracy bounds for majority voting, PRM voting, and PRM max aggregation provides useful theoretical grounding.\n\nWeaknesses:\n\n- there is no comparison with various strategy selection techniques such as MRP, RTR, or TATA frameworks\n- Insufficient discussion of how EPIC differs from existing routing approaches\n- the aggregation bound is a known results from probability theory\n\n\nreferences: \n\n-  “Meta Reasoning for Large Language Models” (Meta-Reasoning Prompting, MRP). arXiv:2406.11698 .\n-  Zhihong Pan, Ruohan Zhang, et al. “Route to Reason: Adaptive Routing for LLM and Reasoning Strategy Selection” (RTR). arXiv:2505.19435 .\n-  Xin Xu, Yan Xu, et al. “Teaching LLMs According to Their Aptitude: Adaptive Reasoning for Mathematical Problem Solving” (TATA). arXiv:2502.12022 .\n- Michael Mitzenmacher and Eli Upfal. “Probability and Computing: Randomization and Probabilistic Techniques in Algorithms and Data Analysis.” Cambridge University Press, 2017. \nOlivier Germain, Amaury Habrard, and Édouard Marchand. “PAC-Bayesian C-bounds for Majority Vote.” In Proceedings of ICML, 2009.\n\nQUESTIONS:\n- Can you provide empirical evidence that the theoretical bounds hold in practice for LLM reasoning tasks?\n\nLIMITATIONS:\nDiscussed", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-04"}