Title: Wider or Deeper?  Scaling LLM Inference-Time Compute with Adaptive Branching Tree Search
Venue: NeurIPS 2025
Accepted: yes
Keywords: inference-time compute

tree search

LLMs

programming

reasoning
Abstract: Recent advances demonstrate that increasing inference-time computation can significantly boost the reasoning capabilities of large language models (LLMs). Although repeated sampling (i.e., generating multiple candidate outputs) is a highly effective strategy, it does not leverage external feedback signals for refinement, which are often available in tasks like coding. In this work, we propose Adaptive Branching Monte Carlo Tree Search (AB-MCTS), a novel inference-time framework that generalizes repeated sampling with principled multi-turn exploration and exploitation. At each node in the search tree, AB-MCTS dynamically decides whether to ''go wider'' by expanding new candidate responses or ''go deeper'' by revisiting existing ones based on external feedback signals. We evaluate our method on complex coding and engineering tasks using frontier models. Empirical results show that AB-MCTS outperforms both repeated sampling and standard MCTS, underscoring the importance of combining the response diversity of LLMs with multi-turn solution refinement for effective inference-time scaling.


Peer reviews:
comments: SUMMARY:
The paper proposes  modified forms of MCTS. In standard MCTS, only leaf nodes are expanded and a fixed set of child nodes are expanded. The modified versions of MCTS instead add a child node (to every node) called GEN, which when called, expands the parent node again. Two selection policies are introduced called AB-MCTS-M and AB-MCTS-A that use either a mixed model for scoring or aggregate node values and are estimated with Gaussian or Beta distributions. The different versions are evaluated on different tasks and perform better than different baselines.

STRENGTHS AND WEAKNESSES:
Strengths: 
- The new MCTS variants introduced are novel and a creative solution to fixed-width MCTS. (Originality) 
- The paper identifies an important weakness of MCTS which is that it favors exploration over exploitation. (Significance)
- The methods do well under smaller generation budget. (Quality) 

Weaknesses:
- While the conceptual motivation is clear, the experimental results do not show large gains from the approach making it hard to justify the extra complexity. (Quality)
- It is not clear what the computation cost is from the new methods compared to typical MCTS. (Clarity)

QUESTIONS:
- What is the computation cost from adaptive MCTS? 
- Why are there two versions of adaptive MCTS? Is one better for some applications than others?

LIMITATIONS:
yes
TITLE: Official Review
CLARITY: 2
IMPACT: 3
ORIGINALITY: 4
DATE: 2025-06-30

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents an adaptive-branching version of MCTS-based LLM inference to find the highest performing sequence under a fixed budget of LLM calls, given an external verifier. The approach uses a Bayesian selection strategy to either generate a new child node or refine an existing child at a particular node in the search tree based on an estimated posterior predictive and Thompson sampling.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:** I like the motivation of the paper and think the presented approach makes sense given that each problem instance may require very different search tree shapes to behave optimally. The paper is written clearly, and the experiments seem thorough.\n\n**Weaknesses:** It would've been useful to tune the standard MCTS baseline to provide a more robust comparison, which would have also strengthened the case for wanting a dynamic branching factor. Additionally, standard deviations for the experiment results would have been very useful since temperature-based LLM sampling can have high variance.\n\nQUESTIONS:\n1. How are the parameters decided for the priors used in the mixed and aggregate models? Do you have experiments evaluating how sensitive the performance is to this choice?\n2. Why is the width for standard MCTS fixed to 5, simply because previous works did that? For a fair comparison, it would be helpful to find the width (even a single setting across tasks) that performs the best and then show a comparison with the adaptive method.\n3. Why are the standard deviations not reported despite repeat runs being executed? It would be useful to conclude whether the reported performance differences are significant or not.\n4. While a reasonable choice, it is unclear to me how much the Bayesian selection mechanism adds value beyond a simpler UCT-style counting mechanism. Related, it remains unclear from the current description in the main text why progressive widening is not applicable in this setting. Could you provide an extended explanation?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper extends MCTS-based LLM test-time computation to include progressive widening, while novelly leveraging statistical information stemming from all actions being drawn from the same LLM distribution, resulting in a potentially more efficient search. Consequently, two different approaches to utilizing this information are introduced: Adaptive-Branching MCTS Mixed-model and Adaptive-Branching MCTS Node-Aggregation. Experiments across LiveCodeBench, CodeContest, ARC-AGI, and MLE-Bench suggest that this approach is more robust in different settings at improving performance through test-time compute, compared to Best-of-N, Self-refine, and MCTS baselines.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n\n* **Problem significance**. Test-time compute has shown great potential and is an area of great relevance and timeliness \\[1\\].  \n* **Originality/impact**. Adaptive branching could hypothetically reach the Pareto frontier for the different test-time compute methods, which seem to perform with different success for different prompts \\[1\\]. Consequently, the ideas proposed in this paper can be valuable to the community, and the different design choices can be progressively improved in future work to approach the Pareto frontier for varying test-time budgets.\n\n## Weaknesses\n\n* **Clarity.** The paper is overall well-written, but the method section lacks precision due to relying solely on text-based exposition instead of presenting the method mathematically. Since this is the core contribution of the paper, I believe a clear explanation of the method (e.g., incorporating elements from Section A.2.2, but ideally with greater clarity) would significantly enhance the paper’s quality.  \n  * A minor suggestion would also be to include in the appendix a walk-through example of each approach, clearly illustrating, for example, the effect of the shared parameters on the overall update for AB-MCTS-M.  \n* **Quality/Evaluation**. I have some doubts about the overall soundness of the empirical evidence provided by the authors.  \n  * **Baselines**. As far as I understand, the baselines used for MCTS (e.g., LATS \\[2\\] and RAP \\[3\\]) consider nodes as steps toward the final answer, rather than as full solutions, which differs from the formalism presented in this work. Therefore, although MCTS on solutions is a valid baseline for this paper, it misrepresents previous works. Vanilla progressive widening could also be included as a valuable ablation.  \n  * **Metrics**. LiveCodeBench and CodeContest are evaluated at Pass@1, while ARC-AGI is evaluated at Pass@2. I would encourage transparency and completeness to provide at least Pass@1 for ARC-AGI, too, and ideally also Pass@2 in the other tasks. Additionally, it is unclear what the metrics displayed in Tables 1 and 2 represent, especially given the inconsistency in Figures 4 and 5\\.  \n  * **Statistical significance**. Gains in Tables 1 and 2 appear modest, especially when compared to Repeated Sampling, and it is unclear whether they are statistically significant, as no error bars are provided. In fact, confidence intervals in Figure 4 and Figure 8 in the appendix seem to suggest that gains are non-statistically significant, and the results do not evidently extend qualitatively to other LLMs. \n\nRegarding my score, the main pain points are the “clarity” point mentioned above and the lack of transparency in the representation of other works from baselines, the metrics provided, and the statistical significance as outlined above. I am not looking for further experiments, and I don’t think modest and mixed performance improvements alone would be a reason to reject, especially given the potential long-term impact this paper could have with follow-up works that find better approaches to the Pareto frontier. Therefore, I invite the authors to address these comments and provide a clearer picture; in that case, I am willing to increase the score to ‘weak accept’. More conclusive results with more relevant improvements would be required for me to raise my score to a full ‘accept’.  \n—  \n\\[1\\] Snell, C., Lee, J., Xu, K., & Kumar, A. (2024). Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters. *ArXiv*. [https://arxiv.org/abs/2408.03314](https://arxiv.org/abs/2408.03314) \n\\[2\\] Zhou, A., Yan, K., Wang, H., & Wang, Y. (2023). Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models. ArXiv. https://arxiv.org/abs/2310.04406\n\\[3\\] Hao, S., Gu, Y., Ma, H., Hong, J. J., Wang, Z., Wang, D. Z., & Hu, Z. (2023). Reasoning with Language Model is Planning with World Model. ArXiv. https://arxiv.org/abs/2305.14992\n\nQUESTIONS:\n* The MCTS baselines with a branching factor of 5 end up generating 2^7-2 branches, or do you cut the last generation to 2 instead of 5? I would clarify this for completeness. \n* In 4.3, is the overall width of the tree really representative of your method? Perhaps you could explore plotting the distribution of node degrees, which I believe would better show the strength of your approach.  \n* What do you think are the following steps to explore to try to reach the Pareto frontier? I believe it would be helpful to include a more comprehensive discussion of different designs, such as including problem difficulty and other metrics used in other works that control the relative benefit of width vs depth.\n\n### Other suggestions\n\n* I agree with the distinction regarding Progressive Widening in the related work. Still, given its relevance as prior work, I would encourage you to make this point more precise, potentially in the appendix.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}