Title: SolverLLM: Leveraging Test-Time Scaling for Optimization Problem via LLM-Guided Search
Venue: NeurIPS 2025
Accepted: yes
Keywords: MCTS

LLM

Optimization Problem
Abstract: Large Language Models (LLMs) offer promising capabilities for tackling complex reasoning tasks, including optimization problems. However, existing methods either rely on prompt engineering, which leads to poor generalization across problem types, or require costly supervised training. We introduce SolverLLM, a training-free framework that leverages test-time scaling to solve diverse optimization problems. Rather than solving directly, SolverLLM generates mathematical formulations and translates them into solver-ready code, guided by a novel Monte Carlo Tree Search (MCTS) strategy. To enhance the search process, we modify classical MCTS with (1) dynamic expansion for adaptive formulation generation, (2) prompt backpropagation to guide exploration via outcome-driven feedback, and (3) uncertainty backpropagation to incorporate reward reliability into decision-making. Experiments on six standard benchmark datasets demonstrate that SolverLLM outperforms both prompt-based and learning-based baselines, achieving strong generalization without additional training.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nSolverLLM is a training-free, test-time–scaling framework that pairs a large language model with Monte Carlo Tree Search to translate natural-language optimization problems into mathematical formulations and solver-ready code.  Its search adds *dynamic expansion* (revisiting and refining partial formulations), *prompt backpropagation* (using solver feedback to guide later LLM prompts), and *uncertainty backpropagation* (down-weighting noisy rewards), all organized around a six-element schema (Type, Sets, Parameters, Variables, Objective, Constraints).  Across six standard benchmarks, SolverLLM surpasses state-of-the-art prompt- and learning-based baselines—improving solving accuracy by up to 10 percentage points while preserving near-100 % code-execution rates, and it does so without any additional training data or fine-tuning.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n\n* Achieves up to 10 pp absolute accuracy gains over state-of-the-art prompt- and learning-based baselines while preserving near-100 % executable code, showing practical value for real-world optimization tasks.\n* Integrates Monte Carlo Tree Search with three novel feedback mechanisms (dynamic expansion, prompt & uncertainty back-propagation) that demonstrably boost solving accuracy without any extra training.\n\n# Weaknesses\n\n* MCTS plus iterative solver calls increase test-time latency and resource consumption. The paper would benefit from concrete runtime and cost statistics to contextualize this overhead\n* I do understand that detail steps of MCTS could be complicate. Despite the detailed write-up already, several steps of the MCTS procedure remain opaque and would profit from clearer explanations or illustrative examples (see questions).\n* The design of prompt back-propagation offers limited novelty, as similar ideas—such as prompt reflection—have been explored in prior work [1][2]. \n* The design of uncertainty back-propagation is somewhat confusing. First, the method already employs the UCB algorithm to balance exploration and exploitation during the selection phase. Second, it uses predictive entropy during prompt back-propagation to determine whether a node should be “activated” for dynamic expansion; however, predictive entropy is generally not a reliable uncertainty metric for guiding selection [3] and it entangles with the UCB equation. Third, it introduces yet another uncertainty estimate by computing a semantic uncertainty score based on the variance of judge rewards from the LLM, and then applies a heuristic reweighting scheme using this variance. The interaction among these three overlapping uncertainty mechanisms feels unnecessarily complex and poorly motivated. Moreover, the repeated sampling required by this setup leads to significant computational overhead. According to Table 3, the ablation study suggests that uncertainty back-propagation contributes little to the overall performance.\n* The experimental comparison of prompt-based methods would be stronger if it incorporated additional MCTS-based baselines, which are more directly relevant to the proposed approach.\n\n\n\n1. Shinn, Noah, et al. \"Reflexion: Language agents with verbal reinforcement learning.\" Advances in Neural Information Processing Systems 36 (2023): 8634-8652.\n2. Zhou, Andy, et al. \"Language agent tree search unifies reasoning acting and planning in language models.\" arXiv preprint arXiv:2310.04406 (2023).\n3. Guo, Pei-Fu, Yun-Da Tsai, and Shou-De Lin. \"Benchmarking Large Language Model Uncertainty for Prompt Optimization.\" arXiv preprint arXiv:2409.10044 (2024).\n\nQUESTIONS:\n* Section 3.2.1 still leaves the relationship between nodes granularity, the six-element schema, and a “complete path” ambiguous. Does one node correspond to a single element (e.g., Variables) or to a full partial formulation spanning several elements? What exactly constitutes a terminal path in the tree?\n* What is x^* on line 174? explain more about the objective_score and \"Optimality\" factor. Is there a solver output as ground truth ? what does it mean by \"estimate relative solution quality even in the absence of exact ground-truth labels or reference objectives\"?\n* How were the MCTS expansion depth, rollout count, and UCB hyper-parameters chosen? A sensitivity study would show whether SolverLLM’s gains persist under tighter time or compute budgets and ensure a fair comparison to prompt-only baselines.\n* How are all uncertainty signals combined, and why are all three necessary? Could they conflict or over-count uncertainty? Empirical evidence isolating each contribution would help establish whether the added complexity is warranted.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a test-time computation scaling method for solving optimization problem with LLMs. The method is built upon MCTS and has three key improvement. First, the authors extend the five-elements abstraction to include another dimension of problem class, which allow the LLM to focus on particular problem types in subsequent modeling. Second, the proposed method allows an internal node to be eligible for expansion if it is marked as active node by the critic. Finally, the authors accumulate the textual feedbacks from critic to guide the further revisions for each layer. Besides, there are also some minor improvements like taking uncertainty into the considerations. Experimental results confirm the merits of the proposed methods.\n\nSTRENGTHS AND WEAKNESSES:\n## Strength\n- Generally, the paper is well-written. The contribution is clearly presented (except for prompt backpropagation point)\n- The inclusion of type element is sensible, serving as a general guideline to narrow down the possible modeling choices for LLMs\n- The use of active nodes allows dynamically retrying at the specific levels, which is also a sensible design choice\n- The integration of knowledge-base for each level provides insights for revising incorrect formulations, akin to improvement-based local search\n\n## Weakness\n- The significance and novelty seem limited to me. Although all improvements to vanilla MCTS seem sensible, they are considered to be moderately important and novel from my perspective.\n- Prompt backpropagation is somewhat misleading. If I understand correctly, the authors use the critic to judge each node in the currently selected reasoning path, and then store the textual feedback to the knowledgebase of each level. If it is the case, there is no \"backpropagation\" happened because the operation can be localized into each individual layer, as long as the partial formulation and the full consequence visible to the critic.\n- The experiments are also not fully clear. How do you terminate your MCTS? How uncertainty threshold $\\eta$ affects the performance? Also, I am expecting the results of token consumption of each methods, which is crucial to ensure a fair comparison for test-time scaling methods.\n- A minor note: In line 216, $\\lambda_s$ should be $\\gamma_s$.\n\nQUESTIONS:\nSee my comments on Strengths And Weaknesses section.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

comments: SUMMARY:
SolverLLM couples LLMs with an MCTS-style search to auto-generate mathematical programs from Natural Language descriptions, then hands the resulting model to Gurobi/Pyomo solvers for solution. Experiments on six academic benchmarks show higher solving accuracy than prior prompt-engineering baselines.

STRENGTHS AND WEAKNESSES:
### Strengths

* Integrates three search refinements (dynamic expansion, prompt back-propagation, uncertainty weighting) into a coherent framework and ablates each component.

* Demonstrates consistent accuracy gains over existing prompt-based methods on benchmark datasets.

### Weaknesses

* **No solver-algorithm contribution.** Because the method defers all heavy lifting to a commercial solver, it adds nothing to algorithmic advances. If the solver already supports the model class, SolverLLM merely adds minutes of latency; 

* **No comparison on solver time cost**. There is no evidence that the solverLLM could provide problem formulation which requires much less time to solve than human experts.

QUESTIONS:
1. Can you show Industry OR results with problems cluster in easy-hard? It would be better to understand where the gain comes from in real-world problems.  

2. Scalability remains speculative.

2. What guardrails prevent LLM hallucinations from generating infeasible or numerically unstable formulations that crash the solver?

LIMITATIONS:
Please refer to questions.
TITLE: Official Review
CLARITY: 3
IMPACT: 1
ORIGINALITY: 3
DATE: 2025-07-03

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes SolverLLM, an LLM-guided MCTS framework for solving optimization problems. SolverLLM contains 4 phases: selection, dynamic expansion, simulation and backpropagation. The main new components in this work are: (1) dynamic expansion of non-leaf nodes in the search tree; (2) prompt backpropagation to incorporate rich non-scalar feedback; and (3) uncertainty backpropagation to deal with the high variance of LLM-based scoring. The authors evaluate their approach on 6 optimization benchmarks, and they demonstrate that SolverLLM consistently improves the performance over baseline methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. Using LLM for optimization problems is an important topic, and this work demonstrates strong results on diverse benchmarks.\n\n2. The new design upon the standard MCTS framework is well-motivated and effective.\n\nWeaknesses:\n\n1. One main weakness of this work is the lack of a rigorous study of the inference cost of SolverLLM and other test-time scaling baselines. Specifically, the authors only demonstrate the final results and average generation time. However, there are no scaling curves to show how the model performance improves with more search budget, such as the number of search iterations, search nodes, etc. Therefore, it is unclear how much improvement SolverLLM provides when fixing the same search budget compared to baselines. It would be more informative to present such test-time scaling curves to compare different methods.\n\n2. Related to the above question, there is no discussion on the token budget of SolverLLM compared to baselines, and how it grows with more search iterations. It is possible that SolverLLM uses a higher token budget in each search iteration compared to baselines.\n\n3. Another major weakness is that the lack of clarity on method description. It would be helpful if the authors can present a small example to illustrate how SolverLLM works, including the concrete prompt in each stage, how does the solver and LLM-generated feedback look like, what is the output format of the LLM-based scorer and how is it used, etc.\n\n4. I am confused about how prompt propagation works. What are the concrete formats of reasoning signals and the trigger, and how are they generated? How do we incorporate these reasoning signals for future prompt construction? Do we concatenate all past evaluation outcomes in the prompt?\n\n5. I didn't find the prompt for generating objective scores. How does the prompt look like? What is the format of the objective score?\n\nQUESTIONS:\n1. Please show scaling curves on how the model performance improves with more search budget, such as the number of search iterations, search nodes, etc.\n\n2. Please add discussion on the token budget of SolverLLM compared to baselines, and how it grows with more search iterations.\n\n3. It would be helpful if the authors can present a small example to illustrate how the full workflow of SolverLLM, including the concrete prompt in each stage, how does the solver and LLM-generated feedback look like, what is the output format of the LLM-based scorer and how it is used, etc.\n\n4. I am confused about how prompt propagation works. What are the concrete formats of reasoning signals and the trigger, and how are they generated? How do we incorporate these reasoning signals for future prompt construction? Do we concatenate all past evaluation outcomes in the prompt?\n\n5. I didn't find the prompt for generating objective scores. How does the prompt look like? What is the format of the objective score?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-20"}