Title: Let's Revise Step-by-Step: A Unified Local Search Framework for Code Generation with LLMs
Venue: NeurIPS 2025
Accepted: yes
Keywords: LLM

Code generation

Local Search
Abstract: Large Language Models (LLMs) with inference-time scaling techniques show promise for code generation, yet face notable efficiency and scalability challenges. Construction-based tree-search methods suffer from rapid growth in tree size, high token consumption, and lack of anytime property. In contrast, improvement-based methods offer better performance but often struggle with uninformative reward signals and inefficient search strategies. In this work, we propose $\textbf{ReLoc}$, a unified local search framework which effectively performs step-by-step code revision. Specifically, ReLoc explores a series of local revisions through four key algorithmic components: initial code drafting, neighborhood code generation, candidate evaluation, and incumbent code updating, each of which can be instantiated with specific decision rules to realize different local search algorithms such as Hill Climbing (HC) or Genetic Algorithm (GA). Furthermore, we develop a specialized revision reward model that evaluates code quality based on revision distance to produce fine-grained preferences that guide the local search toward more promising candidates. Finally, our extensive experimental results demonstrate that our approach achieves superior performance across diverse code generation tasks, significantly outperforming both construction-based tree search as well as the state-of-the-art improvement-based code generation methods.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes ReLoc, a framework for improving LLM-based code generation through iterative revision. The authors frame the problem as a local search, where an initial code draft is progressively refined. The framework can be instantiated with different local search algorithms, with the paper focusing on Hill Climbing (HC) and a Genetic Algorithm (GA). A key contribution is a specialized \"revision reward model\" trained to estimate the \"revision distance\"—the number of steps needed to fix a piece of code. This model is intended to provide a more fine-grained evaluation signal than simple pass/fail rates on test cases. The authors conduct experiments on the LiveCodeBench and TACO benchmarks, claiming that their approach outperforms existing construction-based and improvement-based methods in both performance and efficiency\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**:\n\nThe paper addresses the important and timely problem of scaling LLM inference for complex code generation tasks. The core idea of using a more nuanced reward signal that goes beyond binary pass/fail feedback is a reasonable direction for research. The proposed framework is also flexible in principle, capable of incorporating various search heuristics.\n\n**Weaknesses**:\n\n- The central proposal is to apply classic local search heuristics (Hill Climbing, Genetic Algorithms) to the output of an LLM. These are standard, well-understood algorithms from the field of optimization. While their application to LLM-based code generation is relevant, it represents an incremental step rather than a novel breakthrough. The paper frames this as a \"unified framework,\" but this feels like an overstatement for what is essentially the implementation of two common search methods. The novelty, therefore, rests almost entirely on the \"revision reward model.\"\n\n- While the method shows performance gains, the significance of these gains is debatable. For instance, on the code generation task in LiveCodeBench, ReLoc_HC (33.4%) is only marginally better than the Plan Search baseline (32.7%). The reliance on a complex, separately trained reward model, which itself requires a massive data generation effort, seems like a very high price to pay for such modest improvements. The work adds significant complexity to the overall pipeline for gains that are not consistently transformative.\n\n- The ablation study in Table 3 is meant to show the importance of each component, but it inadvertently raises questions. Removing \"Revision Strategies\" from the Hill Climbing variant results in only a minor performance drop (e.g., 38.4% -> 35.9% on LiveCodeBench). This suggests that the structured, multi-step revision planning, a key part of the described method, has a surprisingly small impact, weakening the overall narrative about the importance of fine-grained, planned revisions.\n\nQUESTIONS:\n- The data generation for the revision reward model, which involves building large code trees via breadth-first search, seems extremely expensive. Could you quantify the computational cost (e.g., total tokens, GPU hours) required to create the training dataset for this model? How does this pre-computation cost weigh against the claimed inference-time efficiency gains?\n\n- The abstract claims a \"37% reduction in token consumption,\" citing Figure 4. Could you please provide a clear, step-by-step calculation showing how this number was derived? Which specific methods, tasks, and performance levels are being compared to arrive at this figure?\n\n- The \"crossover\" operation for the Genetic Algorithm is critical but is described abstractly as prompting an LLM to combine strengths. Could you please provide the exact prompt template (GA_PT) used? Furthermore, what evidence do you have that this approach produces meaningful and diverse offspring, rather than simply averaging or slightly modifying one of the parents?\n\n- The ablation study shows that removing explicit revision strategies causes a relatively small drop in performance. This seems to contradict the paper's emphasis on guided, step-by-step revision. Could you elaborate on why this component has such a limited impact and whether this undermines the core motivation for the framework?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces ReLoc, a new framework for search based code generation. The authors identify that existing methods for code generation using LLMs, such as construction based tree search and improvement based methods, have significant drawbacks. Construction based methods are inefficient and not scalable, while improvement based methods often use uninformative reward signals and inefficient search strategies. To address these issues, ReLoc uses a step by step revision process to improve upon an initial code draft. The framework has four main components:drafting an initial code version, neighborhood code generation, candidate evaluation using reward model, and incumbent code updating. The authors developed a special revision reward model that is used to evaluate the quality of the code candidates. This model is trained to prefer code that is closer to a correct solution, which helps to guide the search. The authors show that ReLoc can be used to implement different local search algorithms like Hill Climbing and Genetic Algorithm. The experimental results show that ReLoc outperforms other methods on two different code generation benchmarks, LiveCodeBench and TACO.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n\n- The proposed ReLoc framework is well designed and the different components are clearly explained. The revision reward model is a clever way to guide the local search and overcome the limitations of using simple pass rates or self evaluation scores. \n- The experimental evaluation is extensive and the results are promising. The authors compare their method with a wide range of baselines on two different benchmarks and show significant improvements in performance and token efficiency.\n- The authors clearly explain the problem with existing methods and motivate the need for a new approach. The ReLoc framework and the revision reward model are explained in a clear and concise way, with helpful figures and algorithms. \n- The paper is original in its approach to code generation. While local search as well as reward models to filter candidates have been used across domains, the proposed combination for code generation is novel. \n- All experiments are conducted using standard open source datasets and models. \n\nWeaknesses:\n- Based on the current set of experiments, it's not clear if the proposed algorithm is equally effective across model families. In particular, do we get similar gains if we use a frontier model like GPT/Sonnet. We expect frontier models to perform well for both initial code generation as well as at revising code based on errors. \n- Paper does not provide insights into what kind of errors are recovered during the revision phrase.\n\nQUESTIONS:\n- Can you provide a comparison of algorithms using more capable frontier models? In particular, use GPT, Sonnet, or Gemini instead of Qwen 32B as the generation model. This experiment will tell us more about whether one search framework is more or less effective with better models.\n- Paper is using Plan then Generate paradigm to create the initial code sample population. How important is the quality of the initial code to the final performance of ReLoc? Have you experimented with other methods for generating the initial code, such as simply sampling from the LLM and using the reward model to select an initial candidate? \n- What is the accuracy of reward model in identifying a correct code sample given a correct and incorrect solution pair on unseen samples?\n- How did you decide 7k token budget for experiments? From Fig 4, it seems ReLoc keep improving with larger context budget compare to other baselines.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel decoding strategy for multi-turn code generation by training a reward model based on revision distance. The authors construct a series of search trees and collect the distance from each node to the correct solution as a reward signal that reflects the node's correctness. A reward model trained on this signal is then used to guide tree search during inference.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- Using edit distance as a supervisory signal for reward model training is interesting and novel.\n\n- The proposed reward model achieves substantial improvements on benchmarks on LiveCodeBench and TACO, outperforming existing reward models by a significant margin. ReLoc also demonstrates strong scaling behavior with increased computational budget.\n\n- The paper includes a number of illustrative diagrams, which enhance the clarity and readability of the methodology and experimental results.\n\n**Weaknesses**\n\n- In my opinion, the effectiveness of the revision distance-based reward depends on the tree construction algorithm. This raises concerns about its generalization across different tree search strategies. As noted in Line 275, various search algorithms have distinct characteristics and application scenarios. It is unclear whether revision rewards collected using specific neighborhood sampling methods (e.g., hill climbing and genetic algorithms employed in the paper) can be reliably transferred to other search paradigms.\n\n- The paper lacks a hyperparameter sensitivity analysis. Several unique hyperparameters are introduced (e.g., d_max and K), yet it remains unclear how their values affect the final performance.\n\nQUESTIONS:\nAfter training, what is the prediction accuracy of the reward model on the edit distance metric over the training and validation sets? Providing this information would help assess how the reward signals can be learned effectively.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this work, the authors present ReLoc, a lightweight and unified local search framework for improvement-based code generation with LLMs. Unlike computationally expensive construction-based inference time scaling methods like ToT and MCTS, ReLoc finds high-quality solutions and enjoys the anytime property by exploring a series of local revisions of an established code sample. Besides, compared to the existing improvement-based methods, ReLoc leverages simple yet effective decision rules to navigate the search space. Furthermore, a specialized revision reward model effectively differentiates code samples based on the potential of each code sample being corrected in future steps, which provides fine-grained preferences when the correctness signal is uninformative. Finally, the authors show the flexibility and expressiveness of ReLoc by developing two well-known local search algorithms, i.e., Hill Climbing and Genetic Algorithm.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n1. The authors propose ReLoc, a lightweight and unified local search framework for code generation.\n\n2.  The authors develop a revision reward model trained with pairwise supervisions derived from revision distance comparisons. \n\n3. The authors conduct extensive experimental evaluations on popular code generation benchmarks.\n\nWeaknesses\n\n1. Although the paper includes some ablation results in the table, it would be better to conduct a more thorough ablation study focusing specifically on the local search algorithm and the revised reward model, as these are the core components of the proposed method.\n\n2. Limited baselines.\nWhile the paper compares against several baselines, the field has progressed rapidly, and many relevant works have been published recently (e.g., in 2025 ICLR). It would strengthen the paper to include comparisons with these newer methods and to discuss their differences more thoroughly.\n\n3. Efficiency of the local search algorithm.\nThe paper claims that the proposed local search algorithm is more efficient, but this is not demonstrated in the experimental results. It would be helpful to include experiments that explicitly evaluate and compare the efficiency of the local search component.\n\n4. Practicality of reward model training.\nTraining the reward model requires 872K training pairs, which seems impractical for real-world scenarios. It is unclear how the revised reward model performs when only a small number of training examples are available. Additional experiments or discussion on the sample efficiency and robustness of the reward model under limited data would improve the paper.\n\nQUESTIONS:\nsee Weaknesses\n\nLIMITATIONS:\nYES", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-05"}