Thinking Before Running! Efficient Code Generation with Thorough Exploration and Optimal Refinement (2025)

Abstract: Code generation is crucial in software engineering for automating the coding process efficiently. While test-time computation methods show promise, they suffer from high latency due to multiple computation rounds.To overcome this, we introduce ThinkCoder , a framework that combines thorough exploration with optimal refinement.The exploration phase diversifies the solution space by searching for potential solutions, followed by a refinement phase that enhances precision.This approach allows us to select the best solution through careful consideration before taking action, avoiding excessive trial and error.To further minimize test-time computation overhead, we introduce preference-driven optimization with Reinforced Self-Training (ReST), which uses exploration trajectories from ThinkCoder to guide LLM’s evolution.This approach enhances LLM’s exploration efficiency via preference learning, cutting costs while maintaining accuracy.ThinkCoder boosts the performance with a single LLM, excelling on benchmarks like HumanEval and MBPP. Compared to SOTA models, it improves Pass@1 by 3.0% over MapCoder with just 6.4% of the computation cost.Against AgentCoder, ThinkCoder achieves a 0.5% higher Pass@1 after 2 rounds, outperforming AgentCoder’s 5 rounds.Additionally, ReST with success trajectories enhances efficiency, allowing models like LLaMA2-7B to achieve competitive results using only 20% of the computational resources. These results highlight the framework’s effectiveness and scalability.

## introduction recent research advances indicate that large language models (llms) have demonstrated remark-able capabilities in various programming-related domains, such as code generation @xcite @xcite , code refinement @xcite @xcite @xcite , and software testing @xcite @xcite . by utilizing extensive training data, llms can understand complex programming tasks, generate syntactically accurate code, and even enhance the quality of their solutions through iterative improvement. in the context of code generation, researchers have made significant progress in improving prompts to encourage llms to generate higherquality answers. this includes two main directions: optimizing the path for answer exploration and incorporating rich reflection information. in the first direction of path optimization, for example, self-planning @xcite reduces problem complexity in code generation by introducing a planning phase, which refines the solution generation process. similarly, codecot @xcite improves the exploration by employing a chain-of-thought(cot) prompt, enabling models to reason through each step systematically. in the second direction, the integration of reflection, test cases, and feedback is progressively being used to steer and enhance the generation process. for instance, rethinkmcts @xcite integrates verbal feedback into its monte carlo tree search framework to correct errors and improve code generation. similarly, agentcoder @xcite boosts efficiency and accuracy by using interactions between multiple agents to provide valuable input. despite the significant potential of using cot for planning code improvements, the thought process incurs a substantial token overhead. to enhance efficiency and minimize trial-and-error costs, it is crucial to clarify the optimization direction before implementing refinement. existing methods, such as mapcoder, which leverages four llm-based agents, consume a considerable amount of computational resources. based on these insights, we propose thinkcoder, a novel framework for efficient code generation that combines thorough exploration and optimal refinement. first, we simplified the multi-agent framework into a single exploration agent and a codeverifier. the exploration agent is responsible for in-depth analysis and generating various codes, while exploring diverse test cases to build a robust testing pool. the codeverifier, using the testing pool, independently determines the optimization direction without relying on llms. by continuously improving the testing pool throughout the refinement process, thinkcoder ensures accurate verification of generated code in each round, providing efficient guidance for subsequent optimizations. second, to tackle the additional computational load resulting from extensive exploration, we incorporate a preference-driven optimization phase using reinforced self-training (rest) in thinkcoder. rest leverages successful exploration trajectories to train llm, enabling it to generate optimal solutions more efficiently during exploration. this approach significantly reduces the computational cost of test-time computations, enabling models like llama2-7b to achieve notable improvements in both performance and efficiency. our contributions are as follows: ## related work test-time compute. test-time computing methods use self-play and additional verifiers to enhance performance, despite the computational overhead. for example, training a verifier with preference reward model data and combining strategies like best-of-n weighted, beam search, and lookahead search can help @xcite . toolllm @xcite evaluates tool usage by comparing success rates and solution quality to chatgpt-react, employing tooleval as a verifier for optimal api calls using depth-first search. however, api-based verifiers are limited by training data and struggle with generalization. in contrast, our approach employs a task-based testing tool as a verifier, offering a simpler and more effective solution that is closely aligned with the specific task requirements. ## preference-driven optimization in thinkcoder, the generated code evolves as the pass rate on the testing pool improves. each increase in the pass rate signifies the development of better codes. this progression demonstrates that the exploration agent is gradually producing higher-quality outputs. by collecting the trajectory of these improvements, we aim to fine-tune the model, enabling the llm to generate superior code and test cases more effectively. this approach helps to refine the exploration process, clarify its direction, and reduce the time required for exploration. figure 2 describes the process of trajectory collection and fine-tuning of the llm. we will explain this process from the perspectives of data collection, the model training and inference. ## evaluation ## conclusion in this paper, we introduce thinkcoder, a framework designed to improve the code generation efficiency of llms. the proposed approach incorporates llm's exploration with a non-llm-based codeverifier that greatly reduces the token usage. to further minimize computational costs during exploration, we capture successful trajectories through thinkecoder to fine-tune the llm. this process enhances code generation performance while reducing refinement iterations for optimal solutions. we evaluate our framework across llms of varying sizes and demonstrate its superior performance over current composite models.