SWE-Search: Enhancing Software Agents with Monte Carlo Tree Search and Iterative Refinement
Authors: Antonis Antoniades
Albert Örwall
Kexun Zhang
Yuxi Xie
Anirudh Goyal
William Yang Wang
Venue: ICLR 2025 Poster (2025)
Keywords: agents
LLM
SWE-agents
SWE-bench
search
planning
reasoning
self-improvement
open-ended
Primary area: applications to robotics, autonomy, planning
TL;DR: Introduce an inference-time Monte Carlo Tree Search method for Software Agents.

Abstract: Software engineers operating in complex and dynamic environments must continuously adapt to evolving requirements, learn iteratively from experience, and reconsider their approaches based on new insights. However, current large language model (LLM)-based software agents often follow linear, sequential processes that prevent backtracking and exploration of alternative solutions, limiting their ability to rethink their strategies when initial approaches prove ineffective. To address these challenges, we propose SWE-Search, a multi-agent framework that integrates Monte Carlo Tree Search (MCTS) with a self-improvement mechanism to enhance software agents' performance on repository-level software tasks. SWE-Search extends traditional MCTS by incorporating a hybrid value function that leverages LLMs for both numerical value estimation and qualitative evaluation. This enables self-feedback loops where agents iteratively refine their strategies based on both quantitative numerical evaluations and qualitative natural language assessments of pursued trajectories. The framework includes a SWE-Agent for adaptive exploration, a Value Agent for iterative feedback, and a Discriminator Agent that facilitates multi-agent debate for collaborative decision-making. Applied to the SWE-bench benchmark, our approach demonstrates a 23% relative improvement in performance across five models compared to standard open-source agents without MCTS. Our analysis reveals how performance scales with increased inference-time compute through deeper search, providing a pathway to improve software agents without requiring larger models or additional training data. This highlights the potential of self-evaluation driven search techniques in complex software engineering environments.

Published as a conference paper at ICLR 2025
SWE-SEARCH: ENHANCING SOFTWARE AGENTS WITH MONTE CARLO TREE SEARCH AND ITERATIVE REFINEMENT
Antonis Antoniades Albert Örwall
Kexun Zhang', Yuxi Xie Anirudh Goyal", William Wang! University California, Santa Barbara, "Moatless AI, "Carnegic Mellon University, National University of Singapore, Miil
ABSTRACT
Software engineers operating in complex and dynamic environments must con- tinuously adapt to evolving requirements, learn iteratively from experience, and reconsider their approaches based on new insights. However, current large lan- guage model (LLM))based software agents often follow linear, sequential pro- cesses that prevent backtracking and exploration of alternative solutions, limiting their ability rethink their strategies when initial approaches prove ineffective. To address these challenges, we propose SWE-Scarch, a multi-agent framework that integrates Monte Carlo Tree Search (MCTS) with self-improvement mecha- nism to enhance software agents' performance on repository- levell software tasks. SWE-Search extends traditional MCTS by incorporating a hybrid value function that leverages LLMs for both numerical value estimation and qualitative evalua- tion. This enables self-feedback loops where agents iteratively refine their strate- gies based on both quantitative numerical evaluations and qualitative natural lan- guage assessments of pursued trajectories. The framework includes a SWE--geet for adaptive exploration, Value Agent for iterative feedback, and Discrimi- nator Agent that facilitates multi-agent debate for collaborative decision-making. Applied to the SWE-bench benchmark, our approach demonstrates a 23% rela- tive improvement in performance across five models compared to standard open- source agents without MCTS. Our analysis reveals how performance scales with increased inference-tin compute through deeper search, providing a pathway to improve software agents without requiring larger models or additional training data. This highlights the potential of self-evaluation driven search techniques in complex software engincering environments.
INTRODUCTION
Software engineering is complex and iterative process involving exploration, problem-solving, and decision- decision-making under uncertainty. Tasks such as debugging, feature development, and code refactoring require continuous assessment of different approaches, frequent backtracking, and the incorporation of new information. While machine learning has made progress automating parts of this workflow (Li al.) 2022 OpenAl et al. 2024 Ouyang al. 2022; Yang et al.] 2024b), replicating the adaptive and strategic behavior of human engineers remains a significant challenge. This is due the inherently non-linear and iterative nature of software engineering, where engi- neers dynamically explore various solutions, refine strategies based on feedback, and collaborate identify the most effective path forward. Current large language model (LLM)-based software agents (Xia et al. [2024] Zhang et al.] 2024d), while powerful, often struggle with complex, long- horizon tasks that require adaptive strategies and flexible reassessment over time. These agents can become trapped in repetitive patterns, limiting their effectiveness in tackling more intricate software engineering problems.
Denotes equal contribution. Correspondence to: antonis @ushh.du moatlessai
Code: grt.hbb.cm//ovalllmaattetttteeee Demo: httpa://streem 1amammit.mmm... moat Tess.a1

---

Published as a conference paper at ICLR 2025
To address these challenges, we introduce SWE-Search, multi-agent system that replicates the adaptability, iterative learning, and collaborative decision makiin of human engineers. SWE-Search is designed to address three critical needs in software engineering:
Flexible Exploration and Adaptation: Engincering problems often require exploring multiple ap- proaches and adapting strategies based evolving information (Li et al., 2022). SWE-Search's SWE-Agent operates in a flexible state space, allowing it to fluidly transition between actions such as planning, searching, and editing. This design mirrors the way engineers backtrack and adjust their approach dynamically, ensuring the agent can revise its course when faced with new challenges or information, and points towards the direction of more general, open-ended systems (Wang et al.
2023
Ma et al. [2244a] Lu et al. 2024b; Faldor et al. 2024 Hu et al. 2024] Lu et al. 2024a).
Iterative Learning through Feedback: Effective engineering relies heavily on continuous testing and refinment. To replicate this, SWE-Search integrates Monte Carlo Tree Search (MCTS) (Silver ct al. 2016b) planning module paired with a Value Agent. The MCTS module balances exploration and exploitation to guide the agent through complex solution spaces. The Value Agent augments this process by providing both utility estimates and qualitative feedback, allowing the agent to iteratively improve its decision-making based on past experiences, similar to how engineers refine their work through feedback and debugging.
Collaborative Decision-Mee Decision-Making: Complex problems often benefit from diverse perspectives (Khan ict al. 2024] Amayyelas et al. 2024, Du et al. 2023| Zhang et al. 2024c). In SWE-Search, once a set of potential solutions generated, the Discriminator Agent facilitates multi-agent debate. Each agent advocates for different solutions by presenting arguments, which are critically evaluated by a judge agent. This process mirrors real-world engineering collaboration, where teams deliberate to refine and select the most robust solutions.
We evaluate SWE-Search the SWF-hench-litte benchmark which tests agents' ability to resolve real-world repository-level issues by generating code patches that fix failing tests. SWE-Search demonstrates 23% relative performance improvement across five models compared to standard open-source agents. We explore how performance scales with increased search depth and identify key factors that enhance essf-assesmment in software agents. Our work demonstrates the potential of MCTS and iterative learning to improve agent reasoning and planning in dynamic, complex domains like software engineering, introducing a new paradigm for autonomous software development.
2 RELATED WORK
Search methods Various search approaches have been applied to Large Language Models (LLMs)
to facilitate System 2 (Kahneman] 2011 Saha al. 2024 Pan al.. 2023 Bounsi al.) 2024)
thinking in non-linear reasoning structures. A critical feature of these approaches is their ability to backtrack. Unlike greedy processes (Black 2005), search algorithms explore multiple branches at each step, potentially escaping paths that lead to dead ends. These methods differ in their strategies for exploring and memorizing possible choices, and their heuristics for switching between them. Breadth- first search (Moore 1959) maintains all possible search paths, incurring significant memory and computational costs. Depth-first search (Cormen al., 2009), in contrast, prioritizes the most promising path in more greedy manner. When applied to LLMs, these methods demonstrate trade-off between diversity and quality in text generation (Yao er al., (023). The A algorithm (Hart et al., 1968) combines aspects of breadth-first and greedy search to find optimal solutions using a predetermined evaluation function. In this work, we adopt Monte Carlo Tree Search (MCTS) Coulom 2007), an advanced search algorithm that conducts statistical tree search without requiring dedicated evaluation heuristics for each state. MCTS has achieved impressive results in complex strategy games (Silver et al. 2016b), protein folding (Jumper et al. 2021), and algorithm discovery (Fazzi et al. 2022).
Software Agents Software agents are designed to perform autonomous actions within large code- bases. Given a repository-level task, these agents typically locate relevant files and code segments before implementing necessary changes. We focus on the SWE-bench task (Jimenez et al. 2024), which involves resolving real-world GitHub issues. Among the agents with disclosed technical de- tails on SWE-bench, Yang et al. (2024b) introduced the concept of agent-computer interfaces with SWF-agent. OpenDevin (Wang et al. (024b) presents collection of community-drive agents,
2

---

Published as a conference paper at ICLR 2025
including CodeAct (Wang et al. 2024a). The Agentless approach demonstrated competitive perfor- mance using a simple two-step process of localization and repair. AutoCodeRover (Zhang et al, 2024d) incorporated advanced code tools such abstract syntax trees and spectrum based fault localization. The Alibaba Lingma Agent (Ma et al., 2024b) introduced a search-based approach for repository exploration, followed by structured editing phase. While effective, it constitutes more hand-designed solution specifically designed to interface with the search functionality of their agent. Finally, Brrwn et al. (2024) show that repeated trajectory sampling using the exact same agent/model setup can yield results with high variance.
METHODOLOGY
SWF-Search is a multi-agent system designed to tackle complex software engineering tasks by inte- grating dynamic planning, value estimation, and deliberative decision-makinn making. The core motivation behind this method is to emulate the sophisticated, iterative workflows of human software engi- neers, where exploration, planning, and collaboration are crucial solving intricate problems. By leveraging the strengths of Monte Carlo Tree Search (MCTS) for planning. a Value Agent for utility estimation and feedback, and a Discriminator Agent for final Search provides comprehensive, adaptive framework capable of navigating and solving real-world software engineering challenges.
through debate, SWE-
SWE-Search consists of four primary components that work in synergy:
SWE-Search Framework and Action Agent: Building on the moatless-tools framework (Örwall, 2024), SWE-Search operates in dynamic code environment with flexible state-space and git- like commit tree structure. This design facilitates efficient backtracking to previous states, enabling the Action Agent to explore diverse solution trajectories. The adaptable state-space enhances the system's 's ability to exploit the MCTS module effectively.
To approximate the utility of each observation, we employ an LLM-based value function, which in addition to outputting a value, also generates an explanation in natural be leveraged to improve subsequent actions from parent nodes,
Value (Function) Agent:
language. This explanation enabling iterative celf-improvement the search process.
Search Algorithm: The core of SWE-Search 's exploration strategy is based Monte Carlo Tree Search (MCTS) which uses a heuristic-hased selection process similar to AlphaZero (Silver et al. 2016a), specifically tailored for software engineering tasks. This modified MCTS algorithm effec- tively balances exploration and exploitation, helping the agent explore a diverse set of solutions and converge quickly on the most promising strategies.
Discriminator Agent: In the final stage of SWE-Search, the Discriminator Agent evaluates the solu- tions generated by the search process. Inspired by multi-agent debate frameworks (Du et al. 2023] Khan et al. 2024]] Amayuelas et al., (2024), this agent engages in a structured debate, where multiple agents argue for or against the proposed solutions. The debate process not only surfaces diverse perspectives but also leads to more rigorously justified final decision.
3.1 PROBLEM FORMULATION
The task of the SWE agent can be formalized as a tuple M (S, Po.P). Here, S represents the state space, encompassing all possible states such as the current context of the files the agent working on and overall status of the codebase. The context space, denoted as C, includes metadata about the repository and the initial problem description. The value function V assigns a utility score to each state-action pair (a, t), guiding the agent's 's decisions.
The environment's 's dynamics are defined by coneext-dependent transition function P S S X A x + (SS), which models the evolution of the repository's 's state after each action. The initial state distribution, Po C - ASS), specifies how the initial state depends on the given context, while E (CC) defines the distribution over contexts.
Given initial context P and an initial state so Pu(- c). the SWE agent executes its policy क : S + A(A), which selects actions based on the current state and context. At each time step the agent takes action of type 7, 0g,7 (8t,r, e) and receives corresponding reward
3

---

Published as a conference paper at ICLR 2025
Tree Search
Evaluation
Expansion
seling
Feedback (3)
Execute Action
Feedbaçk
Tespectory Info
Selectec
Contest
Eeecation
LLM
Action Agent
FAIL FAEL
Value
Abammaaoo Aetion
Start
Hirdhgg1/ sedtack
identily
Value
Feedback
GeltdenCCnnted
Figure 1: SWE-Search Overview. Tree search. Each state is represented as a node from which the agent can expand from, and each corresponding action is presented as edge. Evaluation. Uses all relevant context including trajectory information, file context, and executed tests, to provide quantitative value estimation and qualitative explanation in natural language. Expunsion. Nodes can be expanded using value function feedback from future actions.
R(s,,r, dt,,, c). The environment then transitions new state 8t+1 the agent continues to observe this updated state. Over time, this process generates a trajectory := (st,r. Ct, T the agent interacts with the environment.
P(. St.r: Ot c), and
The agent's objective is to maximize the cumulative reward over the trajectory, which is captured by the value function v(st,7; (t {st (si)==0 {a:)/=)) This value function depends only the current state and action but also on the history of previous states and actions, which deviates from the assumptions a Markovian process. Formally, the agent seeks maximize the expected cumulative
reward, defined as: max, VT(p) max, E,
Σ R(St.r, a1
p;
This optimization captures the agent's (in-context) process, as it adjusts its policy क to achieve the highest expected return across multiple trajectories, considering both current and historical informa- tion.
3.2 SWE-SEARCH FRAMEWORK AND ACTION AGENT
The SWE-Search Action Agent builds on the moatlcss-tools framework (Orwall 2024 Its action space, A, is organized as two-tier hierarchy, comprising both action types and their corresponding specific actions. Formally, this can be expressed as A = (t, T, a.) f E T,, a E Ar, where T rep- resents the set of action types (e.g., Search, Plan, Edit), t), and A, the corresponding to each type These actions range from tool invocations and code modifications to the generation of structured text. When action is executed, the Agent transitions state of the corresponding action type $1+1,7 To enhance the base agent's effectiveness in search-driven tasks, we introduced the following modifications:
possible actions
One key modification we implemented is the expansion of the Plan state, allowing it to transition flexibly any other state, rather than being limited transitioning only Edit. This change is motivated by the need to enable more dynamic and adaptive problem-solving behaviors within the agent. In the context of software engineering, rigid state transitions can be overly restrictive. For instance, during code modification tasks, an agent might recognize mid-process that further planning. additional searches, or different types of analysis are necessary before proceeding with edits. Restricting transitions only to editing would artificially constrain the agent, potentially leading it to suboptimal actions or causing it to become stuck in unproductive loops. By allowing transitions information it arises (Fig. [). exploring
any state, empower the agent adapt
4

---

Published as a conference paper at ICLR 2025
wider variety of trajectories. This enhanced flexibility reflects the iterative and often non-linear nature of real software engineering workflows, where engineers frequently revisit planning, testing, and research phases before committing to edits.
Second, the agent is empowered to execute any tests within the codebase at its discretion, as well create and implement function and the agent's subsequent decision-making process. It is crucial to highlight that the tests required resolve given instance (i.e., fail-to-pass tests) are not explicitly revealed the agent. However, the agent can leverage any pre-existing tests within the repository, simulating the behavior of real-world software engineer. We refer this enhanced base agent as Moatless-Adapted,
tests. The results of these tests are incorporated into both the value
3.3 VALUE (FUNCTION) AGENT
The role of the Value Agent extends beyond simply estimating the expected utility of a given state- action pair (ss,, dz). In addition to calculating the value Un. the Value Agent generates written explanation, denoted as E. This explanation serves a dual purpose: it provides transparency into the decision-making process and functions as feedback for the Action Agent, which can leverage this explanation when re-expanding from the parent node of O,, (see Figure hindsight feedback). process, mirroring how
This approach enables the system to iteratively refine its decision-makin a human software engineer continuously re-evaluates their approach based on new information to improve their problem-solving strategy.
The input to the value function consists of all state-action pairs up to and including the current state being evaluated, alongside specific instructions how to assess the state. This allows the Value Agent to contextualize the decision within the trajectory, accounting for the sequence of actions and states leading up to the present. The final output of the value function can formalized as:
(ve, Ex) = V88,. 0e (8thso... e-1, (-. (a:))=0...-1)
(1)
Here, represents the expected utility of the current state-action pair, while e, is the accompanying explanation.
In practice, the Value Agent is tasked with analyzing the entire trajectory leading up to the current state-action pair, providing not only the required utility estimate it, but also a detailed explanation e, This explanation is critical for the agent's 's overall performance, as it offers insight into the rea- soning behind utility estimates, which in turn informs the Action Agent's ' future decisions. We have observed that one of the key factors driving the effectiveness of the Value Agent lies in the clarity and specificity of these explanations. well-articulated explanation can illuminate the strengths and limitations of different state types (e.g., Search, Edit t, P Plan), helping the Action Agent better understand which types of states are more promising or risky to pursue.
By providing detailed feedback on the potential utility of different actions and contextualizing them within the broader trajectory, the Value Agent enables more informed and strategic decision-making by the Action Agent. This integration of both quantitative and qualitative feedback leads to improved performance and more adaptive behavior throughout the task (Fig, 4a).
3.4 SEARCH ALGORITHM
Our search tree is structured with nodes representing states Str and edges representing actions Ar,. The search algorithm employed a modified Monte Carlo Tree Search (MCTS), specifi- cally adapted for the tasks of the SWE-Agent. Unlike prior approaches for web agents that utilize language models in the selection process (Koh et al. 2024 Zhang et al. 2024b), we deliberately choose not rely language models for node selection. Instead, adopt more straightforward heuristic-bart based selection function, similar to the approach used in AlphaZero (Silver et al.] 2016a 2018). This decision is driven by the need for interpretabilityy efficiency, and the focus tasks where heuristic-based exploration suffices to guide the agent effectively through complex software engineering environments.
'This approach aligns with the practices of other SWE agents, and has heen validated hy the authors of SWE--ennh, who confirmed its legitimacy long as the fail-to-pass tests remain concealed from the model.
5

---

Published as a conference paper at ICLR 2025
At the core of our algorithm is modified Upper Confidence Bound for Trees (UCT) selection crite- rion (Kocsis & Szepesvári 2006), which determines the next node to expand. This criterion balances exploitation of known high-reward actions with exploration of less-visited states. We introduce ad- ditional terms to encourage strategic exploration early in the search process, and to penalize over- exploration at later stages when convergence the optimal solution is desired. The modified UCT function is expressed as:
UCT(s,a) = exploilation + exploration + early depth bonus late depth penally (2)
This can be expressed more formally as:
[(s (CT((,))=V(s.d)) ( ((s.a) + ace -8d-11 -vv
V(s, a) is the value estimate of the state-action pair, (s, a) the number of times the state-action pair (s, a) has been visited, (s) is the visit count of state s, d is the depth of the node in the search tree, and C, a, 8, and y are constants that control the balance between exploration, exploitation, and depth-dependent rewards and penalties.
This formulation is inspired by the way software engineers explore potential solutions to a task. In practice, an engineer's search process can be broken down into the following key phases, which our algorithm mirrors:
Early Exploration: Initially, an engineer explores wide variety of potential approaches to fully understand the problem and identify promising strategies. This is encouraged in our algorithm by the deeddddehhbbaalt represented by the term ae -9(d11, which rewards exploration shallow depths, simulating the early phases of wide exploration.
Convergence and Exploitation: As the engineer gains more information and narrows down the op- tions, the focus shifts to exploiting the most effective solution paths. This transition is handled by the standard UCT exploitation term V(s, a) and further reinforced by the late detphdephh.gg wel.detthpeeeattt. (-yvd), d), which discourages over-exploration as the agent delves deeper into the search tree.
Quick Abandonment of Poor Strategies: Software engincers are also adept abandoning poor strate- gies when new information indicates that a particular approach is not viable. We capture this be- havior by implementing simple heuristic rule that abandons nodes associated with consecutive low rewards, ensuring that the agent does not waste resources on unproductive trajectories.
At each step, the node with the highest UCT value is selected for expansion, formalized as:
arg max (S (+.0)
This approach effectively mimics the decision decision-making making process software engineer, who balances exploration of potential strategies with focus converging towards the optimal solution, while remaining flexible enough to backtrack when necessary. By incorporating heuristic feedback and depth-based adjustments, the algorithm avoids getting stuck in unproductive paths and enhances the agent's 's ability to identify high-reward strategies with minimal computational overhead Appendix
3.5
DISCRIMINATOR AGENT
The final stage of SWE-Search involves the Discriminator Agent, whose role to evaluate the candidate solutions generated by the search process and select the one most likely to resolve the issue at hand. This module accepts up to five final solutions produced by the search and engages in multi-agent debate to determine the most promising option. Drawing inspiration from recent work on persuasive multi-agent debates (Khan et al. 2024 Amayuelas et al., 2024). the Discriminator leverages the collective reasoning of multiple agents ensure more robust final selection. The complete prompts used in both the debate rounds and conclusion phase are provided in Appendix Configuration and hyperparameter details can be found in Table 2

---

Published as a conference paper at ICLR 2025
Finiish
STATUB: FAILED
Feadback
"fiier reooon- The re lancion conrectly hand es Eq oxjects and fundtions, and testo are passing, The issue is fully rasolved."
cheeaatt::
readiving the laaredtror
Fistuccfona AAidd the
ted yor de remains, repuiring
impprt Nlttmmnnt for
1urther
Function objects
NNamann Emar.
The path "symyy!
geomerryiun podia"
Edit
STATUS: TESTS FAILED
Edit
STATUS: TESTS PASSED
Fintsh STATUS: RESOLVED
po Lone
"inattucioona SAdu loste
with Eq objects and functionc lowomarryleon
Instruction" Modily
handle
"Srash mexion "ot er Lanctoon has boen supcosstully medifedd handla
and pasca
plarrin Lalepal
beats resoived
"pooudo, codethha unst wille
wit nincr lone Mext
pyl
ppocado, code hayds
The, puth "aymes wow.oooooooodllllig..o..
"naxe"s
cojects functions,
Frnnce conet, "slerned stapa" "Panun veilly dif
iplemned stape "Haau tests vertty
passing."
Figure 2: Hindsight feedback error correction. Instance sympy sympyy-mpp11777 SWE-Search with Qwen2.5-72B I-Intruuct... Initially, the Action Agent performs edits and runs tests, which pass. prematurely concludes the search. Without actually knowing the proposed solution does not resolve the issue, the Value Agent identifies potentially missed tests and assigns low reward. Upon re- expansion using the Value Agent's feedback, new tests fail, prompting the Action Agent to make additional edits, which result in preferred solution which ultimately resolves the issue.
The discriminator stage extends the MCTS framework by enabling direct comparison between can- didate solutions through structured debate. While our value function achieves 73% accuracy in solution evaluation, empirical results show that the discriminator improves final selection accuracy to 84% (Fig. 8a). This improvement is particularly notable for cases where multiple solutions have similar value estimates but differ in implementation details. The structured comparison helps identify subtle trade-offs that may not be captured by the value function alone.
4 EXPERIMENTS
Benchmark For experiments, utilize SWE-bench Lite, curated subset of the official SWE-bench, containing 300 instances. This dataset is specifically designed to self-contained and focuses primarily evaluating functional bug fixes, providing controlled environment assess the performance of our system.
Evaluation Metries We use two metries: resolve rate (Pass@) and @ass@5. Resolve rate the percentage of issues successfully resolved, measuring overall effectiveness. Pass@5 is the percent- age of issues where a correct solution found within five attempts. This allows to assess the efficiency of the search in identifying successful bug fixes within limited number of iterations.
Baselines Software agents leverage diverse tools, architectures, and models, leading to variability in their performance subsets of the SWE-bench Lite dataset (Zhang al. (024a). For compar- ison, we build upon the moatless-tools framework (Orwall) 2024). a high-performing open-source agent commonly used in research settings (Chowdhury al. 2024). To isolate the impact of search approach, we adapt moatless-tools v0.0.2 as our baseline, referred to as Moatless-Adaae This allows to fairly compare the performance of SWE-Search against Moatless-Adapted across various models, including two dlosed-source models (GPT-4o, GPT-4ominn) and three open-soune spen-source models (Qwen2.5-722) 1-222.5577B-Intruuu Instructtt (Yang et al. 2024a), Llama-3.1 1amma-3.1-74 1- 7-0---u-nttt (Dubey et al. 2024) and DeepSeek-V22 V2.5 (DeepSeek-Al et al. 20244). We also reference official moatless- GPT-40 results on SWE-bench Lite to ensure a fair consistent comparison.
identical prompts across all models. In SWE- Search, we limit each node to maximum of three expansions and cap the total search iterations at 100. Further details model hyperparameters can be found in Appendix,
Implementation Details For consistency,

---

Published as a conference paper at ICLR 2025
Table 1: Resolve Rate Comparison, SWE-bench Lite
Model
Moatless-v0.0.2
Mootless-AAapter
SWE-Search
% A
GPT-4o
24.3
25.7
31.0
+17
GPT-44-mini
13.0
17.0
+24
Qwen-2.5-72b-Instteee
18.0
24.7
+27
Deepseek-V2.5
16.3
21.0
+22
Llama-3 3. 1-b Instruct
13.6
17.7
+23
Mean % A
+23
4.1
EXPERIMENTAL RESULTS
4.1.1 SWE-SEARCH SURPASSES ALL CORRESPONDING BASE AGENTS AND ENABLES SMALLER, OPEN SOURCE MODELS TO APPROACH GPT-40
On average, SWE-Search outperforms the baseline agent across all five models, achieving 23% relative improvement (Table Notably, SWE-H,, with Qwen-2.5-72B 1-5-72B-Instruct exceeds the per- formance of GPT-4o using the original Moatless-v0.0.. framework, and closely matches its perfor- mance when compared with the Moatless- Sratless-Adapted agent, with only slight difference (A = - 1%). Interestingly, all five models demonstrate significant improvement when utilizing the proposed ap- proach, with consistent gains across different models.
4.1.2 SEARCH ENABLES AGENTS TO MAKE BETTER USE OF MORE FLEXIBILITY
To prevent goal divergence, most agents, including moatless-tools, rely strict transition rules, where state transitions follow predetermined sequences (e.g.. Search -> Identify, Plan - Edit). In boatless-Adapted
introduce more flexible transition logic that allows Plan state to transition into any other state type. This added flexibility has both advantages and drawbacks. On the positive side, it enables the agent autonomously correct its trajectory without external feedback, particu- larly when the necessary adjustments span only a limited portion the task. However, this increased flexibility also introduces the risk of the agent becoming trapped in infinite loops. Without high- level control mechanism to detect and mitigate these situations. the agent may fail to recover from such loops. This trade-off's is evident in the modest performance difference between Moatless-v0.0.2 and Moatless-Adapted, with slight performance improvement of only 1.4% (Table
4.1.3 IMPACT OF HINDSIGHT FEEDBACK ON AGENT PERFORMANCE
One key advantage of utilizing LLMs for value estimation is their dual ability to provide both quan- titative value estimates and qualitative assessments in natural language. These qualitative insights can significantly enhance the agent's 's action generation and search process by offering detailed feed- back on potential errors or overlooked aspects of the task. In practice, feedback was also crucial in eliciting diversity in the actions taken by the agent, as without it, the agent would often take very similar actions when re-expanding from a parent node.
As shown in Figure this mechanism plays a critical role in improving the agent's performance. During the initial expansion, the agent prematurely concludes that the task is complete. However, the value function correctly identifies gaps in the test coverage, specifically in addressing potential corner cases, and assigns a low reward. This feedback prompts the agent to re-expand the parent state, leading to the introduction of new tests, which subsequently fail. The agent then performs series of edits (summarized in the figure for brevity), ultimately resolving the task correctly. Em- pirically, observe that the instances unresolved by Moatless-Adapted but successfully solved by SWE-Search are often attributed to this search-and-feedbac" loop, where iterative feedback drives the agent toward correct solution.
4.2 IMPORTANCE OF COMPREHENSIVE STATE INFORMATION FOR VALUE FUNCTION PERFORMANCE
8

---

Published as a conference paper at ICLR 2025
lileniiyy
b
Issues Resolved vs. Transitioss
100
RequarttionCCnteet
Requastieersartoo
80
Resolved lesues (m]
Poodback
<Rewards:
Rowardi
60
<Eaptssations The
Esplanatiore: requial
QuerySet_ batchhed Insert contert cucucil tangates
directty Impactiul for implemerting Rs TURNING civsows tur diveon?
40
Model
prosooe changes.. Exolinng the current code and supgessing speciit moditicatiraa would
This metted te key for
RETURNING classes bulk
20
CPT-4o
4PT-4o
salueie The
create with contier upditee, This lasy ackrton process sign ficarity.
Dwwr22 3-720-tmstrct
DeepSeck-V25
Usma- 1-700-irssttt
outpue
100
Transitions
Figure 4: (a) Importance of state-specific value prompts. On the left and right are the respective Value Agents' outputs with and without state-specific prompts. While the action in both cases is effective in finding the right file, the non-state -seat--peccific scenario does not recognize this and assigns low reward. On the contrary, the state-specific prompt correctly assigns high reward to this state. (b) Performance scaling with search depth across different language models. The graph shows the number of issues resolved as function of the number of transitions (search iterations) for all models used.
Model
Pass@1
Pass@5
The effectiveness of SWE-Search hinges on the value function's ability to accurately differentiate between desirable and undesirable states, and provide ac- tionable feedback that drives improvement. However, our experiments revealed that the value function some- times failed to recognize critical decision points in the search tree. It frequently misinterpreted the purpose of certain actions, leading to the undervaluation of effec- tive strategies by assigning low rewards. As shown in Figure 4h before the introduction of state-specific value prompts, the agent consistently assigned low rewards even when the Action Agent correctly identified the need for additional context, such as locating relevant files. This issue persisted despite the agent successfully identifying the files later. By implementing state-specific prompts across core state clusters (Searching, Planning, Edit- ing), the value function became significantly more adept interpreting the intent behind actions and evaluating their outcomes within each state. For further details on experiments distinguishing between effective and ineffective states, refer Appendix 8
GPT-4o
31.0
34.0
GPT-4o-mini
17.0
22.3
Owen-2.5-72b--nsttmn
24.7
25.7
Deepseek-V2.5
21.0
23.3
1iima-3.--70--nsttee
21.0
22.3
Figure 3: SWE-bench SWE-Search results
Scaling SWE agents with Inference-time Compute The success of large language models (LLMs) has traditionally been attributed to the expansion of training data and model size, i.e., training-time compute (Wei et al. 2022 Chung et al. 2022). Recently, researchers have started ex- ploring how different methods scale with inference-time (OpenAl 2024 Snell et al. 2024 Dubey et al. 2024 Here, we study the performance of software engineering agents through increased inference-ting time compute. As shown in Figure 4b, increasing search iterations leads to consistent rise in the number of resolved issues. To ensure experimental feasibility across the 300 instances in the SWE-hench Lite dataset, we applied conservative parameters (maximum iterations = 100, maximum expansions per node = 3). Approaches like SWE-Search enable the allocation of greater resources to specific challenges, such as addressing critical software vulnerabilities (Rigaki et al. Fang el al. 2024), offering scalable solution to complex tasks.
2024
Convergence of Value Function and Discriminator to Right Solution The search process can yield multiple proposed solutions. Ideally, the mean trajectory value of the the proposed solution that resolves the issue will always be the highest, which would yields the ideal performance of the agent (Figure 3) In practice, the value function successfully converged on the correct solution 73% of the time average across the five models. The discriminator module performed even better, increasing the proportion of correct solutions selected to 84%. While in typical large action spaces, Monte Carlo Tree Search (MCTS) is run for thousands of iterations (Silver al. 2016b).

---

Published as a conference paper at ICLR 2025
a
Progorion Correct Solutions
Ovariap of Resolved issues
GPT4c
1.0
0.95
GPT-4amm
Ouend 5-250-rratre
DeepSeel-322
0.8-
0.77
0.79
Uama-d3
0.8%
0.62
2.3%
(1]
0.8%
0.6
(2)
5%
(7)
0.4
2115
0.38
76
(2)
0.2
Discriminator Moan Value
9.3%
(1)
(4)
0.0
GPT-40
GPT-40 mini
Owen2.5 Instruci
DeepSesk-V2
Liama 3.1 9.1-708 Instruci
0.4%
Figure 5: (a) Value Function vs. Discriminator Comparison. Comparison of value function discriminator ability to discern the final solution that resolved the issue when there is one. The discriminator performs better across all models except GPT-4o-mini. DeepSeck-V22 V2.5 had the smallest disparity between the two methods, suggesting ability function. (b) Model-Specifie Issue Resolution. Venn diagram of resolved issues by model. Each model can solve handful of unique instances.
well-calibrated value
the value function's 's success rate remains impressive given the computational constraints. However, SWE-Search could further benefit from enhanced methods for identifying the correct solutions more consistently, allowing it to fully reach its potential.
Different Models can Resolve Vastly Different Issue Subsets When comparing the resolved instances across the five models, we observed significant diversity in the subsets of issues each model successfully solved. As shown Figure 5 each model managed to resolve least one unique instance. Notably, surprising number of issues (33) were solved by other models but not by GPT-4o. This suggests that model diversity could play an important role, at least short term, in enhancing the performance of SWE-agents.
5 DISCUSSION AND CONCLUSION
In this paper. we introduced SWE-Search, a general framework that integrates Monte Carlo Tree Search (MCTS) and qualitative feedback to enhance the performance of software engineering agents. The proposed approach demonstrated improvements over different baseline models, highlighting the potential of search-based methods in software engineering tasks.
One of the key advantages of search-based based approaches, demonstrated in our work, is their ability to scale performance with increased inference-time compute. This flexibility allows the system to adapt to problems that require higher computational resources, such discovering software vul- nerabilities or even generating large codebases from scratch. Future research should focus on two main directions: (a) investigating how search agents scale with computational resources, and (b) expanding the application of software agent search to a broader range of complex use cases.
Given that search techniques like MCTS closely resemble the problem-solving processes of human software engineers, we expect these methods to become increasingly prevalent in agent-driven sys- tems. As the nature of software engineering tasks evolves, system architectures will need become more fluid and adaptable, fully leveraging the potential of search-based techniques. This evolution will likely lead the development of larger, more general agentic systems capable of tackling wide array of software engineering challenges.
ACKNOWLEDGMENTS
Research was sponsored by the U.S. Army Research Office and accomplished under cooperative agreement W91 1O1INF-11922000 for the Institute for Collaborative Biotechnologies.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Alfonso Amayuelas, Xianjun Yang, Antonis Antoniades, Wenyue Hua, I Liangming Pan, and William Wang. Multiagent collaboration attack: Investigating adversarial attacks in large language model collaborations via debate, 2024. URL
Paul E. Black. greedy algorithm, feb 2005. URL greedya go. html Accessed: TODAY.
HTMLL
Wilffried Bounsi, Borja Ibarz, Andrew Dudzik, Jessica B. Hamrick, Larisa Markeeva, Alex Vitvit- skyi. Razvan Pascanu, and Petar Veličković. Transformers meet neural algorithmic reasoners, 2024.
org/ ht////////// 09308
Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoe V. Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling, 2024.
org/ abs /2407 21787
Neil Chowdhury, James Aung, Chan Jun Shern, Oliver Jaffe, Dane Sherburn, Giulio Starace, Evan Mays, Rachel Dias, Marwan Aljubeh, Mia Glaese, Carlos E. Jimenez, John Yang, Kevin Liu, and Aleksander Madry. Introducing SWE-bench verified, August 2024. URL htttp:// openai.
som/research. int int rntucinoonnnn
www-bbbnnhhheeeeeeeeeeee verified OpenAl Blog.
Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghami, Siddhartha Brahma, Albert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdhery, Alex Castro-Ros, Marie Pel- lat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu, Slav Petrov, Ed H. Chi, Jeff Dean, Jacob Devlin, Adam Roberts, Denny Zhou, Quoc V. Le, and Jason Wei. Scaling cnerrcction-inntuunee language models, 2022. URL htt./arxiivv.///i org. abs/ /210 11416
Thomas H. Cormen, Charles F. Leiserson, Ronald I.. Rivest, and Clifford Stein. Introduction to Algorithms, Third Edition. The MIT Press, 3rd edition, 2009. ISBN 0262033844.
Rémi Coulom. Efficient selectivity and backup operators in monte-carlo tree search. In H. Jaap van den Herik, Paolo Ciancarini, and H. H. L. M. (Jeroen) Donkers (eds.). Computers and Games, pp. 72-83, Berlin, Heidelherg, 2007. Springer Berlin Heidelberg. ISBN 978-3-540-75538-8;
DeepSeck-AI, Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, Runxin Xu, Y. Wu, Yukun Li, Huazuo Gao, Shirong Ma, Wangding Zeng, Xiao Bi, Zihui Gu, Hanwei Xu, Damai Dai, Kai Dong, Liyue Zhang, Yishi Piao, Zhibin Gou, Zhenda Xie, Zhewen Hao. Bingxuan Wang, Junxiao Song, Deli Chen, Xin Xie, Kang Guan, Yuxiang You, Aixin Liu, Qiushi Du Wenjun Gao, Xuan Lu, Qinyu Chen, Yaohui Wang, Chengqi Deng, Jiashi Li, Chenggang Zhao, Chong Ruan, Fuli Luo, and Wenfeng Liang. Deepseek-coder-v2 Breaking the barrier of closed-source models in code intelligence, 2024. URL httth:tt///aaaii
org/ abs / 2406 11931
Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. Improving factuality and reasoning in language models through multiagent debate, 2023. URL https:
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshor,, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern. Charlotte Caucheteux. Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell, Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song. Danielle Pintz, Danny Livshits, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovic, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Ander- son, Graeme Nail, Gregoire Mialon, Guan Pang, Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Ma- hadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy
11

---

Published as a conference paper at ICLR 2025
Fu, Jianfeng Chi, Jianyu Huang, Jiawen Liu, Jie Wang, Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe, Junteng Jia, Kalyan Vasuden Al- wala, Kartikeya Upasani. Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Lauren Rantala- Yeary, Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Man- nat Singh, Manohar Paluri, Marcin Kardas, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kambadur, Mike Lewis, Min Si, Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladri Chatterji, Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang. Pengwei Li, Petar Vasic, Peter Weng. Prajiwal Bhar- gava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sum- baly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang. Saghar Hosseini, Sahana Chennabasappa, Sanjay Singh, Sean Bell, Seohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan, Shruti Bhosale, Shun Zhang, Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Seialom, Tobias Speckbacher, Todor Mihaylov, Tong Xiao, Ujjwal Kar,, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do, Vish Vogeti, Vladan Petro- vic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers. Xavier Martinet, Xiaodong Wang, Xiaoqing Ellen Tan, Xinfeng Xie, Xuchao Jia, Xuewei Wang. Yaelle Goldschlag, Yashesh Gaur, Yasmine Babaei, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh, Aaron Grattafiori, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg. Alex Vaughan, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Anam Yumus, Andrei Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, Annie Franco, Aparajita Saraf, Arkabandhu Chowdhury, Ashley Gabriel, Ashwin Bharambe, Assaf Eisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock, Bram Wasti, Bran- don Spence, Brani Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Mejia, Changhan Wang. Changkyu Kim, Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai,, Chris Tindal, Christoph Feichtenhofer, Damon Civin, Dana Beaty, Daniel Kreymer, Daniel Li, Danny Wyatt, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang, Due Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Eleonora Presani, Emily Hahn, Emily Wood, Erik Brinkman, Esteban Ar- caute. Evan Dunbar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Firat Ozgenel, Francesco Caggioni, Francisco Guzmán, Frank Kanayet, Frank Seide, Gabricla Medina Florez, Gabriclla Schwarz, Gada Badeer, Georgia Swee, Gil Halpern, Govind Thattai, Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayanann Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Aspegren, Hunter Gold- man, Ibrahim Damlaj, Igor Molybog, Igor Tufanov, Irina-Elena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Japhet Asher, Jean- -aptistt Gaya, Jeff Marcus, Jeff Tang, Jennifer Chan. Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang. Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torre, Josh Ginsburg, Junjie Wang, Kai Wu, Kam Hou U. Karan Saxena, Karthik Prasad, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kun Huang. Kunal Chawla, Kushal Lakhotia. Kyle Huang, Lailin Chen, Lakshya Garg, Lavender A. Leandro Silva, Lee Bell, Lei Zhang, Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Maria Tsimpoukelli, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev Maxim Naumov, Maya Lathi, Meghan Ke- neally, Michael I.. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Mo- hammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navy- ata Bawa, Nayan Singhal, Nick Egebo, Nicolas Usunier, Nikolay Pavlovich Laptev, Ning Dong, Ning Zhang, Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji, Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj. Qian Liang, Rachad Alao,
12

---

Published as a conference paper at ICLR 2025
Rachel Rodriguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang, Rohan Maheswari, Russ Howes, Ruty Rinott, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Sa- tadru Pan, Saurabh Verma, Seiji Yamamoto, Sharadh Ramaswamy, Shaun Lindsay, Shaun Lind- say, Sheng Feng, Shenghao Lin, Shengxin Cindy Zha, Shiva Shankar, Shuqiang Zhang. Shuqiang Zhang, Sinong Wang, Sneha Agarwal, Soji Sajuyigbe. Soumith Chintala, Stephanie Max. Stephen Chen, Steve Kehoe. Steve Satterfield, Sudarshan Govindaprasad, Sumit Gupta, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Kohler, Thomas Robinson, Tianhe Li, Tianjun Zhang, Tim Matthews, Tim- othy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi, Victoria Montanez, Vijai Mohan, Vinay Satish Kumar, Vishal Mangla, Vitor Albiero, Vlad Ionescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Con- stable, Xiaocheng Tang, Xiaofang Wang. Xiaojian Wu, Xiaolan Wang, Xide Xia, Xilun Wu, Xinbo Gao, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi, Yenda Li, Yilin Zhang, Ying Zhang, Yossi Adi, Youngjin Nam, Yu, Wang, Yuchen Hao, Yundi Qian, Yuzi He, Zach Rait, Zachary Vitoto, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, and Zhiwei Zhao. The llama 3 herd of models, 2024. org/ abs 240.. 21783
URL hhttps://arxt
Maxence Faldor, Jenny Zhang, Antoine Cully, and Jeff Clune. Omni-epic: Open-endedness via models of human notions of interestingness with environments programmed in code, 2024. URL
atttps://ariiss www.aasg///ooooooooo//.o../aaaaa abs///a.//24 15568 15568
Richard Fang, Rohan Bindu, Akul Gupta, Qiusi Zhan, and Daniel Kang. Teams of Ilm agents can exploit zero-day vulnerabilities, 2024. URL https::/arxiv..rr//// abs 2406. 01637
A. Fawzi, M. Balog, A. Huang. T. Hubert, B. Komera-Paredess M. Barekatain, A. Novikov, J. R.
Ruiz, J. Schrittwieser, G. Swirszez, D. Silver, D. Hassabis, and Kohli. Discovering faster matrix multiplication algorithms with reinforcement learning. Nature, 610(7930):47-53, 2022. doi: 111103//441566-02--551111
Peter E. Hart, Nils J. Nilsson, and Bertram Raphael. A formal basis for the heuristic determination of minimum cost paths. IEEE Trans. Syst. Sei. Cybem., 4(2):100-107 1968. doi: 10.1109/TSSC. 1968.300136. URL http:///////........0.000/1////0
1968. 300136
Shengran Hu, Cong Lu, and Jeff Clune. Automated design of agentic systems, 2024. URL https:
7 //arxiv.org/aaaaa....
08435
Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. Swe-bench: Can language models resolve real-world github issues?, 2024. URL
httg//b/////. https: //tp///avvvv////////a//eaa/aaaaa org/ abs/ 0ttps/2310.6777
J. Jumper, R. Evans, A. Pritzel, T. Green, M. Figurnov, O. Ronneberger, K. Tunyasuvunakool, R. Bates, A. Žídek, A. Potapenko, A. Bridgland, C. Meyer, S. A. A. Kohl, J. Ballard, A. Cowie, B. Comera-Paredes, S. Nikolov, R. Jain, J. Adler, T. Back, S. Petersen, D. Reiman, E. Clancy, M. Zielinski, M. Steinegger, M. Pacholska, T. Berghammer, S. Bodenstein, D. Silver, O. Vinyals, A. W. Senior, K. Kavukcuoglu, P. Kohli, and D. Hassabis. Highly accurate protein structure pre- diction with AlphaFold. Nature, 596(7873):583-58) 2021. doi: 10.1038/s41186- 12188-002-0381--22
Daniel Kahneman. Thinking, fast and slow. Farrar, Straus and Giroux, New York, NY, US, 2011. ISBN 978-0-374-27563-1
Akbir Khan, John Hughes, Dan Valentine, Laura Ruis, Kshitij Sachan, Ansh Radhakrishnan, Edward Grefenstette, Samuel R. Bowman, Tim Rocktäschel, and Ethan Perez. Debating with more per- suasive llms leads to more truthful answers, 2024. URL /ttttttsss///////ii/ http::::////xx..oo//abb////// 06782
Levente Kocsis and Csaba Szepesvári. Bandit based monte-carlo planning. Johannes Fürnkranz, Tobias Scheffer, and Myra Spiliopoulou (eds.), Machine Learning: ECMI. 2006, pp. 282-293, Berlin, Heidelberg, 2006. Springer Berlin Heidelberg. ISBN 978-3-540-46056-5.
Jing Yu Koh, Stephen McAleer, Daniel Fried, and Ruslan Salakhutdinov. Tree search for language model agents, 2024. URL ht /ttpp//////////////eaeaaeaa http://wwww..opgp///////.////../
13

---

Published as a conference paper at ICLR 2025
Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d' d'Autume. Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, James Molloy, Daniel J. Mankowitz, Esme Sutherland Robson, Push- meet Kohli, Nando de Freitas, Koray Kavukcuoglu, and Oriol Vinyals. Competition-level code generation with alphacode. Science, 378666244 2022. doi: 10.1126/science.ab bbbp://ceenceag...l.... 158.
URL http::tt////m www. science. org. doi
wttp://www.cccennneee...mm science.ecccccccc abgpa/ee abg1158
Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The ai
scientist: Towards fully automated open-ended scientific discovery, 2024a. URL https:
//arxiv.org/ab///22/ http://aa//22...ee 06292
Cong Lu, Shengran Hu, and Jeff Clune. Intelligent go-explore: Standing on the shoulders of giant foundation models, 2024b. URL https:::: /ttp://www...rra//////./....01..
Yecheng Jason Ma, William Liang, Guanzhi Wang, De-An Huang, Osbert Bastani, Dinesh Jayara- man, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Eureka: Human-level reward design via coding large language models, 2024a. URL /tRt:sts:://// /ttp//aaw...g.a..aaaaa///// abs /a.b//22310 12931
Yingwei Ma, Qingping Yang, Rongyu Cao, Binhua Li, Fei Huang, and Yongbin Li. How to under- stand whole software repository?, 2024b. URL https:://axxi..ot
/bss/206... 01422
E.F. Moore. The Shortest Path Through a Maze, Bell Telephone System. Technical publications. monograph. Bell Telephone System., 1959. URL
publiiical
[d=IVZBHAAACAAJ
OpenAl. OpenAl ol System Card, September 2024. URL https://openai..
research/ol
system-------- eesearch/olssysttemm
card Online report.
OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidr, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Moham- mad Bavarian, Jeff Belgum, Irwan Bello, Jake Berdine, Gabriel Bernadett-Shapirg Christopher Berner, Lenny Bogdonoff, Oleg Boiko, Madelaine Boyd, Anna-Luisa Brakman, Greg Brock- man, Tim Brooks, Miles Brundage, Kevin Button, Trevor Cai, Rosie Campbell, Andrew Cann, Brittany Carey, Chelsea Carlson, Rory Carmichael, Brooke Chan, Che Chang, Fotis Chantzis, Derek Chen, Sully Chen, Ruby Chen, Jason Chen, Mark Chen, Ben Chess, Chester Cho, Casey Chu, Hyung Won Chung, Dave Cummings, Jeremiah Currier, Yunxing Dai, Cory Decareaux, Thomas Degry, Noah Deutsch, Damien Deville, Arka Dhar, David Dohan, Steve Dowling, Sheila Dunning, Adrien Ecoffet, Atty Eleti, Tyna Eloundou, David Farhi, Liam Fedus, Niko Felix, Simón Posada Fishman, Juston Forte, Isabella Fulford, Leo Gao, Elie Georges, Christian Gib- son, Vik Goel, Tarun Gogineni, Gabriel Goh, Rapha Gontijo-Lopes, Jonathan Gordon, Morgan Grafstein, Scott Gray, Ryan Greene, Joshua Gross, Shixiang Shane Gu, Yufei Guo, Chris Hal- lacy, Jesse Han, Jeff Harris, Yuchen He, Mike Heaton, Johannes Heidecke, Chris Hesse, Alan Hickey, Wade Hickey, Peter Hocschele, Brandon Houghton, Kenny Hsu, Shengli Hu, Xin Hu, Joost Huizinga, Shantanu Jain, Shawn Jain, Joanne Jang, Angela Jiang, Roger Jiang, Haozhun Jin, Denny Jin, Shino Jomoto, Billie Jonn, Heewoo Jun, Tomer Kaftan, Lukasz Kaiser, Ali Ka- mali, Ingmar Kanitscheider, Nitish Shirish Keskar, Tabarak Khan, Logan Kilpatrick, Jong Wook Kim, Christina Kim, Yongjik Kim, Jan Hendrik Kirchner, Jamie Kiros, Matt Knight, Daniel Kokotajlo, Eukasz Kondraciuk, Andrew Kondrich, Aris Konstantinidis, Kyle Kosic, Gretchen Krueger, Vishal Kuo, Michael Lampe, Ikai Lan, Teddy Lee, Jan Leike, Jade Leung, Daniel Levy. Chak Ming Li, Rachel Lim, Molly Lin, Stephanie Lin, Mateusz Litwin, Theresa Lopez, Ryan Lowe, Patricia Lue, Anna Makanju, Kim Malfacini, Sam Manning. Todor Markov, Yaniv Markovski, Bianca Martin, Katie Mayer, Andrew Mayne, Bob McGrew, Scott Mayer McKinney, Christine McLeavey, Paul McMillan, Jake McNeil, David Medina, Aalok Mehta, Jacob Menick, Luke Metz, Andrey Mishchenko, Pamela Mishkin, Vinnie Monaco, Evan Morikawa, Daniel Mossing, Tong Mu, Mira Murati, Oleg Murk, David Mély, Ashvin Nair, Rejichiro Nakano, Ra- jeev Nayak, Arvind Neelakantan, Richard Ngo, Hyeonwoo Noh, Long Ouyang, Cullen O' Keeff, Jakub Pachocki, Alex Paino, Joe Palermo, Ashley Pantuliano, Giambattista Parascandolo, Joel Parish, Emy Parparita, Alex Passos, Mikhail Pavlov, Andrew Peng, Adam Perelman, Filipe de Avila Belbute Peres, Michael Petrov, Henrique Ponde de Oliveira Pinto, Michael, Pokorny,
14

---

Published as a conference paper at ICLR 2025
Michelle Pokrass, Vitchyr H. Pong, Tolly Powell, Alethea Power, Boris Power, Elizabeth Prochl, Raul Puri, Alec Radford, Jack Rae, Aditya Ramesh, Cameron Raymond, Francis Real, Kendra Rimbach, Carl Ross, Bob Rotsted, Henri Roussez, Nick Ryder, Mario Saltarelli, Ted Sanders, Shibani Santurkar, Girish Sastry. Heather Schmidt, David Schnurr, John Schulman, Daniel Sel- sam, Sheppard, Toki Sherbakov, Jessica Shieh, Sarah Shoker, Pranav Shyam, Szymon Sidor, Eric Sigler, Maddie Simens, Jordan Sitkin, Katarina Slama, Ian Sohl, Benjamin Sokolowsky, Yang Song, Natalie Staudacher, Felipe Petroski Such, Natalie Summers, Ilya Sutskever, Jie Tang, Nikolas Tezak, Madeleine B. Thompson, Phil Tillet, Amin Tootoonchian, Elizabeth Tseng, Pre- ston Tuggle, Nick Turley, Jerry Tworek, Juan Felipe Cerón Uribe, Andrea Vallone, Arun Vi- jayvergiya, Chelsea Voss, Carroll Wainwright, Justin Jay Wang, Alvin Wang, Ben Wang. Jonathan Ward, Jason Wei, CJ Weinmann, Akila Welihinda, Peter Welinder, Jiayi Weng, Lilian Weng, Matt Wiethoff, Dave Willner, Clemens Winter, Samuel Wolrich, Hannah Wong, Lauren Work- man, Sherwin Wu, Jeff Wu, Michael Wu, Kai Xiao, Tao Xu, Sarah Yoo, Kevin Yu, Qiming Yuan, Wojciech Zaremba, Rowan Zellers, Chong Zhang, Marvin Zhang, Shengjia Zhao, Tianhao Zheng, Juntang Zhuang, William Zhuk, and Barret Zoph. Gpt-4 technical report, 2024. URL https: arxiv.vi.................a.i..ve www.or/gbbs....
Albert Örwall. Moatless tools, jun 2024.
URL httpp://github.co hwtp::tttth.u..o.oooooooooooaeme www.w////a..waw/aoooooo
moatlees tools Accessed: 2024-07-16.
Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kel- ton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback, 2022. URL httttttst///aiiiie org/ abs/ 2203 02155
Lianggnigg Pan, Alon Albalak, Xinyi Wang, and William Yang Wang. Logic-Im: Empowering large language models with symbolic solvers for faithful logical reasoning, 2023. URL https:
// //arxii.crq/////aa.l..a.aa abbec///ba//aa /2005 12295
Maria Rigaki, Carlos Catania, and Sebastian Garcia. Hackphyr: local fine-tuned Ilm agent for network security environments, 2024.
224.: 11276
Swarnadeep Saha, Archiki Prasad, Justin Chih- Yao Chen, Peter Hase, Elias Stengel- and Mohit Bansal. System- 1.x: Learning to balance fast and slow planning with language models, hwtg/abs//4440
2024. URL htttps:/s//a
14414
David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy Lillicrap, Madeleine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. Mastering the game go with deep neural networks and tree search. Nature, 529(7587):884 489, 2016a. ISSN 1476-4687. doi: 10. 10.8//r// 16961. URL https: /oi. org
David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driess- che, Julian Schrittwieser, loannis Antonoglou, Vedavyas Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy P. Lilli- crap, Madelcine Leach, Koray Kavukcuoglu, Thore Graepel, and Demis Hassabis. Mastering the game of go with deep neural networks and tree search. Nat., 529(7587):484 489, 2016b. doi: 10. 10..1////TTUREIG00
URL /attps:///oi... wtt.:///.1..11///aaaaae http:///abb..ooo.....././aaaaaae
David Silver, Thomas Hubert, Julian Schritwieser,, Ioannis Antonoglou, Matthew Lai, Arthur Guez, Mare Lanctot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, and Demis Hassabis. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science, 36206411) 2018, doi: 10.1 aci4n0e.aar6404. URL https: /wwwwwww.. www.scceeecc...cc...o wttp://ppp/p///oooottt////oooatt http:///w/.eeeee..o/ooo///////// 122 ence. aar6404]
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute optimally can be more effective than scaling model parameters, 2024. URL https: /arxiv.orggg/////............
abs 2408. 03314
15

---

Published as a conference paper at ICLR 2025
Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models, 2023. URL wttptttttsss......ooo wttp://w...r/aa///2/////
16291
Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Exe- cutable code actions elicit better Ilm agents, 2024a. URLhtttps://axxiv wrw. abs//4bs22222 01030
Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng, Heng Ji, and Graham Neubig. Opendevin: An open platform for ai software developers generalist agents, 2024b. URL https:
Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yo- gatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682 2022.
Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. Agentless: Demystifying Ilm- based software engineering agents, 2024. URL /tEtps://arxive: org /2bb/2440.. 01489
An Yang, Baosong Yang. Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong. Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang. Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jin- gren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang. Keming Lu, Keqin Chen, Kexin Yang. Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wen- bin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Xuejing Liu, Yang Fan, Yang Yao, Yichang Zhang. Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, Zhifang Guo, and Zhihao Fan. Qwen2 technical report, 2024a. URL https://arxixs wttp:/wwwwgg/bb/////o/
John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. Swe-agent: Agent-computer interfaces enable automated software engineering, 2024b. URL aattps://ariiv.
abs 240./ 15793
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA,, USA, December 10. 16, 2023, 2023. URL, http: /tap:rs///ppppsppppeaaaaeee n 37
nips ps. cc wwwpapee.pppppppcppccp fi wefiless es http:/pppp///22333 2tt::7003--bbattarr
Conferenceeceeecccecooomeee oote:////oAAbtaaatttooooeeeeeee html
Kexun Zhang, Weiran Yao, Zuxin Liu, Yihao Feng, Zhiwei Liu, Rithesh Murthy, Tian Lan, Lei Li, Renze Lou, Jiacheng Xu, Bo Pang. Yingbo Zhou, Shelby Heinecke, Silvio Savarese, Huan Wang, and Caiming Xiong. Diversity empowers intelligence: Integrating expertise of software engineering agents, 2024a. URL http://arxiv... org/ abs/ 2ttp://aaa.....0//aa/2/288000
Yao Zhang, Zijian Ma, Yunpu Ma, Zhen Han, Yu Wu, and Volker Tresp. Webpilot: A versatile and autonomous multi-agent system for web task execution with strategic exploration, 2024h. URL
htttps::/arxxv..rrgg//oo
15978
Yiqun Zhang, Xiaocui Yang, Shi Feng, Daling Wang, Yifei Zhang, and Kaisong Song. Can Ilms beat humans in debating? a dynamic multi-agent framework for competitive debate, 2024c. URL
https://arxxvz
wttp:////w..a//////////// httpes:/aaxxv.oorg//b///222
04472
Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury, Autocoderover: Autonomous program improvement, 2024d. /tl/
http://////.../......... 05427
16

---

Published as a conference paper at ICLR 2025
A
REPRODUCIBILITY
All models and data used in our work are publiely available. We additionally provide hyperparameter details in Appendix 2
The code will be released public repository upon publication.
B ADDITIONAL IMPLEMENTATION DETAILS
Moadles--Aaptee is extended version of the moatless-tools library with support for tree struc- ture, the ability to revert to earlier versions of the codebase, and the capability to run tests.
The standard implementation of moatless-tools is based state holds information about file context and properties set in the configuration or from previous states, can then transition to a new state when action is executed. The request that initiates the action is created by an LLM. This follows linear structure where one state can transition to another state. In Moatless-Adapted this model is extended so that a state can expand by using actions to create more states. The connections between states are then represented in tree structure with nodes.
finite state machine structure where
Each state has file context associated with it. This file context will be included in the prompt sent to an L.M.. To limit the size of the prompt, files are divided into "spans," where a span could be, for example, section of code (c.g., imports), class, span IDs. Thus, the LLM sees a limited part the code at a time but can request more context by searching for adding files and spans. The file context therefore changes over time, and a specific state of file context is linked to a specific state. In the standard implementation of moatless-tools changes to the codebase are made linearly, and each change is saved directly the file system. In Moatless-AGee Adaated, however, there a need to be able to revert to earlier states and thus return to a previous version of the codebase. To handle this, the code is stored in git repository where each change is committed, and each state has a reference to commit as well as the current patch of the diff from the initial commit that existed before starting. This way, one can go back an earlier state by specifying the state ID) and the commit that was current at that time will be checked out.
function. These are identified by
The test files present in the file context are run each time the Plan state is initiated, and the test results are provided to the state. The tests are then run in Docker images built via the -WE-bench library. To use this approach in a benchmark where a larger number of instances should be able to run simultaneously, solution is used where these images are run pods in a Kubernetes cluster. Moatless-tools communicates with the testbed by applying patches and running commands via an API. When new instance starts, a pod is created which is then reset at each run, applying the current patch and runming tests according to the test command specified in the SWE-bench library. It's important add here that the agent is not aware of the PASS TO PASS FAIL.TO P PASS tests the SWE-bench harness, but only knows how to run the tests. This corresponds to a real engineering environment where each project can have its own test commands.
17

---

Published as a conference paper at ICLR 2025
C MCTS HYPERPARAMETERS
The Monte Carlo Tree Search (MCTS) algorithm used in this study employs several hyperparame- ters.
Table 2: MCTS Hyperparameters
Hyperparameter
Description
Default
c-param
UCT exploration parameter
1.41
max maxxexxansions
Max children per node
5
max _iterations
Max MCTS iterations
100
provide_ feedback
Enable feedback
Truc
best_first
Use best-first strategy
True
value function function,.emperature
Value function temperature
0.2
max depth
Max tree depth
20
UCT Score Calculation Parameters
exploration weight
UCT exploration weight
1.0
deith
Depth penalty weight
0.8
Bonth_bonus_facee
Depth bonus factor
200.0
high_value thheehod
High-value node threshold
55.0
low waa.vallueteesseeeee
Low-value node threshold
50.0
very high value threshold
Very high-value threshold
75.0
high value leaf bonus constant
High-value leaf bonus
20.0
high value bad children bonus constant
High-valus bad children bonus
20.0
High-valus child penalty
5.0
Action Model Parameters
action action_madel model Lemperature
Action model temperature
0.2
Discriminator Parameters
number agents
Number Discriminator Agents
5
number of round
Number of debate rounds
3
discriminator temperature
Discriminator temperature
1.0
These hyperparameters can be adjusted to fine-tune the MCTS algorithm's 's performance for specific problem domains or computational constraints. The values listed here are the defaults as defined in the TreeSearchSet.e ings class and the MCTS implementation.
18

---

Published as a conference paper at ICLR 2025
D
ABILITY OF MCTS TO ESCAPE UNPRODUCTIVE LOOPS VS. BASELINE
Moatless-adapted
SWE-search
Semtioode
RegukiCalicangg
Requeetioo
Reward
Requeelcerrhaagg
Meda7
Requeslo
[TERMINATED]
Figure 6: Avoiding Repetitive Actions, django__djjago, _etooaano____111441 We found that the base agent can often get stuck performing repetitive actions that do not bring it closer to solving the issue, and which commonly lead unresolvable dead-ends. In this example, the base agent was stuck implementing wrong tests which continuously returned errors. In contrast, when this happens in SWE-Search, the Value Agent recognizes this, terminating these trajectories quickly, as happens in Node 73 (orange).
19

---

Published as a conference paper at ICLR 2025
E MODEL INSTANCE RESOLUTION UNIQUENESS
To understand the complementary strengths of different models in resolving software issues, we analyzed how unique their resolved issue subsets where. Figure illustrates the resolution patterns for each model across five of the codebases in SWE-bench-lite.
django
matplotlib
pytest
sympy
scikit-learn
GPT-4o
GPT.4o-mini
Diven2.5-78-instruet
DeepSeck-V2.5
Llama-3.1-70--nntte
Figure 7: Unique Issue Resolution Patterns Across Models and Libraries. Each column repre- sents a different Python repository, and each row within a column represents a specific issue. Colored blocks indicate successful resolution by the corresponding model (see legend). White spaces denote unresolved issues. This visualization highlights the diverse problem- problem-solving capabilities of different models across various software domains, demonstrating that no single model dominates across all issues and libraries.
20

---

Published as a conference paper at ICLR 2025
F ABILITY OF VALUE FUNCTION TO DISCERN SUCCESSFUL TRAJECTORIES
Before implementing SWE-Search, we conducted a general study across many models to evaluate the models' ability differentiate states which led resolved vs. unresolved issues. Figure shows the results of this study. We found that in general, models assigned higher rewards to states which eventually led to resolved issues. Of particular interest was the Deepseek model, which seemed to identify critical errors in trajectories effectively. This was also observed in the final agent (see Fig. (h))
Average State Reward Comparison
Average State Reward
40-
20
Resolved
Unresolved
-20
Meta-l - Llama-3 3-88 innsrruct
Owen/Owen2: 728- Instruct
Owen/ 578 144 fB-Instruct
0p1-40
deepseek codee---ode----oe (atterr--3--mtt -nsiinttuuct
Average
Figure 8: Average State Reward Comparison Across Models. This graph compares the average state rewards assigned by different language models for resolved (green) and unresolved (red) issues. Error bars indicate standard deviation. Most models consistently assign higher rewards to states leading to resolved issues, with the exception of the. The 'Average" column represents the mean across all models, demonstrating a clear distinction between resolved and unresolved states.
21

---

Published as a conference paper at ICLR 2025
G
VALUE FUNCTION PROMPTS
Value Function Search Prompt
Your task is to evaluate a search action executed by Al agent, considering the search parameters, the resulting file context, and the identified code from the search results. Your evaluation will focus on whether the search action was well-constructed, whether the re- sulting file context is relevant and useful for solving the problem at hand, and whether identified code is appropriate and helpful.
You will be provided with four inputs: Problem Statement: This will be provided within the <prob com.statementt XML tag and contains the initial message or problem description the coding agent is trying to solve.
The Search Request:
This will be provided within the esearch_requeste XML tag and contains the search parameters used by the agent to define the search.
The Search Result: The content retrieved based on the search parameters provided within
<search.resultae XML tag.
Identified Code: The specific code identified from the search results, provided within the iidentified.cod: XML tag.
Search request parameters:
File Pattern (file_patte pattern): Glob patterns (e.g., **/*.py) to filter search results to specific files or directories.
Query (query): A natural language query for semantic search.
Code Snippet (code_snippet)) Specific code snippets for exact matching.
Class Names (class nasee):: Specific class names to include in the search.
Function Names (function names): Specific function names to include in the search.
Evaluation Criteria:
Search Parameters:
Are they appropriately defined to focus the search on relevant files or code?
Do they align well with the problem statement?
Resulting File Context:
Does it contain relevant and useful information for solving the problem?
Are there missing or irrelevant results indicating a need to refine the search?
Identified Code Review (most crucial):
the identified code directly related the problem?
Does provide the necessary functionality address the issue?
22

---

Published as a conference paper at ICLR 2025
Overall Relevance:
Does the combination of search parameters, file context, and identified code effectively address the problem?
Could there be better approach improvements?
Reward Scale and Guidelines:
Assign a single integer value between -00 and 100 based on how well the search action, resulting file context, and identified code addressed the task at hand. Use the following scale:
100:
Search Parameters: Precisely match the problem needs; no irrelevant or missing elements.
Identified Code: Completely and accurately solves the problem with no issucs.
75 to 99:
Search Parameters: Well-defined and mostly relevant; minor improvements possible.
Identified Code: Effectively addresses the problem with minor issues that easily fix- able.
0 to 74:
Search Parameters: Partially relevant; noticeable inaccuracies or omissions.
Identified Code: Partially solves the problem but has significant gaps or errors.
-1 to -49:
Search Parameters: Misaligned with the problem; poorly defined.
Identified Code: Fails to address the problem effectively; may cause confusion.
-50 to -100:
Search Parameters: Irrelevant or incorrect; hinders problem-solving.
Identified Code: Unrelated to the problem; provides no useful information.
Output Format:
Please ensure your output strictly adheres to the following structure: <Explanat ion> [A brief explanation of the evaluation in max one paragraph.]
<Reward> [A single integer reward value between -100 and 100]
23

---

Published as a conference paper at ICLR 2025
Value Function Plan Prompt
Your role to evaluate the executed action of the search tree that our Al agents are travers- ing, to help determine the best trajectory to solve programming issue. The agent is responsible for identifying and modifying the correct file(s) in response to the problem state- ment.
Input Data Format:
cerroblemmstti em at ement> XML tag and contains the initial message problem description the coding agent is trying to solve.
Problem Statement: This will be provided within the <prob
The relevant code context will provided within the <file_context> XML tag and pertains to the state the agent is operating
File Context:
The sequence of state transitions and actions taken prior to the current state will be contained within the <hi chistory> XML tag. This will include information on the parts of the codebase that were changed, the resulting diff, test results, and any reasoning or planned steps.
History:
The last executed action of the coding agent will be provided within the <execut www.tteedddiiioooo XML tag, this includes the proposed changes and the resulting diff of the change.
Executed Action:
Full Git Diff: The full Git diff up to the current state will be provided within the sfull.git.ciff XML tag. This shows all changes made from the initial state to the current and should be considered in your evaluation ensure the modifications align with the overall solution.
Test Results: The results of any test cases run on the modified code will be provided within the <test_ress resuttrrestt>>>>> XML tag. This will include information about passed, failed, or skipped tests, which should be carefully evaluated to confirm the correctness of the changes.
Evaluation Criteria:
Code Correctness:
Evaluate whether the implemented code correctly addresses the prob-
lem. This includes verifying that the correct lines or sections of code have been identified and modified appropriately. Ensure that the changes are both syntactically and logically cor- rect, and that the diff's accurately represent the intended modifications without introducing unrelated changes. Assess whether the modifications effectively solve the problem without introducing new issues or inefficiencies.
Mistakes in Editing Code: Identify any errors made during the code editing process. This
involves checking for unintended deletions, incorrect modifications, or syntax errors intro- duced through the changes. Ensure that the Git diffs maintain integrity by only including the intended modifications and no accidental alterations unrelated parts of the codebase.
Testing: Assess the proposed changes against existing test cases. Determine if the changes
pass all relevant tests and evaluate whether any test failures could have been reasonably foreseen and avoided by the agent. Consider whether the agent anticipated potential test outcomes and addressed them proactively in the solution.
24

---

Published as a conference paper at ICLR 2025
History and Action Evaluation: Review the agent's previous state transitions and actions
to determine if the current action contributes positively to solving the problem. Pay special attention to detect if the agent is engaging in repetitive actions without making meaningful progress. Evaluate whether the last executed action is appropriate and logical given the current progress and history of actions.
Reward Scale and Guidelines:
The reward value must be based on how confident you are that the agent's solution is the most optimal one possible with no unresolved issues or pending tasks. The scale ranges from -100 100, where:
100: You are fully confident that the proposed solution is the most optimal possible, has
been thoroughly tested, and requires no further changes.
75-99:
The approach is likely the best one possible, but there are minor issues or opportu-
nities for optimization. All major functionality is correct, but some small improvements or additional testing may be needed. There might be some edge cases that are not covered.
0-74:
The solution been partially implemented or is incomplete or there are likely alter-
native approaches that might be better, i.e., this is likely not the most optimal approach. The problem might be addressed, but there significant issues with tests, logical flow, or side effects that need attention. There are likely alternative approaches that are much better.
0: The solution is not yet functional or is missing key elements. The agent's assertion that
the task is finished is incorrect, and substantial work is still required to fully resolve the issue. Modifying the wrong code, unintentionally removing or altering existing code, introducing syntax errors, producing incorrect diffs fall into this range.
-1 to -49: The proposed solution introduces issues regresses existing functionality,
but some elements of the solution show potential or may be salvageable. Repetitive actions without progress fall into this range.
-50 to -100: The solution is entirely incorrect, causing significant new problems, or fails
to address the original issue entirely. Immediate and comprehensive changes necessary. Persistent repetitive actions without progress should be heavily penalized.
Output Format:
Please ensure your output strictly adheres to the following structure: <Explanat ion> [Your brief explanation of the evaluation max one paragraph.]
<Reward> [A single integer reward value between -100 and 100)
25

---

Published as a conference paper at ICLR 2025
Value Function Request More Context Prompt
Your role to evaluate the executed action of the search tree that our Al agents are travers- ing, specifically for the RequestMoreConte requests to see code that not in the current context, potentially revealing an understand- ing that relevant code is wholly partially not visible, and enabling the agent to uncover important missing information.
action. This action is used when the agent
Evaluation Criteria:
Relevance: Are the requested files and code spans likely to be relevant to the problem at hand?
Necessity:
Is the additional context truly needed, or is the agent unnecessarily expanding the scope?
Specificity: Has the agent been specific in its request, is it asking for overly broad sections of code?
Contextual Understanding: Does the request demonstrate good understanding of the codebase structure and the problem domain?
Efficiency: Is the agent making targeted requests, or is asking for too much unnecessary information?
Progress: Does this request seem likely to move the problem-solving process forward?
Input Data Format:
Problem Statement: Provided within the <problem.stateeeeet XML tag, contain- ing the initial problem description.
File Context: The current code context within the
XML tag.
History: Previous state transitions and actions within the <history> XML tag.
Executed Action:
The RequestMoreContex action details within the <executed.actions XML tag, including the files and code spans requested.
Reward Scale and Guidelines: Assign single integer value between -100 and 100 based
on how well the RequestMoreContex action addresses the task at hand:
100: Perfect request that is highly likely provide crucial missing information.
75-99;
Good request with minor improvements possible in specificity relevance.
0-74: Partially relevant request, but with noticeable inaccuracies or potential for better tar-
geting.
-1 to -49: Poor request that is likely to provide mostly irrelevant information or expand the
scope unnecessarily.
-50 to -100: Very poor request that is entirely irrelevant demonstrates fundamental
26

---

Published as a conference paper at ICLR 2025
misunderstanding of the problem codebase structure.
Output Format: Please ensure your output strictly adheres the following structure:
<Explanati minpanaatio>> [Your explanation of the evaluation in max two paragraphs.] <Reward> [A single integer reward value between -100 and 100]
Value Function Edit Prompt
Your role is to evaluate the executed action of the search tree that our Al agents are travers- ing, with the goal of ensuring that a complete and verified solution in place. The agent believes that it has finished solving the programming issue.
Evaluation Criteria
Solution Correctness and Quality: Verify that the proposed changes logically address the
problem statement. Ensure the changes fit contextually within the existing codebase without introducing new issues. Confirm syntactic correctness and that there are no syntax errors typos. Assess whether the solution represents overall improvement and is the most optimal approach possible.
Accuracy of Code Modifications: Check that the agent correctly identified the appropriate
code spans to modify. Ensure the changes made are accurate and do not include unintended modifications. Look for any alterations to unrelated parts of the code that could introduce new problems.
Testing and Test Results Analysis:
Importance of Test Updates: It is crucial that agent updated existing tests or added tests to verify the solution. Failure to do should be heavily penalized. The agent should ensure that code changes are validated by appropriate tests to confirm correctness and prevent regressions.
Assess Test Coverage: Evaluate whether the agent has adequately tested the solution, including adding new tests for new functionality or changes. Verify that the tests cover relevant cases and edge conditions.
Penalization for Lack of Testing: When calculating the reward, heavily penalize the agent if they failed update or add necessary tests to verify the solution.
Consideration of Alternative Approaches: Always assess whether there could be better
alternative approach to the problem. Mention any potential alternative solutions in your explanation if they are applicable.
Identification and Explanation of Mistakes: If the agent made incorrect actions, identify
exactly where and why the mistakes occurred. Explain the impact of any syntax errors, incorrect code modifications. or unintended changes.
27

---

Published as a conference paper at ICLR 2025
Assessment of Agent's Completion Assertion: Verify if the agent's assertion that the task
finished is accurate. Determine substantial work is still required fully resolve the issue and address this in your evaluation.
Input Data Format:
Problem Statement: This will be provided within the <probl marobblemsste we..stattttt.eeeee..........mmtt ement> XML tag and contains the initial message problem description the coding agent is trying to solve.
File Context:
The relevant code context will provided within the <file context> XMI. tag and pertains to the state the agent is operating on.
The sequence of state transitions and actions taken prior to the current state will be contained within the <hi <history> XML tag. This will include information on the parts of the codebase that were changed, the resulting diff, test results, and any reasoning planned steps.
History:
Reasoning for Completion: The reasoning provided by the agent for why the task is finished will be provided within the <reasoning for.complet. ion> XML tag. This includes the agent's s explanation of why no further changes or actions are necessary.
Full Git Diff: The full Git diff up to the current state will be provided within the walllggitdifff.. XML tag. This shows all changes made from the initial state to the current and should be considered in your evaluation ensure the modifications align with the overall solution.
Test Results: The results of any test cases run on the modified code will be provided within the <test resstreeslts>> XML tag. This will include information about passed, failed, or skipped tests, which should be carefully evaluated to confirm the correctness of the changes.
Reward Scale and Guidelines:
The reward value must be based on how confident you are that the agent's solution is the optimal one possible with no unresolved issues pending tasks. It is important that the agent updated or added new tests to verify the solution; failure to do so should be heavily penalized. The scale ranges from -100 to 100, where:
100: You are fully confident that the proposed solution is the most optimal possible, has
been thoroughly tested (including updated or new tests), and requires no further changes.
75-99:
The approach is likely the best one possible, but there are minor issues or opportu-
nities for optimization. All major functionality is correct, but some small improvements or additional testing may be needed. There might be some edge cases that are not covered.
0-74: The solution has been partially implemented or incomplete, or there are likely
alternative approaches that might be better. The core problem might be addressed, but there significant issues with tests (especially if the agent did not update or add new tests), logical flow, or side effects that need attention.
0: The solution is not yet functional or is missing key elements. The agent's assertion that
28

---

Published as a conference paper at ICLR 2025
the task is finished is incorrect, and substantial work is still required fully resolve the issue.
-1 to -49: The proposed solution introduces new issues or regresses existing functionality,
but some elements show potential may be salvageable. Modifying the wrong code, unin- tentionally removing or altering existing code, introducing syntax errors, producing incorrect diffs, failing to update add necessary tests fall into this range.
-50 to -100: The solution is entirely incorrect, causing significant problems failing
address the original issue entirely. Immediate and comprehensive changes are necessary. Persistent repetitive actions without progress, or failure to update or add tests when neces- sary, should be heavily penalized.
Output Format: Please ensure your output strictly adheres the following structure:
<Explanat ion> (Your explanation of the evaluation in max two paragraphs.] <Reward> [A single integer reward value between -100 and 100]
H MOATLESS TOOLS STATE RIGIDITY
The Moatless-tools version (v0.0.2) enforces rigid transition structure where agents must follow a specific sequence (search- + identify - plan - edit). The implementation of this state transition system can be found here:
H.1
STATE TRANSITION SYSTEM
The transition system is configured through a function that accepts three optional parameters:
www.tokkens.inneddt..poooom Controls the token limit edit operations
global.params: Defines parameters applicable across all states
state_params: Specifies state-specific parameters
H.2
STATE FLOW
The system defines directed graph of states with specific transition rules:
1. Search Phase (SearchCode):
Initial state for code operations
Can transition to IdentifyCode upon successful search
Can move directly to PlanToCode when complete
2. Identification Phase I'dentifyCode
Processes search results
Can return to SearchCode if needed
Progresses to DecideRelevance when finished
3. Decision Phase (DecideRelevance),
Evaluates identified information
Can trigger new searches
http:///aaaaammmmmmmsssssooooo s/bl /bl (obllb///eee /ec5555 a ec31 11331f9966 hor
1/mott less-too
29

---

Published as a conference paper at ICLR 2025
Transitions to planning when ready, excluding message field
This rigid structure ensures that tools are accessed in a predictable sequence, preventing conflicts while maintaining system integrity. Additional transitions defined in TEI.TTRANSIIOON com- plete state machine's 's behavior set.
I
CoST ANALYSIS
Table presents the API costs for Moatless-Adapeed and SWE-Search across different models. Search-based exploration of multiple solutions results higher computational costs.
Model
Moatless-Adapted SWE-Search
GPT-4o
$40.86
$576.00
GPT-4o-mini
$9.90
$52.34
Qwen-2.5-72b-Imstruut
$8.50
$42.50
DeepseekCoderV2..
$3.66
$18.37
Llama-3.1-70b-Inste
$9.00
$45.00
Estimated costs based on comparablc API pricing
Table 3: Cost comparison (USD) between Moatless-Adapted and SWE-Search
J COMPUTE-MATCHING ANALYSIS
Table 4 compares SWE-Search against compute-matched baselines. SWE-Search Pass@5 uses the 5 generated answers in run, while for Moatless-Aapted the 5 generated solutions across runs. We avoid doing the comparison on GPT-4o to avoid exorbitant API costs.
SWE-Search
Moatless-Aaapted
Model
Pass @
P
P
GPT-4o
31.0
34.0
GPT-4o-mini
17.0
22.3
17.0
Dwwe-22.572b-Innsrrue
24.7
25.7
22.3
DeepseekCCodev22.5
21.0
23.3
22.0
1llma-3..-700-Insse
17.7
22.3
21.7
Table Performance comparison (%) between SWE-Search and compute- compute-matched baselines
K LNTERACTIVE DEMO
To help visualize the search process and provide transparency into method, provide inter- active demo at watttps://// :ttps://st... rearreealll moa t ess. ai] The demo presents a tree visualization where each node represents state/action pair in the search process. Clicking on node reveals detailed information including:
Complete L.M interactions and tool calls
State-specific value function outputs and reasoning
Context information used for decision-makinn
File changes and test results where applicable
Test creation/execution and their outputs
This interface allows readers to explore how the search algorithm navigates through different states, makes decisions, and evaluates potential solutions. The visualization particularly highlights how state-specific value functions guide the exploration process and how the discriminator compares candidate solutions.
30

---

Published as a conference paper at ICLR 2025
L DISCRIMINATOR DEBATE PROMPTS
Solution Comparison Debate Round Prompt
Below are a series of suggested changes to address the iProblem Statementy. Your task is to carefully evaluate each change and decide which one is the most appropriate to address the issue. Input Format:
Problem Statement: Original issue to be solved
Solutions: Multiple candidate solutions, each with:
Unique solution ID
Git patch showing code changes
Task: Evaluate each proposed solution and identify which one most effectively addresses the
problem statement. Consider the completeness, correctess,, and efficiency of each approach. Output Format:
<Explanation> Comprehensive explanation and reasoning behind your decision <ID>: The ID of the change you believe is the appropriate
Solution Comparison Conclusion Prompt
Based on the Problem Statement and previous responses from the debate, determine the optimal solution. Input Format:
Initial Context: Original problem statement
Agent Answers: Evaluations and solution ID choices from the debate phase
Task: Synthesize the evaluations to identify the most appropriate solution. Report your
conclusion without refereneing individual debate participants. Output Format:
<Explanation>: Reasoning for the selected solution
<ID>: Final chosen solution ID
31


Reviews:
[{"review_id": "pnsR5fSXcG", "reviewer": "Reviewer_j8qB", "summary": "Large language models have been used to automate repository-level tasks in software engineering. Current models used rigid, hand-crafted rules for decision making in these tasks. However, these constraints can cause these software agents to be less adaptable to long-horizon tasks. This paper proposes SWE-search, a framework incorporating multiple agents and Monte Carlo tree search (MCTS) to perform these software tasks. This framework consists of a search agent to perform exploration over software actions, a value agent for self-learning feedback and a discriminator agent for deciding on actions. The paper also includes empirical comparisons of the proposed approach against currently used software agents.", "questions": "1. Line 181: Typo in \"including\"\n\n2. In Equation 1, why is value function not dependent on the context space?\n\n3. In Equation 2, in search and RL, a balance between emphasizing the importance of long-term and short-term rewards is made using a discount factor. The $early\\textunderscore depth\\textunderscore bonus$ and $late\\textunderscore depth\\textunderscore penalty$ appear to be an attempt to handcraft this balance. Why not just use discounting?\n\n4. Lines 411-415: I do not understand what that segment is trying to say.", "limitations": "", "rating": 3, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "**Originality:** The three types of agents being trained for the framework are based on pre-existing models. However, novelty arises in the way they are combined to form a automated SE framework.\n\n**Significance:** The use of LLMs for automating SE tasks is already pretty widespread. Significantly improving on the current SOTA would further increase their adoption into more use cases.\n\n**Clarity:** The idea presented was clearly laid and explained well.\n\n**Quality:** The problem is well-motivated and plainly laid out. The intuition behind the proposed framework is conceptually simple and the way the multiple agents in the framework are used is sound and reasonable. The experimental design was sound and the baselines against which their approach was compared were sensible. I appreciated the demonstration for the importance of state information in Sec 4.2.", "weaknesses": "There are however a few concerns that need to be addressed.\n\n1. There were a number of (I think) formatting issues. This resulted in some the equations not making sense.\n    - the arguments in the value function on line 193 do not make sense\n    - the expected cumulative reward on line 196 also does not make sense.\n    - $O$ is never defined\n\n2. Section 3 was very confusing:\n    - $t$ is used to refer several different things. In 3.1 it is a state and later it is used to define time step. In section 3.1.1, $t$ is used to refer to action type.\n    - In 3.1.1, actions, states and action-types seem to be used interchangeably. I can't tell if the action-types are the states or actions. Fig 1 suggests they are the states in the search tree so why not call them that in all instances?\n    - $T$ is not defined on line 190.\n\n3. In Sec 4, I'm not exactly sure what the moatless-adapted baselines entails. I read after under the impression that they are the models using the moatless tools with the strict search restrictions.\n\n4. There is no Table 3. There is, however, a Figure 3 that is a table that is never referenced but the values do not align with the descriptions of Table 3.\n\n5. MCTS is stochastic algorithm. As such the evaluations should be performed using more than one random seed on the evaluation set. This would provide the expected performance wrt solve rate, computation, etc. Most importantly, it would provide some measure of significance testing to see if the results support the claims made by authors about the performance gains of SWE.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sUDCRWFwVo", "reviewer": "Reviewer_TtHv", "summary": "The authors propose a new agentic system for software development applications based on frontier models and MCTS. They show that this leads to a performance improvement relative to a separate baseline model across several different LLM engines.", "questions": "Please respond to the above when possible as well.\n\n- What is the exact data flow here? There are a few modules, but how do they actually connect together? The text is quite sparse on that and seems to instead focus on describing only the virtues of their approach.\n- What's the change in cost here? How much more expensive is this approach compared to other ones? I would like to see something like Table 1 but with USD API costs.\n- Is that doing significantly better on SWE-Bench than the norm? A quick look says no (but they look like contemporary works, so I won't hold it against you, and it is optional to address this question). Would this naturally integrate with some of those approaches?\n- Line 479-481 seems highly significant at first glance. Is there a reason that isn't a bigger topic in the paper?\n- Can you give more details on the difference between your work and the Alibaba system? Is that contemporaneous work?", "limitations": "", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 3, "strengths": "- The topic is very relevant and timely.\n- An improvement on the problem tackled here has incredible potential.\n- MCTS is a good approach to take for this kind of problem, and finding a good way to integrate it has been thought about a lot, so this work is quite useful to the community.\n- This approach is modular enough that it could feasibly be integrated effectively with other approaches, leading to rich future work.", "weaknesses": "- I think the Brown M&M here is really the citation used for MCTS. MCTS was absolutely not proposed for the first time in Silver et al.'s seminal work (Silver even cites the original paper), and using that here hints that this work is very unpolished. But this citation is just one of the symptoms of a greater problem: this paper is poorly written to the point that it reads more like an advertisement than a technical, scientific paper (lines 346-348 are a good example, but the issues are throughout and I'm not confident this can be solved without a full rewrite). While clarity is often overlooked, I'm not sure the rest of the paper is of the exceptional standards needed to justify getting away with it.\n- There are serious typos in a few places (e.g., the paragraph at 92 or all the missing parentheses around citations). I think the work needs---at a bare minimum---a full run-through to correct all these.\n- The primary area listed here is wrong. The taxonomy here refers to autonomy and planning in the robotics domain. This work has nothing to do with that.\n- The paper only mentions its tested on SWE-Bench Lite later on. That should be clarified in the abstract or, at least, in the introduction. That seems like a big difference to me here.\n- There's only a single baseline here (Moatless). I would have expected, say, three different baselines. The multiple LLMs here serve the role of something similar to random seeds in demonstrating that the results aren't a statistical fluke.\n- The performance improvement with the baseline seems quite incremental despite all the additional complexity.\n- \"general value functions\" are a particular concept in RL research. The authors need to change that term out.\n- SWE-Bench really is not a comprehensive benchmark and should not be said to be one. Not that it's a bad benchmark here, but you should remove the statement saying it is.\n- It would be good to swap out the lines in Figure 4(b) and the bars in 5(a) with different styles to make the paper more friendly to both colorblind readers and black and white printouts.\n\nThe above is my reading of this work, and unfortunately, I'm not really sure how much ground the authors can make with me if they try to argue against it (do please still try, though, as there's no point not doing so). However, the reviewing process is noisy, and no matter how anyone feels about their evaluation, the data says that even the best reviewers can only provide truly medium-confidence reviews. Thus, I'm inclined to acquiesce and change my score if the other reviews disagree.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0OxM2wvJee", "reviewer": "Reviewer_rRDq", "summary": "The paper proposes the use of MCTS and LLM self-evaluation to improve the performance of software agents. The agent achieves a strong 23% relative improvement on top of moatless-tools across the SWE-bench lite benchmark with a variety of different foundation models, and shows good results for test-time compute scaling with more environment steps.", "questions": "- How specific to the moatless-tools agent is the author’s algorithm? While the results on SWE-Bench are impressive, since then other agents have achieved 43% performance on the lite benchmark. (I recognize these agents may have been submitted after the author's results, but some comment on generality is needed)\n- A natural baseline for scaling inference time compute would be pass@n and compute matching. Does the MCTS agent beat this simple baseline?", "limitations": "", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Very well-motivated and natural extension to coding agents\n- Clear theoretical presentation of the RL/MCTS setting used\n- Strong improvements across a wide variety of foundation models on the SWE Bench lite benchmark\n- Promising results for scaling inference time compute with more environment steps", "weaknesses": "- Line 14: Unclear that the best software agents have “rigid processes”. Indeed, the meatless-tools framework that the authors use has a flexible choice of tools at every single timestep already. I agree with the author's justification that some search is necessary, but this claim should be more accurate.\n- Completely unclear what value or reward function that value agent is optimizing in Section 3.1.2. I cannot find a definition for the value in the main paper. Appendix G is also unclear, hard to see why the value function would return values like 40, or 75 in the main paper in Figure 4. There needs to be far more analysis/ablations and justification of this design choice.\n- Table 1 has no error bars or confidence intervals, which makes it hard to judge statistical significance.\n- Experimental evaluation is limited to the lite dataset with a single agent (moatless-tools), it would strengthen the paper to show improvements on the full benchmark as well with other base SWE agents\n- Unclear what prompts are used for the discriminator agent at the end, in general, the paper does not have enough details for full reproducibility\n- Comparison to other SWE agents is missing.\n\nMinor: \n- The presentation is overly abstract and general until far later in the paper. It would be helpful for the authors to ground their initial theoretical presentation with concrete SWE examples from the start.\n- Inconsistent use of \\citep and \\cite in the paper, please move in-text cites to \\citep", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "n30AstXBnr", "reviewer": "Reviewer_H9kG", "summary": "This paper introduces SWE-search, a multi-agent framework that integrates Monte Carlo Tree Search (MCTS) with self-improvement mechanisms to enhance the performance of LLM-based software agents in dynamic repository-level tasks. The motivation is to  enable agents to iteratively refine their strategies using both numerical and qualitative feedback. By facilitating multi-agent debates, SWE-search optimizes the selection of solutions for complex problems through diverse perspectives. In experiments, the authors demonstrate the effeciveness of SWE-search by comparing it with a baseline called Moatless-adapted.", "questions": "Please respond to the questions in the Weakness part.", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The motivation of this paper is clear, in the sense that software development is a non-linear process by its nature. The authors elaborate how their method mimic human work patterns in real world, which makes sense to me. \n\n2. The idea of using MCTS to search for better solutions is novel.", "weaknesses": "1. The problem is not well-defined. Software development includes various kinds of tasks such as requirement analysis, architecture design, code generation from scratch, patching, debuging, testing, etc. The problem formulation part does not indicate which types of problems they are trying to sovle. In the experiment part, it seems that SWE-search aims to address issues in existing code bases. But it is still unclear how this problem is formulated as a sequential decision making problem. For example, in line 206, the authors describe the action space A_t as \"a set of actions corresponding to type t\". What EXACTLY are these actions? code generation? code repair? Overall, the problem formulation seems to be a general MDP-like formulation, with littel connections to software development tasks. \n\n2. It is unclear how the value agent evaluate actions. It makes sense to me that the feedback is divided into an utility value and a piece of explanation. However, I doubt whether a normal LLM could provide accruate values to the SWE agent. Recall that in reinforcement learning, a value function is crucial to policy learning, and learning the value function is a very hard task. In software development, the value function should provide comprehensive testing results for given code and requirement list. Given the high-level descriptions in this paper, I am not convinced that a normal LLM could provide accurate feedback. \n\n3. It is unclear how the MCTS runs. The authors only describe a way  to compute UCT in equation 3. However, many key information is missing, For example, what are the solutions we are searching for? What are the nodes (states) and edges (actions) in the context of software tasks? How to perform a node expansion? \n\n4. Lack of justification of the Discriminator agent. Intuitively, if the value agent is accurate and the search algorithm works well, we would not need a multi-agent debate process. \n\n5. Lack of baselines. Since the contributions of this paper mainly lie in the multi-agent framework, including the MCTS search, it is necessary to compare with existing Agent-level framework such as MetaGPT, ChatDev and GPT-Engineer. However, the authors only compare SWE-search with moatless-adapted.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)