Kimi-Dev: Agentless Training as Skill Prior for SWE-agents
Authors: Zonghan Yang
Shengjie Wang
Kelin Fu
Wenyang He
Weimin Xiong
Yibo Liu
Yibo Miao
Bofei Gao
Yejie Wang
YINGWEI MA
Yanhao Li
Yue Liu
Zhenxing Hu
kaitai zhang
Shuyi Wang
Huarong Chen
Flood Sung
Yang Liu
Yang Gao
Zhilin Yang
Tianyu Liu
Venue: ICLR 2026 Poster (2026)
Keywords: coder LLM
Agentless
SWE-Agent
Reinforcement Learning
Primary area: foundation or frontier models, including LLMs
TL;DR: We present Kimi-Dev that obtains 60.4% pass rate on SWE-bench Verified with Agentless training, and demonstrate it provides strong skill priors that enable efficient and effective SWE-Agent adaptation.

Abstract: Large Language Models (LLMs) are increasingly applied to software engineering (SWE), with SWE-bench as a key benchmark. Solutions are split into SWE-Agent frameworks with multi-turn interactions and workflow-based Agentless methods with single-turn verifiable steps. We argue these paradigms are not mutually exclusive: reasoning-intensive Agentless training induces skill priors, including localization, code edit, and self-reflection that enable efficient and effective SWE-Agent adaptation. In this work, we first curate the Agentless training recipe and present Kimi-Dev, an open-source SWE LLM achieving 60.4\% on SWE-bench Verified, the best among workflow approaches. With additional SFT adaptation on 5k publicly-available trajectories, Kimi-Dev powers SWE-Agents to 48.6\% pass@1, on par with that of Claude 3.5 Sonnet (241022 version). These results show that structured skill priors from Agentless training can bridge workflow and agentic frameworks for transferable coding agents.

Published as a conference paper at ICLR 2026
KIMI-DEV: AGENTLESS TRAINING AS SKILL PRIOR FOR SWE-AGENTS
Zonghan Yang#*i, SShenjie Wang**, Kelin Fu', Wenyang He Weimin Xiong" , Yibo Liu, Yibo Miao, Bofei Gao", Yejie Wang Yingwei Ma, Yanhao Li, Yue Liu Zhenxing Hu, Kaitai Zhang", Shuyi Wang, Huarong Chen, Flood Sung, Yang Liu# Yang Gao# Zhilin Yang, Tianyu Liu Moonshot AI, # *THU, PKKU, SUASS, BUPT, + NUS, Shanggai Qi Zhi Institute
Q
Git http::///gtthh..o.oo///ooooaaaaa SaggingFace: http:////wagggaaa.....ooo///////
72B
ABSTRACT
Large Language Models (LLMs) are increasingly applied to software engineering (SWE), with SWE-bench a key benchmark. Solutions split into SWE-Agent frameworks with multi-turn interactions and workflow-based Agentless methods with single-turn verifiable steps. We argue these paradigms not mutually ex- clusive: reasoning intensive Agentless training induces skill priors, including lo- calization, code edit, and self-reflection that enable efficient and effective SWE- Agent adaptation. In this work, we first curate the Agentless training recipe and present Kimi-Dev, open-source SWE LLM achieving 60.4% SWE-bench Verified, the best among workflow approaches. With additional SFT adaptation on 5k publicly-available trajectories, Kimi-Dev powers SWE-Agents to 48.6% pass@ 1, on par with that of Claude 3.5 Sonnet (241022 version). These results show that structured skill priors from Agentless training can bridge workflow and agentic frameworks for transferable coding agents.
innsoningiintnsive
INTRODUCTION
Recent days have witnessed the rapid development Large Language Models (LLMs) automating
Software-Enginee Engineering (SWE) tasks (Jimenez et al. 2023 Yang et al. 2024a] Xia et al. 2024 An-
thropic 2024 Pan et al.) 2024 Wang al. 2025a Wei et al. 2025 Yang al. 2025a Kimi al.
2025 OpenAI 2025c). Among the benchmarks that track the progress of LLM coding agents in SWE scenarios, SWE-bench (Jimenez et al. 2023) stands out as one of the most representative ones: Given issue that reports bug in real-world GitHub repository, model is required to produce a patch that fixes the bug, the correctness of which is further judged by whether the corresponding unit tests passed after its application. The difficulty of the task (as of the date the benchmark was proposed), the existence of the outcome reward with the provided auto-eval harness, as well as the real-world economic value it reflects, have made the SWE-bench focal point of the field.
Two lines of solutions have emerged for the SWE-bench task. Agent-based solutions like SWE- Agent (Yang et al., (2244) and OpenHands (Wang et al., (025a] take an interactionist approach: Instructed with the necessary task description, a predefined set of available tools, as well as the specific problem statement, the agent is required to interact with an executable environment for multiple turns, make change to the source codes, and determine when to stop autonomously. In contrast, workffow-based solutions like Agentless (Xia et al., 2024) pre-define the solving progress as a pipeline, which consists of steps like localization, bug repair, and test composition. Such task decomposition transforms the agentic task into generating correct responses for a chain of single-
turn problems with verifiable rewards (Guo et al. 2025 Wei et al. 2025 He et al. 2025).
The two paradigms have been widely viewed as mutually exclusive. On the one hand, SWE-Agents are born with higher potential and better adaptability, thanks to the higher degree of freedom of the
Indicates equal contribution. Joint leads.

---

Published as a conference paper at ICLR 2026
multi-turn interaction without the fixed routines. However, it has also proved more difficult to train with such frameworks due to their end-to- sod--oo-nd nature (Luo et al. 2025 Cao et al. 2025). On the other hand, Agentless methods offer better modularity and the case train with Reinforcement Learning with Verifiable Rewards (RLVR) techniques, but more limited exploration space and flexibility, and difficulty in behavior monitoring the erroneous patterns appear only in the single-turn long rea- soning contents (Pan et al., 2024). However, we challenge the dichotomy from the perspective of training recipe: We argue that Agentless training should not be viewed as the ultimate deliverable, but rather as a way to induce skill priors atomic capabilities such as the localization of buggy im- plementations and the update of erroneous code snippets, as well self-reflection and verification, all of which help scaffold the efficient adaptation of more capable and generalizable SWE--ggnts.
Guided by this perspective, we introduce Kimi-Dev, an open-source code L.LM for SWE tasks. Specifically, we first develop an Agentless training recipe, which includes mid-training, cold-start, reinforcement learning, and test-time self-play. This results in 60.4% accuracy on SWE-bench Ver- ified, the SoTA performance among the workflow-based solutions. Building on this, we show that Agentless training induces skill priors: a minimal SFT cold-start from Kimi-Dev with 5k publicly- available trajectories enables efficient SWE-agent adaptation and reaches 48.6% pass @ I score, sim- ilar to that of Claude 3.5 Sonnet (the 20241022 version, Anthropic (2024)). We demonstrate that these induced skills transfer from the non-agentic workflows the agentic frameworks, and the baked through Agentless training further enable the agen- tic model to leverage more turns and succeed with longer horizon. Finally, we also show that the skills from Agentless training generalize beyond SWE-bench Verified to broader benchmarks like SWE-bench-live (Zhang et 2025) and SWE-bench Multilingual (Yang et al. 2025c). Together, these results reframe the relationship between Agentless and agentic frameworks: not mutually ex- clusive, but as complementary stages in building transferable coding L.MSs. This shift offers a prin- cipled view that training with structural skill priors could scaffold autonomous agentic interaction.
self-reflection in long Chain-of-Thouglhe
The remainder of this paper is organized as follows. Section 2 reviews the background of the frame- work dichotomy and outlines the challenges of training SWF Section presents our Agent- less training recipe and the experimental results. Section demonstrates how these Agentless- induced skill priors enable efficient SWE-Agent adaptation, and evaluates the skill transfer and generalization beyond SWE-bench Verified.
4
2 BACKGROUND
In this section, we first review the two dominant frameworks for SWE tasks and their dichotomy in Section 1. We then summarize the progress and challenges of training SWE-Agents in Section 2.2 The background introduction sets the stage for reinterpreting Agentless training as skill priors for SWE-Agents, central theme developed throughout the later sections.
2.1
FRAMEWORK DICHOTOMY
Two paradigms currently dominate the solutions for automating software engineering tasks. Agent- less approaches decompose SWE tasks into modular workflows (Xia et al. 20224) Wei et al. 2025) Ma et al. 2025a bl Xie al. 2025). Typical workflows consist of bug localization, bug repair, and test generation. This design provides modularity and stability: each step could be optimized sepa- rately as a single-turn problem with verifiable rewards (Wei et al.) 2025 He et al., 2025). However, such rigidity comes at the cost of flexibility. When encountering scenarios requiring multiple rounds of incremental updates, the Agentless approaches struggle to adapt.
By contrast, SWE-agents adopt an end-to-end, multi-turn reasoning paradigm (Yang et al. 2024a Wang et al. 2025a). Rather than following a fixed workflow, they iteratively plan, act, and reflect, resembling how human developers debug complex issues. This design enables greater adaptability, but introduces significant difficulties: trajectories often extend over tens or even hundreds of steps, context windows of the LLMs must span over the entire interaction history, and the model must handle exploration, reasoning, and tool use simultaneously.
The dichotomy between fixed workflows (e.g., Agentless) and agentic frameworks (e.g., SWE- Agent) has shaped much of the community's 's perspective. The two paradigms are often regarded mutually exclusive: one trades off flexibility and performance ceiling for modularity and stabil-
2

---

Published as a conference paper at ICLR 2026
Generate Test Case
File Localization
Localization
BugFixer
TestWriter
LLM
Code Edit
Code Edit
Fix Bugs
Figure 1: Agentless framework for Kimi-Dev: the duo of Buglixer and TestWriter.
ity, whereas the other makes the reverse compromise. Our work challenges this dichotomy, as we demonstrate that Agentless training induces skill priors that make further SWE-aggnt training both more stable and more efficient.
2.2 TRAINING SWE-AGENTS
Training SWE-agents relies acquiring high-quality trajectories through interactions with exe- cutable environments. Constructing such large-scale environments and collecting reliable trajecto- ries, however, requires substantial human labor well as costly calls to frontier models, making data collection slow and resource- demuurdce- (Pan et al., [2024] Badertdinov et al.) 2024b). Recent studies also attempt scale environment construction by synthesizing bugs for the reverse construction of executable runtime (Jain et al., 2025; Yang et al. 2025c
However, credit assignment across long horizons still remains challenging, outcome rewards are sparse and often only available when final patch passes its tests. Reinforcement learning techniques have been proposed, but frequently suffer from instability collapse when trajectories exceed dozens of steps (Luo et al.) 2025 Cao et al., 2025). SWE-agent training is also highly sensitive initialization: starting from generic pre-traine trained model often leads brittle behaviors, such as failing to use tools effectively or getting stuck in infinite loops of specific action patterns (Pan
al. 2024 Yang et al. 2025c).
These limitations motivate our central hypothesis: instead of training SWE-agents entirely from scratch, one can first induce skill priors through agentless training. enhancing the atomic capabilities like localization, repair, test composition, and self-reflection. These priors lay foundation that makes subsequent agentic training both more efficient and more generalizable.
3 AGENTLESS TRAINING RECIPE
Instead of training SWE-agents from scratch, we leverage Agentless training to induce skill priors. Skill priors enhanced by Agentless training include but are not limited to bug localization, patch gen- eration, self-reflection and verification, which lay the foundation for end-to-end agentic interaction. In this section, we elaborate our Agentless training recipe: the duo framework design of Bug and TestWriter, mid-training and cold-start, reinforcement learning, and test-time self-play. Sec- tions 3.J 3.4 detail these ingredients, and Section 3.5 presents the experimental results for each of them. This training recipe results in Kimi-Dev, an open-souree 72B model that achieves 60.4% on SWE-bench Verified, the SoTA performance among the workflow-basee solutions.
3.1 FRAMEWORK: THE DUO OF BUGFIXER AND TESTWRITER
In GitHub issue resolution, we conceptualize the process as the collaboration between two important roles: the BugFixer, who produces patches that correctly address software bugs, and the TestWriter, who creates reproducible unit tests that capture the reported bug. A resolution is considered suc- cessful when the BugFixer's patch passes the tests provided for the issue, while high-quality test from the TestWriter should fail on the pre-fix version of the code and pass once the fix is applied.
Each role relies on two core skills: (i) file localization, the ability to identify the specific files relevant to the bug test, and (ii) code edit, the ability to implement the necessary modifications, For Bug- Fixer, effective code edits repair the defective program logic, whereas for TestWriter, they update precise unit test functions that reproduce the issue into the test files. As illustrated in Figure [[ these two skills constitute the fundamental abilities underlying GitHub issue resolution. Thus, we enhance these skills through the following training recipes, including mid-training, cold-start, and RI.
3

---

Published as a conference paper at ICLR 2026
3.2 MID-TRAINING & COLD START
To enhance the model's prior as both a BugFixer and a TestWriter, we perform mid-training with 150B tokens in high-quality and real-world data. With the Qwen 2.5-72B-BB (Qwen et al. 2024) model as starting point, we collect millions of GitHub issues and PR commits to form its mid-training dataset, which consists of (i) ~50 tokens in the form of Agentless derived from the natural diff patch, (ii) ~20 tokens of curated PR commit packs, and (iii) ~20B tokens of synthetic data with reasoning and agentic interaction patterns (upsampled by a factor of 4 during training). The data recipe is carefully constructed to enable the model to learn how human developers reason with GitHub issues, implement code fixes, and develop unit tests. We also performed strict data decontamination to exclude any repository from the SWE-bench Verified test set. Mid-training sufficiently enhances the knowledge in the model about practical bug fixes and unit tests, making it better starting point for later stages. The details of the recipe covered in Appendix
To activate the model's long Chain-of-Thought (CoT) capability, we also construct a cold-start dataset with reasoning trajectories based on the SWE-Gym (Pan et al., (2024) and SWE-bench-extra (Badertdinov et al. 2024a) datasets, generated by the DeepSeek RI model (Luo et al. (2025), the 20250120 version). In this setup, RI acts the roles of Bugfixer and Testwriter, producing out- puts such as file localization and code edits. Through supervised finetuning as cold start with this dataset, we enable the model to acquire essential reasoning skills, including problem analysis, method sketching, self-refinement, and exploration of alternative solutions.
3.3 REINFORCEMENT LEARNING
After mid-training and cold-start, the model demonstrates strong performance in localization. Therefore, reinforcement learning (RL) focuses solely on the code edit stage. We construct a training set specifically for this stage, where each prompt equipped with executable environ- ment. We further employ multiple localization rollouts from the initial model to generate varied file location predictions, which diversifies the prompts used in code-edit RL.
For the RL algorithm, we adopt the policy optimization method proposed by Kimi k1.5 TTam al. 2025), which has shown promising results on reasoning tasks both math and coding. Kimi kl.5 (Team et al. 2025) adopts a simpler policy gradient approach based on the REINFORCE al- gorithm (Williams] [1922.. Similarly GRPO (Shao et al. 2024). we use the average rewards of multiple rollouts as the baseline to normalize the returns. When adapting the algorithm in our SWE- bench setting, we highlight the following 3 key desiderata:
1. Outcome-based reward only: We rely solely on the final execution outcome from the environment as the raw reward (0 or 1), without incorporating any format- or process- based signals. For BugFixer, a positive reward is given if the generated patch passes all ground-truth unittests. For TestWriter, a positive reward is assigned when (i) the predicted test raises a failure in the repo without the ground-trutt -rrut bugfix patch applied, AND (ii) the failure is resolved once the ground-trute ttutt bugfix patch is applied.
2. Adaptive prompt selection: Prompts with pass 16 = 0 are initially discarded as they do not contribute to the batch loss. This results in an initial prompt set of 1,200 problems and enlarges the effective batch size. A curriculum learning scheme is then applied: once the success rate on the current set exceeds threshold, 500 new (previously excluded) prompts (with initial pass @ 16 = 0 but improved under RL) are reintroduced every 100 RL steps to gradually raise task difficulty.
3. Positive example reinforcement: As performance improvements begin to plateau in later stages of training, we incorporate the positive samples from the recent RL iterations into the training batch of the current iteration. This approach reinforces the model's reliance on successful patterns, thereby accelerating convergence in the final phase.
Robust sandbox infrastructure. We construct the docker environment with Kubernetes (Burns et al. 2016), which provides a secure and scalable sandbox infrastructure and efficient training and rollouts. The infra supports over 10,000 concurrent instances with robust performance, making ideal for competitive programming and software engineering tasks (see Appendix D for details).
4

---

Published as a conference paper at ICLR 2026
Table I: Performance comparison for models on SWE-bench Verified under Agentless- -ike frame- works. All the performances are obtained under the standard 40 patch, 40 test setting (Xia et al, except that Llama3-SWE-o uses 500 patches and 30 tests.
2024
Model
#arams
Resolve Rate (%)
Llama3-SWEL SWE-L Wei et al.
2025
70B
41.0
SehnknS- Seed et 2025
200B
47.0
OpenAT-ol (OpenAT]2024
48.9
DeepSeck-R1-0122) (Guo al.
2025
671B
49.2
OpenAl-o3-mini-hiee (OpenAl (025a))
49.3
Claude 3.5 Sonnet (241022) (Anihropié
2024
50.8
MiniMax-MI (Chen al. 2025a
456B
56.0
DeepSeek-RI-OSN (Guo 2025
671B
57.6
SWE-SWISS (He et al. 2025
32B
58.2
Kimi-Dev (Ours)
72B
60.4
3.4 TEST-TIME SELF-PLAY
After RL, the model masters the roles of both a BugFixer and a TestWriter. During test time, adopts self-play mechanism coordinate its bug-fixing and test-writing abilities.
Following Agentless (Xia et al. 2024), we leverage the model to generate 40 candidate patches and 40 tests for each instance. Each patch generation involves independent runs of the localization and code edit from BugFixer, where the first run uses greedy decoding (temperature O), and the remaining 39 use temperature from TestWriter. For the test patch candidates, to guarantee their validity, we first filter out those failing to raise failure the original repo without applying any BugFixer patch.
ensure diversity. Similarly, 40 tests are generated independently
Denote the rest TestWriter patches as set T, and the BugFixer patches as set B. For each b, E B and I, E T, we execute the test suite over the test file modified by l for twice: first without bis and then with & applied. From the execution log for the first run, we get the count of the failed and the passed tests from ły. denoted as F(j) and P(j). Comparing the exccution logs for the two test suite runs, get the count of the fail-to- poo-pass and the pass-to-pase to-pass tests, denoted Fi(i,j and (),,,,), respectively. We then calculate the score for each b; with
S, E3 FP(L,) E, PP(L.) Σ,
Σ,P()
where the first part reflects the performance of by under reproduction tests, and the second part could be viewed the characterization of b, under regression tests (Xia et al. 2024). We select the BugFixer patch b; with the highest S: score as the ultimate answer.
3.5
EXPERIMENTS
3.5.1
MAIN RESULTS
Table shows the performance of Kimi-Dev on SWE-
36.6
bench Verified (Jimenez et al. 2023). Instead of the text-
36
Pass Rate (%)
similarity rewards used in SWE-RL (Wei et al. 2025). we adopt exccution-based signals for more reliable fix qual-
32.6
ity. Our two-stage TestWriter also improves over prior Agentless systems (Xia et al. 2024, Guo et al.] 2025 He et al., (025), which rely on single root-level test, by better capturing repository context and mirroring real developer workflows (OpenAl 2025). Kimi-Dev attains state-of-the-arr performance among open-source models, resolving 60.4% of issues.
28.6
26
50D
100B
150B
Mid-training tokens
Figure 2: The performance on SWE- bench Verified after mid-training with different training token budgets.
3.5.2 MID-TRAINING
In this section, we evaluate the relationship between the amount of data used during mid- mid-training and model performance. Specifically, finetuned Qwen 2.5-72B- B-see with the subset of mid-training data of 50B, 100B, and approximately 150B tokens, and then lightly activated these mid-trained models using the same set of 2,000 Bugfixer
5

---

Published as a conference paper at ICLR 2026
Length
6500
7500
Pass Rale 1%)
6000
Reproduced Rate (%)
7000
Tokon Length
8000
Pass Rate (%)
Length
65000
Repi Reproduur wwroddoee Rate
5000
30.0
40
M
5500
5000
-2
4000
25.0
4300
3500
4000
3000
50
100
150
200
250
000
350
200
430 500
50
100
150
200
250
300
350
400 450
500
RL Training Steps
RL. Training Steps
72B Joint RI,, BugFixer
(b) 72B Joint RI., TestWriter
Figure 3: Joint code-edit RL experiments on the model after mid-training and cold-start, The pass rate for BugFixer and the reproduced rate for TestWriter are reported as pass@ I with tempera- ture=1.0. The performance improves consistently as the output becomes increasingly longer.
input-output pairs for SFT cold start. We only report BugFixer pass@1 here for simplicity of evaluation. Figure / shows that increasing the number of tokens in mid-training consistently improves model performance, highlighting the effectiveness of this stage.
3.5.3 REINFORCEMEN LEARNING
Experimental setup We set the training step per RL iteration as 5 and sample 10 rollouts for each of the 1,024 problems from the union of SWE-gym (Pan et al. 2024 and SWE-bench-exxta (Badertdinov et al. 2024b). We dynamically adjust the prompt set every 20 iterations to gradually increase task difficulty. We fix the maximum training context length as 64k tokens, since the prompt input contains the contents of the entire files localized by the initial model in advance.
Results sshww the performance and response length curves the test set during RL train- ing. The pass rate and the reproduced rate are calculated from pass and temperature=1. Specifi- cally, we observe that both model performance and response length steadily increase, reflecting the expected benefits of RL scaling. Similar RL scaling curves are also observed in our ablation ex- periments
models, proving the effectiveness of the RL training recipe across models of different sizes. The experimental details, as well as the ablation studies on positive example reinforcement in Section 3.3] are listed in Appendix C.2p. The lengthy outputs consist of in-depth problem analysis and self-reflectin patterns, similar to those in the math and code reason- ing tasks (Team al. [2255] Guo al. 2025). We have also observed that for TestWriter, occasional false-pose positive examples take place during RI. training due to the lack of reproduction coverage. We leave the case studies in Appendix and further improvement for future work.
Owen2.5-144B-Inttre
3.5.4 TEST- SELF-PLAY
No
Selinday
evaluate how the final performance on SWE- bench Verified scales with the num- ber of patches and tests generated. The temperature is fixed at for the initial rollout, and set to 1.0 for the subsequent 39 rollouts. As shown on the left of Figure 4 the final per- formance improves from 48.0% to 60.4% as the number of patch-test pairs increases from 1 1x1 to 40, and consistently surpasses the results obtained from the majority vote the BugFixer patches only.
Following Section 3.4
Higorty Vetag
Pors Rate (R)
Rate
47.5
40.0
363
205
Histe 20.00
40-00
5x6 10x10 20x20 (DweD Number of patches: BE TW
363
Number of patches: BF TW
Figure 4: Test-time self-play SWE-bench Verified. Per- formance improves with more generated patches and tests. Left: Execution-based self-play consistently surpasses Bug- Fixer majority voting. Right: Self-play performances re- main below @ where the ground-truth test patch used, suggesting the room exists for TestWriter to improve.

---

Published as a conference paper at ICLR 2026
Table 2: Single-attempt performance different models on SWE-bench Verified under end-to-end agentic frameworks, categorized by proprietary or open-weight models, and size over under 100B (as of 2025.09). "Internal" denotes results achieved with their in-house agentic frameworks.
Model
System
#Params
Pass Rate (%)
Proprietary
Gemini 2.5 Pro (Comanici et al.
2025
Internal
60.3
OpenAl-o3 (OpenAl) (0255)
Internal
69.1
GPT-5 (OpenAl 2025c)
Internal
74.9
Claude 3.5 Sonnet (24T022) (Anthropic
2024
SWE-Agent
49.0
Claude 3.7 Sonnet (Anthropic
2025a
SWE-Agent
62.3
Claude 4.0 Sonnet
(Anthropic
2025b
SWE-Agent
72.7
Open Weight,
2 100B
gpt-oss- (High) (OpenAl
2025b)
Internal
120B
62.4
DeepSeek-v3.1 (Guo et al. 2025)
Internal
671B
66.0
Kimi-K2-0905 Kumi et al.
2025
SWE-Agent
IT
69.2
Qwen3-Coder (Yng al.
2025a
OpenHands
480B
69.6
Open Weight
100B
Openhands-LM Wang
2025b)
OpenHands
32B
37.2
Skywork-SWE Zeng et al.
2025)
OpenHands
32B
38.0
SWF-agent-LM Yang et al.
2025b)
SWE-Agent
32B
40.2
DeepSWF: (Luo et 2025
2025)
OpenHands
32B
42.2
(AT AI 2025
OpenHands
24B
53.6
gpt-oss-20b (High) (OpenAl 2025b
Internal
20B
60.7
Kimi-Dev (SFTed)
SWE-Agent 72B
48.6
Specifically, the self-play result obtained from 3 patches and 3 tests for each instance has already surpassed the performance with majority voting from 40 BugFixer patches. This demonstrates the effectiveness of additional information from test-time execution. The room for improvement of Test- Writer, though, still exists for more powerful self-play: Shown on Figure 4 self-play performances remain below pass@M N, where ground-truth test cases serve as the criterion for issue resolution. This finding aligns with Anthropic (2024), which introduced final edge-case checking phase to gen- crate more diverse set test cases, thereby strengthening the role the "TestWriter" in their SWE-Agent framework. We also report preliminary observations of potential parallel scaling phenomenon, which requires no additional training and may enable scalable performance improve- ments. The details of the phenomenon and analyses covered in Appendix
4 INITIALIZING SWE-AGENTS FROM AGENTLESS TRAINING
End-to-end multi-turn frameworks, such as SWE-Agent (Yang et al., 2024a Anthropic 2024 and OpenHands (Wang al. 2025a), enable agents leverage tools and interact with environments. Specifically, the system prompt employed in the SWE-Agent framework (Anthropic 2024) outlines five-stage workflow: (i) repo exploration, (ii) eтror reproduction via test script, (iii) code edit for bug repair, (iv) test re-execution for validation, and (v) edge-case generation and checks. Unlike Agentless, the SWE-Agent framework doesn't enforce striet stage-wise workflow; the agent reflect, transition, and redo freely until it deems the task complete and submits.
The performance potential therefore higher without a fixed routine; However, the training for SWE-Agent is more challenging because of the sparsity of the outcome reward for long-horizon credit assignment. Meanwhile, our Kimi-Dev model has undergone Agentless training, with its skills of localization and code edit for BugFixer and TestWriter strengthened elaborately. In this section, we investigate whether can serve an effective prior for multi-turn SWE-Agent scenarios.
4.1 PERFORMANCE AFTER SWE-AGENT FINE-TUNING
We use the publicly available SWE-Agent trajectories to finetune Kimi-Dev. The finetuning dataset used is released by SWE-smith Yang al. 2025b), consisting of 5,016 SWE--ggen trajectories

---

Published as a conference paper at ICLR 2026
60
Paseg1
Passg22
RL Passg3
SFT Pass@2
Pass Rate (%)
SFT Pass@i3
Poss@
MT Paseg22
30
Passgi
Base Pass0]
Base Pess@2
20
Pasag:3
10
2
223
22
11x2" 2 1
1 15x22
SWE-Agent tokens
Figure 5: Comparing the quality of the Base, the Agentless mid-trained (MT), the Agentless mid-trained with reasoning-intensiv cold-start (SFT), and the Kimi-Dev model after RL as the prior for SWE-Agent adaptation. The tokens of the SWE-Agent SFT trajectories are swept over different scales, and the SWE-Agent performances are reported up to pass 3 on SWE-bench Verified.
collected with Claude 3.7 Sonnet (Anthropic 2025a) in the synthetic environments. We perform supervised fine-tuning over Kimi-Dev, setting the maximum context length as 64K tokens during training, and allowing up 128K tokens and 100 turns during inference.
As shown in Table 2 without collecting more trajectory data over realistic environments, or con- ducting additional multi-turn agentic RL, our finetuned model achieves pass score of 48.6% on SWE-bench Verified under the agentic framework setup, without additional test-time scaling. Using the same SFT data, our finetuned Kimi-Dev model outperforms the SWE-agent-LM (Yang et al. (025cc), with the performance comparable to that of Claude 3.5 Sonnet (49% by the 241022 version). The pass @ of our SWE-Agent adapted model 74.0% and surpasses the pass 30 of our model under Agentless (73.8%), proving the higher potential for the SWE-Agent framework.
4.2 SKILL TRANSFER AND GENERALIZATION
The results shown in Section demonstrate that Kimi-Dev, a model with extensive Agentless training. could be adapted to end-to-ena SWE-Agents with lightweight supervised finetuning. As the Agentless training recipe consists of mid- mid-training, cold-start (SFT) and RL, we explore the con- tribution of each part the recipe to the SWE-Agent capability after adaptation.
To figure this out, we perform SWE-Agent SFT on the original Qwen2.5-72BB (Base). the mid- trained model (MT), the model then activated with Agentless-formatte long CoT data (SFT), and the (Kimi-Dev) model after finishing RL training (RL). As are treating the four models as the prior for SWE-Agents" and a good prior always demonstrates the ability of fast adaptation with few shots (Finn et al., 2017, 2017.Brown Brown et al. 2020), we also sweep the amount of SWE-Agent SFT data to measure the efficiency of each prior in SWE-Agent adaptation.
Specifically, randomly shuffle the 5,016 SWE-Agent trajectories and construct nested subsets of sizes 100, 200, 500, 1,000, and 2,000, where each smaller subset is contained within the larger prepend two extreme baselines: (i) zero-shot, where the prior model is directly
ones. In addition, evaluated under the SWE-Agent framework without finetuning, and (ii) one-step gradient descent, where the model is updated with single gradient step using the 100-trajectory subset. This yields range of SFT token budgets spanning {0, 221 223 224, 1.1 x 225 1.1 x 226 1.1 x 227 1.5 x 228 After these lightweight SFT experiments, evaluate performance terms of pass @ (1,2,3) under the SWE-Agent framework, with evaluations for pass @ conducted at temperature 0, and those for pass @2 and pass@) @3 at temperature 1.0.
Figure 5 presents the SWE-Agent performances of each prior (Base, MT, SFT, RL) after being fine-tuned with different amounts of agentic trajectories. We have the following observations:
We slightly abuse the term "prior" to refer to model to be finetuned with SWE-Agent trajectories in the following analysis.
8

---

Published as a conference paper at ICLR 2026
Number of instances resolved (per turns)
160
Number of Resalved Cases
Bagfixer cutoft
#instances resolved
RL.
Reflection
193(+109)
710(+113)
SPT
700
642(+100)
120
MT
570(+94)
Base
500
80
100
40-
300
200-
484
542
605
20
40
90
Bsso
#Turns
Models
Figure 6: Left: Performance of the four priors under turn limits after SWE-Agent adaptation. Right: The characterization of the BugFixer and the reflection skills for each prior by counting the resolved cases of the 3 runs at Stage-3 cutoff moment, and comparing those with the final success cases.
1. The RL prior outperforms all the other models in nearly all the SWE-Agent SFT settings. This demonstrates that the Agentless training recipe indeed strengthens the prior in terms of SWE-Agent adaptation. For example, To achieve the top pass@1 performance of the Base prior, the RL prior needs only 223 SWE-Agent SFT tokens, whereas the Base prior consumes 1.5 x 2 tokens.
2. The MT prior is lagged behind the SFT and the RL ones in extremely data-scarce settings (zero-shot (0) and one-step gradient descent (221)), but quickly becomes on par with them after 200 trajectories (224) are available for finetuning. This indicates that adaptation effi- ciency remains comparable after the prior is strengthened through Agentless mid-training.
3. The performance of the SFT prior is mostly similar to the RL except for two cases: (i) The SFT prior outperforms the RL one under the zero-shot setting. This is reasonable, as the RL prior might overfit to the Agentless input-output format, while the SFT prior suffers less from this, (ii) The SFT prior exhibits a significant degradation with 200 SWE-Agent trajectories (224). A potential reason could be that the 200 trajectories collapse onto single data mode, leading the SFT prior to overfit through memorization (Chu et al. 2025); the RL prior instead embeds stronger transferable skills and thus generalizes better.
From long CoT extended multi-turn interactions. We hypothesize that reflective behav- iors cultivated through long chain-of-thour reasoning may transfer to settings requiring extended multi-turn interactions. To examine this, we evaluate the four priors (Base, MT, SFT, and RI.) by finetuning on the 5,016 trajectories and test on SWE-bench Verified, under varying turn limits with pass@3 as the metric (Figure 6 left). The distinct interaction-lengtth profiles show supportive evi- dence: the RL prior, after finetuning, continues to make progress beyond 70 turns, while the SFT, mid-trained, and raw models show diminishing returns around 70, 60, and 50 turns, respectively.
We further evaluate the efficacy of the Agentless skill priors (BugFixer and reflection) in the SWE- Agent adapted model. For BugFixer, given that the SWE-Agent may autonomously reflect between the five stages, we examine the moment in each trajectory when the bug fix of the third stage is initially completed, and the test rerun of the fourth stage has not yet been entered. Heuristically, when the SWE-Agent just completes the third stage, it has not yet obtained the execution feedback from the fourth stage, and thus has not further reflected based on the execution information or refined the bug fix. We therefore calculate the success rate of direct submission at this cutoff moment, which reflects the capability of the BugFixer skill. Regarding reflection, we further compare the performance at the cutoff point with the performance after full completion for each problem. The increment in the number of successful problems is used to reflect the capability of the reflection skill.
2025) to annotate the SWE-Agent trajectories, iden- We use kimi-k2-071 102-0711-preeve 1-preview (Kimi al.
tifying the stage to which each turn belongs. Figure 6 (right) demonstrates that both skills are strengthened through each stage of the Agentless training recipe: For the BugFixer skill, the cutoff performance at Stage-3 within the SWE-Agent interaction trajectories of the four adapted models shows consistent improvement, ranging from 484 cases resolved by the Base prior to 605 cases by the RL prior, measured by the number of successful resolutions within three passes. For the reflection skill, examining the performance gains from Stage-3 to the end the trajectories re- veals similar trend, with improvements increasing from +94 under the Base prior to +1 113 under

---

Published as a conference paper at ICLR 2026
60
40.0
54-
50
37.5
Poss Rate (%)
56
33.0
Poss Rate
50
54
32.5
00
SPT prior, Augss
52
27.5
RT. price. Angis
SET prior, Mingg
SET prior Pans@:
50
SET prior. Passa55
25.0
prioe. Minde-MModdd
92
RL print, 3
48
RL prior, Passgr
50
100
150
200
250
300
350
100
150
200
250
300
100
150
200
250
300
Steps
Steps
Steps
Figure 7: Comparison between the SFT Prior and the RL Prior when further applied with end-to- end SWE-Agent RL. Left: Pass@1 averaged from 5 runs. Middle: @ass@3. Right: Pass@5. The two priors are activated with the same 2 SWE-Agent SFT tokens (the second column in Figure After end RL, the RL prior slightly outperforms the SFT prior in all the Pass@1 1, Pass and Pass settings, which agrees with their SWE-Agent SFT performance comparison in Figure
the RL prior. Taken together, the adapted model from the RL prior achieves the strongest overall performance across both skills. It should be noted that our analysis of the reflection skill remains coarse-graiin searse-gaiined, since the measured performance gains between the two checkpoints capture not only agentic reflection and redo behaviors, but also the intermediate test-writing process performed by the SWE-Agent, A more fine-grained evaluation that isolates the TestWriter skill prior is left for future work. The prompt for SWE-Agent stage annotation, extended qualitative studies, as well as additional discussions for skill transfer and generalization, are covered Appendix G
End-to-end SWE-Agent RL for prior comparison. To further validate the effectiveness of the priors baked by the Agentless training recipes, we employ end-to-end SWE-Agent RI. (Luo et al.] 2025) with the cold-started priors as the initial models. To maximally alleviate the effect from the patterns of proprictary models within the SWE-Smith trajectories, we leverage the setting with 2a1 SWE-Agent SFT tokens, the second column in Figure where single step of gradient decent takes place on top of each prior. Under the minimal cold-start setup, end-to-end RL reveals the potential of each prior beyond taking the shorteut of imitation (Gudibande et al. 2024 Chu et al. 2025).
To run the end-to-end RL training for prior comparison, we use the SWE-Gym (Pan et al. 2024) and the SWE-bench-extra (Badertdinov et al. 2024a) subsets as the training set. Similarly to the Agentless RL recipe, we first use each initial model to filter out the problems with Pass @ass@8 = O. For the model with the MT prior, 260 out of 6,202 problems remain; for the models with the SFT prior and the RL prior, a total of 2,062 from the 6,202 problems are kept. In all end-to-end RL runs, use the outcome reward only, and the same policy gradient algorithm in Sec. 3.3 without KL or entropy regularization for optimization, with batch size as 256. The results are shown as follows:
For the model with MT prior, the pass@I performance quickly deteriorates to less than 2% after 10 end-to-end RL steps. The potential reason for this could be the lack of available problems to be trained with, reflecting the inferiority of the prior. For the models with the SFT prior and the RL prior, the RL runs last for 300 steps, and we plot the performance comparison in Figure According to Figure 7 the model with the RL prior demonstrates slightly higher scores of Pass @ 1, Pass@3, and Pass @ over the model with the SFT prior. While the phenomenon agrees with the performance comparison under SWE-Agent SFT shown in Figure we observe that the patterns in the interaction trajectories of the models incentivized by end-to-end SWE-Agent RL significantly differ from the patterns of the proprictary models (detailed in Appendix(G.3) These results reveal that the Agentless training recipe curates strong priors for end-to-end learning under SWE-Agent frameworks with the minimal supervision of proprietary end-to-end trajectories. We leave the exploration of more advanced agentic RL techniques for further improvement as future work.
CONCLUSION AND FUTURE WORK
In this work, we reframed Agentless and agentic paradigms for automated software engineering as complementary rather than competing. By introducing Kimi-Dev, we demonstrated that structured Agentless training can induce transferable skill priors, including bug localization, code repair, and relf-reflectio. As a result, Kimi-Dev not only achieves SoTA results on SWE-bench Verified among the workflow-bast approaches, but enables efficient SWE-Agent adaptation as well. These findings establish novel path toward building more generalizable coding agents through staged training.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENTS
This research was conducted with the support of Fundamental and Interdisciplinary Disciplines Breakthrough Plan of the Ministry of Education of China (No. JYB2025XDXMO01, the Shang- hai Qi Zhi Institute & Spirit AI Innovation Program, and the Tsinghua University Dushi Program. Funding and support for this work were also provided by the Tsinghua University Keystone Blec- trical (Zhejiang) Co.,Ltd Joint Research Center for Embodied Multimodal Artificial Intelligence (ICEMAI). Additionally, we would like to extend our thanks to the Xiongan AI Institute.
We thank Yuzhi Wang, Xinyu Zhou, Guokun Lai, Yulun Du, Fang Li, Hao Ding, Dehao Zhang, Enming Yuan, Dikang Du, and Jiacheng You for their valuable suggestions. We also appreciate the members of the infrastructure team at Moonshot Al for their timely support during the project.
ETHICS AND REPRODUCIBILITY STATEMENTS
This work obeys the Code of Ethics required by the ICLR conference. The study does not involve human subjects or animal experimentation. The personally identifiable information from raw data is excluded for privacy consideration (see the mid-training data recipe detailed in Appendix (A). Beyond the scope of this work, we strongly advocate for the community to advance systematic research on agent safety, thereby ensuring responsible progress in this area.
For all of the experiments, we have covered the detailed setups and discussions in the appendices: mid-training for Agentless in Appendix A details of the used dockers in Appendix B Agentless RL in Appendix [C, agent infrastructure Appendix [ case studies under Agentless in Appendix preliminary findings about emergent test-time parallel scaling in Appendix F and extended analysis for SWE-Agents in Appendix
REFERENCES
Mistral AI and All Hands AL. covstral-small2255 https://mistaal.ai
devstral-255 2507 July 2025.
Anthropic. Raising the bar swe-bench verified with claude 3.5 somnet. Online; Al model, Oct
2024. Uttptt :ecttss//Ae //www. anthropic. com/ /ttp:///ennnneeeeinee/////eeeeee sonnet
Anthropic. Claude 3.7 sonnet: Hybrid reasoning model. ht http::/ww..athrrppi....
Bewws/clauee-3---oonee February 2025a.
Anthoopic. Claude sonnet htttps//www..antrrpp....c.///e
May
2025b.
Ibragim Badertdinov, Maria Trofimova, Yuri Anapolskiy, Sergey Abramov, Karina Zainullina, Alexander Golubev, Sergey Polezhaev, Daria Litvintseva, Simon Karasik, Filipp Fisin, et al. Scal- ing data collection for training software engineering agents. Nebius blog, 2024a.
Ibragim Badertdinov, Maria Trofimova, Yury Anapolskiy, Sergey Abramov, Karina Zainullina, Alexander Golubev, Sergey Polezhaev, Daria Litvintseva, Simon Karasik, Filipp Fisin, Sergey Skvortsov, Maxim Nekrashevich, Anton Shevtsov, and Boris Yangel. Scaling data collection for training software engineering agents. Nebius blog, 2024b.
Ibragim Badertdinov, Alexander Golubev, Maksim Nekrashevich, Anton Shevtsov, Simon Karasik, Andrei Andriushchenko, Maria Trofimova, Daria Litvintseva, and Boris Yangel. Swe-rebench: An automated pipeline for task collection and decontaminated evaluation of software engineering agents, 2025. URL
/505. 20411
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert- Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models few-shot learners, Advances in Neural
11

---

Published as a conference paper at ICLR 2026
Information Processing Systems, 33:1877-1901, 2020. URL
14165
Brendan Burns, Brian Grant, David Oppenheimer, Eric Brewer, and John Wilkes. Borg, Omega, and Kubernetes. In Proceedings of the 10th USENIX Symposium Operating Systems Design and Implementation (OSDI 16), pp. 285-301, 2016.
Shiyi Cao, Sumanth Hegde, Dacheng Li, Tyler Griggs, Shu Liu, Eric Tang, Jiayi Pan, Xingyao Wang, Akshay Malik, Graham Neubig, Kourosh Hakhamaneshi, Richard Liaw, Philipp Moritz, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. Skyrl-vO: Train real-world long-horizon agents via reinforcement learning, 2025.
Aili Chen, Aonian Li, Bangwei Gong, Binyang Jiang, Bo Fei, Bo Yang, Boji Shan, Changging Yu, Chao Wang, Cheng Zhu, et al. Minimax-ml: Scaling test-time compute efficiently with lightning attention. arXiv preprint arXiv:2506. 13585, 2025a.
Mouxiang Chen, Binyuan Hui, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Jianling Sun, Junyang Lin, and Zhongxin Liu. Parallel scaling law for language models. arXiv preprint arXiv:2505. 10475, 2025b.
Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V Le, Sergey Levine, and Yi Ma. SFT memorizes, RL generalizes: A comparative study of founda- tion model post-training. In Forty-second International Conference on Machine Learning. 2025.
URL
net forum? id- 3u 3yabMi
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arXiv preprint arXiv:2507.0626.. 2025.
Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta- peta-learnngg for fast adaptation of deep networks. In International conference on machine learning, pp. 1126-11 1135. PMLR, 2017.
Aaron Grattafiori, Abhimanyu Dubey, Abhinay Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, al. The llama 3 herd of models. arXiv preprint 2rXiv:240..21773, 2024.
Xinran Gu, Kaifeng Lyu, Jiazheng Li, and Jingzhao Zhang. Data mixing can induce phase transitions in knowledge acquisition. arXiv preprint arXiv:2505. 18091, 2025.
Arnav Gudibande, Eric Wallace, Charlie Victor Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song. The False Promise of Imitating Proprietary Language Mod- els. The Twelfth International Conference on Learning Representations, 2024. URL https:
/ //openeeview.. net foet/fozze wtr/m//rummmmmmm wtwprummiidi2zeeeeeee K23cckCNN
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability Ilms via reinforcement learning. arXiv preprint arXiv:2501.12945 2025.
Zhenyu He, Qingping Yang, Wei Sheng, Xiaojian Zhong, Kechi Zhang, Chenxin An, Wenlei Shi, Tianle Cai, Di He, Jiaze Chen, Jingjing Xu, and Mingxuan Wang. Swe-swiss: A multi-task fine-tuning and rl recipe for high-performance issue resolution. Notion page 2025.
Naman Jain, Jaskirat Singh, Manish Shetty, Liang Zheng, Koushik Sen, and Ion Stoica. R2e-gym: Procedural environments and hybrid verifiers for scaling open-weights swe agents. arXiv preprint arXiv:2504.00 07164, 2025.
Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.067700 2023.
Team Kimi, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, et al. Kimi k2: Open agentic intelligence. arXiv preprint arXiv:2507.205:.. 2025.
12

---

Published as a conference paper at ICLR 2026
Michael Luo, Naman Jain, Jaskirat Singh, Sijun Tan, Ameen Patel, Qingyang Wu, Alpay Ariyak, Colin Cai, Shang Zhu Tarun Venkat, Ben Athiwaratkun, Manan Roongta, Ce Zhang. Erran Li, Raluca Ada Popa, Koushik Sen, and lon Stoica. DeepSWE: Training State-of-the-Ar Coding Agent from Scratch by Scaling RI.. Notion page 2025. Notion Blog.
Yingwei Ma, Rongyu Cao, Yongchang Cao, Yue Zhang, Jue Chen, Yibo Liu, Yuchen Liu, Binhua Li, Fei Huang, and Yongbin Li. Swe-gpt: A process-centric language model for automated software improvement. Proc. ACM Softw. Eng., 2(ISSTA), June 2025a. doi: 10.1145/3728911. URL
https::/doo /tt../..rg//..111//3332555551 14533729981
Yingwei Ma, Qingping Yang, Rongyu Cao, Binhua Li, Fei Huang, and Yongbin Li. Alibaba ling- maagent: Improving automated issue resolution via comprehensive repository exploration. In Proceedings of the 33rd ACM International Conference on the Foundations of Software Engi- neering, pp. 239, 2025b.
OpenAl. Openai ol system card. Technical report, OpenAI, 2024. URL https:://aaxiy:.
Includes ol and ol oi-m models, safety and evaluation work.
OpenAl. Introducing httppp///goenaa...co///////a//// 2025.
May
OpenAL. Openai 03 and openai o4-mini system card. Technical report, OpenAI, Apr 2025. URL
http://hh....oaacoo/o///////2222 o3 20-and--oa 00----4-miii-- www-mmiiii-isseeeee system cwtssstemmcardd...m card.pdf
OpenAl. Openai 03-mini system card. Technical report system card, OpenAl, 2025a. URL
https://edn http://cddoopp.../eeeeeee owwnaieeaa......o.oo cwm/o3-mm////.. 00mmm/33mm o0mmmn-mmin wwt-ssssteeccaa--eeeeeee 03-mini model, its safety evaluations and reasoning benchmarks.
Description of
OpenAl 12t- & gpt-oss-20b model card, 2025b. URL https:://axiv..rog/ 2558.10922
OpenAl. Gpt-5 system card. OpenAl website, Aug 2025c. URL gpt 5- system-card..dd
ccmm
Jiayi Pan, Xingyao Wang, Graham Neubig. Navdeep Jaitly, Heng Ji, Alane Suhr, and Yizhe Zhang. Training software engineering agents and verifiers with swe-gym. arXiv preprint arXiv:2412.21139, 2024.
Jiayi Pan, Xiuyu Li, Long Lian, Charlie Victor Snell, Yifei Zhou, Adam Yala, Trevor Darrell, Kurt Keutzer, and Alane Suhr. Learning adaptive parallel reasoning with language models. In Second Conference on Language Modeling. 2025. URL
net/ forum?id-
YgWQ7sXPXU
Qwen, An Yang. Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang. Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tianyi Tang, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yugiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report, 2024.
URL https:tp//// /tttps://aaxxi.....a.. http://wwwwwv...aaaa.////. abs /2122241222 15115
ByteDance Seed, Jiaze Chen, Tiantian Fan, Xin Liu, Lingjun Liu, Zhiqi Lin, Mingxuan Wang, Chengyi Wang, Xiangpeng Wei, Wenyuan Xu, al. Seed1. 5-thinking: Advancing superb rea- soning models with reinforcement learning. arXiv preprint arXiv:2504.13911 2025.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Rumxin Xu, Junxiao Song, Xiao Bi. Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models. arXiv preprint arXiv:2402.033000 2024.
Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi kl. 5: Scaling reinforcement learning with llss. arXiv preprint 25X.:55011125000 22599, 2025.
13

---

Published as a conference paper at ICLR 2026
Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng. Heng Ji, and Graham Neubig. Openhands: An open platform for Al soft- ware developers generalist agents. In The Thirteenth International Conference on Learning Representations, 2025a. BtLhttts:// http:s:///peeeeie..oo
Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng. Heng Ji, and Graham Neubig. Openhands: An open platform for AI soft- ware developers generalist agents. In The Thirteenth International Conference on Learning Representations, 2025b. URL
Yuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux, Lingming Zhang, Daniel Fried, Gabriel Synnaeve, Rishabh Singh, and Sida I Wang. Swe-rl: Advancing Ilm reasoning via rein- forcement learning on open software evolution. arXiv preprint (rXiv:2502./84499 2025.
J J Williams. Simple statistical gralient- algorithms for connectionist reinforcement learning. Machine learning, 8(3):229-2) 256, 1992.
Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. Agentless: Demystifying Ilm-based software engineering agents. arXiv preprint arXivv2407.014 240.0019,,, 2024.
Chengxing Xie, Bowen Li, Chang Gao, He Du, Wai Lam, Difan Zou, and Kai Chen. SWE-fixer: Training open-source LLMs for effective and efficient GitHub issue resolution. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Findings of the Association for Computational Linguistics: ACL 2025, pp. 1123-1139, Vienna, Austria, July doi: 10.18653/ org/2022 htd.://....orr/2222....mm.ie.... findings-acl.
2025. Association for Computational Linguistics. ISBN v1/2025. http://..ininggacl.... URL
62/
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao. Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint 27Xiv22505.09388, 2025a.
John Yang, Carlos E Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. Swe-agent: Agent-computer interfaces enable automated software engineering. Advances in Neural Information Processing Systems, 37:50528-50652 2024a.
John Yang. Kilian Leret, Carlos E Jimenez, Alexander Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, and Diyi Yang. Swe-smith: Scaling data for software engineering agents. arXiv preprint arXiv:2504.21798 2025b.
John Yang. Kilian Lieret, Carlos E. Jimenez, Alexander Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, and Diyi Yang. Swe-smith: Scaling data for software engineering agents, 2025c. URL http://////aaaioooo//aa/////////
217988
Zonghan Yang, Peng Li, Ming Yan, Ji Zhang, Fei Huang, and Yang Liu. ReAct Meets ActRe: Autonomous Annotation of Agent Trajectories for Contrastive Selfninaining. In First Con- ference on Language Modeling, 2024b. URL
net ntt/forummidd
OVIBWQGNDA
Liang Zeng. Yongcong Li, Yuzhen Xiao, Changshi Li, Chris Yuhao Liu, Rui Yan, Tianwen Wei, Jujie He, Xuchen Song, Yang Liu, et al. Skywork-swe: Unveiling data scaling laws for software engineering in llms. arXiv preprint arXiv:2506. 19290, 2025.
Linghao Zhang, Shilin He, Chaoyun Zhang, Yu Kang, Bowen Li, Chengxing Xie, Junhao Wang. Maoquan Wang, Yufan Huang, Shengyu Fu, et al. Swe-bench goes live! arXiv preprint arXiv:2505.23419 2025.
14

---

Published as a conference paper at ICLR 2026
APPENDIX
A DETAILS OF MID-TRAINING
We curate mid-training data recipe with a focus on enhancing SWE capabilities. Central this effort is the collection of pull request (PR) data from GitHub, which provides extensive coverage of real-world bug fixes, feature requests, and code enhancements. To ensure data quality, we apply two filters: (i) only retain repositories that have accumulated at least five GitHub stars, thereby excluding sparsely maintained projects with limited community engagement; and (ii) we remove any repositories overlapping with the SWE-hench benchmark (Jimenez al. 2023) prevent potential data leakage. For each candidate repository, we query the GitHub API for all PRs with the state MERGED, while discarding those abandoned, superseded, left under review. To preserve more context information, we also snapshot the entire codebase at the base commit before the first code change the PR.
After data crawling, we incorporate two complementary forms for the natural code change data: (i) natural diff' patches and (ii) PR commit packs. A natural diff patch consolidates all commits in PR into the final code difference, typically expressed as TEARCH-REPLACE format aligns with the Agentless paradigm, in which the model must directly output the final patch. In contrast, a commit pack captures the sequence of human-authore commits within a PR, where each commit message (textual reasoning) is paired with the corresponding code modification (ac- tion). This structure closely parallels the SWE-Agent setting, where intermediate reasoning steps are interleaved with actions. However, the distinction of utilities for the two types data not absolute: commit messages in PR commit pack can still inform the model's knowledge and indirectly strengthen its reasoning ability in the Agentless setting.
blocks. This
Natural diff patches. The natural diff patches used in the mid mid-training data recipe are processed with the following rules:
Incorporate the agentless prompt template (see Prompts 3234 These four prompt tem- plates are also used in the later stages, including cold-start, RL, and test-time self-play), and apply loss mask to the prompt part. For the localization prompt, the response is set the files modified in the ground- truth diff patch.
If related issue to the PR exists, use its content of the related issue; otherwise, use the PR title as the surrogate of the issue content.
If a related issue to the PR exists, prepend the issue discussion at the beginning of output in the code edit response. We aim to strengthen the model's capability of code edit reasoning by leveraging the discussion contents.
Discard PRs that include modifications to files other than {-py, ,md, .rst].
For PRs containing {.md, rss} file modifications, retain only the Python diffs and rewrite them into SEARCH-REPIACE blocks.
Remove PRs involving file additions or deletions.
For the code edits with only line insertions or deletions, preserve the original Git diff hunks as the SEARCH content in the SEARCH-REPLACH blocks.
Ensure that more than three Python files modified per PR.
Apply filtering script to exclude PRs with non-(-py, .md, .ist} modifications, PRs modifying more than three Python files.
Further exclude PRs containing more than five REPLACE blocks.
total of ~50 tokens for natural diff patches are obtained after applying these filtering rules.
Please look through the following Git problem description and Repository structure and provide 1ist of files that one would need to edit to fix the problem.
### GitHub Problem Description ### (related issue / title content)
15

---

Published as a conference paper at ICLR 2026
### Repository structure ##
(file structure induced by the repo snapshot)
###
Please only provide the full path and return most 5 files. The returned files should be separated by lines ordered by most to least important and wrapped with '*.
For example:
filel.py
file2.py
Listing Agentless prompt template: Localization for BugFixer.
Please look through the following GitHub problem description and Repository structure and provide list test files that should be run after applying the patch to fix the issue.
4## SitHub Problem Description ### (related issue / title content)
###
### Repository Structure ###
(file structure induced by repo snapshot)
###
Please only provide the full path and return at most 5 files.
The returned files should be separated by new lines orderea by most to least important and wrapped with
For example:
filel.py file2.py
Listing 2: Agentless prompt template: Localization for Writer..er
We are currently solving the following issue within our repository. Here is the issue text: BEGIN TSSUR ---
(related issue / PR title content)
ISSUE
Below are some code segments, each from relevant file. One of these files may contain bugs.
BEGIN FILE
### (filenamel) (content filenamel]
### [filename2) (content of fi filename2) (...)
16

---

Published as a conference paper at ICLR 2026
...
END FILE
Please first localize the bug based on the issue statement, and then generate *SEARCH/REPLACE* edits fix the issue.
Every +SEACC/REEPAACE> edit must use this format:
1. The file path
The start search block: <<<<<<< SEARCH
contiguous chunk of lines search for in the existing scurce
code
4. The dividing line:
5. The lines to replace into the source code
6. The end of the replace block: >>>>>>> REPLACE
Here is an example:
"'python
### fathweb/Flask/appt
<<<<<<< SEARCH
from flask import Flask
sesssss import math
from flask import Flask
>>>>>>> REPLACE
Please that the ASEARCH/REPLACE. edit REQUIRES PROPER INDENTATION. If you would like to add the line fully write that out, with all those spaces before the code!
print (x) you must
Wrap the -SERCCH/REPLACE. edit in blocks "'python..."
Listing Agentless prompt template: Code edit for BugFixer.
We are currently solving the following issue within our repository. Here is the issue text: BEGIN ISSUE (related issue / PR title content)
END ISSUE
Below are some code segments, each from relevant test file. One or more of these files may be added some new tests which can reproduce the issue.
BEGIN FILE
### (filenamel) (content of filenamel]
### [filename2]
icontent of filename2] (...)
..
END FILE
Please first localize some possible locations in those test files within the repc, and then generate *SFARCH/REPIACE edit updates to the **test** files the repo, so that the erroneous scenaric described in the problem is reproduced.
Every HWARCC/RRPLACR+ edit must use this format:
1. The file path
17

---

Published as a conference paper at ICLR 2026
2. The start of search block: <<<<<<< SEARCH
A contiguous chunk of lines search for in the existing source code
4. The dividing 1ine: ==s_..s
5. The lines to replace into the source code
The end of the replace block: >>>>>>> REPLACE
Here is an example:
"pyyhhon
### mathweb/ /aapp/pppppyyy
<<<<<<< SEARCH
from flask import Flask
=esssss
import math
from flask import Flask
def cost_rulee otw.cssssddo0000rraraeooo
-> None:
try:
sql "SELECT IFNULL (NULL, 100),
(NULL, 100);" result - lint (sql, rules- ("..66("
assert Len (result) ==
except:
print ("Other issues") return
try:
assert result(0] "description") "Use 'COALESCE' instead of IFNULL'
assert result "description") "Use 'CALLESCE' instead of NVL'." print ("Issue resolved")
except AssertionError:
print reproduced") return
return >>>>>>> REPLACE
Please note that the DBEARCH/REFLACE. edit REQUIRES PROPER INCENTATION. If you would like to add the line fully write that out, with all those spaces before the code!
print (x) you must
Wrap the +SEARCH/REPLACE, edit in blocks "'python... 19)
Listing 4: Agentless prompt template: Code edit for TestWriter.
PR commit packs. The PR commit packs used in the mid-training data recipe are processed with the following rules:
Discard PRs that include modifications files other than {-py, .md, .rst].
For (.md, ,ss} file modifications, retain the "diff -git" signature but remove the actual content changes.
Ensure that each PR modifies at most five Python files (with at least one required). PRs exceeding this limit are discarded.
Apply filtering script to exclude PRs containing non-(-py: .dd, .sst file modifications or those modifying more than five Python files.
Filter out all of the developer signatures and GitHub IDs for ethics considerations.
A total of ~20B tokens for PR commit packs obtained after applying these filtering rules.
18

---

Published as a conference paper at ICLR 2026
In addition, incorporate synthetic data to further enhance both the reasoning and agentic capa- bilities of the model. A key observation is that the ground-truth reward for the localization stage in the Agentless setting can be derived directly from the diff patch, since the set of files requiring modification explicitly indicated.
Synthetic reasoning data. To improve reasoning quality, perform lightweight SFT of the Qwen-2.5-72B, Instructt model on 2,000 R1 trajectories. The resulting model is then used to generate large-scale rollouts for the localization stage of both BugFixer and TestWriter. We retain only the rollouts that achieve exactly correct file localizations. This procedure yields approximately ~ ~00 tokens of reasoning
sassoninginntesivee intensive data dedicated Agentless localization in the mid-training recipe.
Synthetic agentic interactions. To strengthen agentic capabilities, we simulate agent- enviromment interactions with custom tool designed to mimic file-system operations without execution. This design is motivated by practical constraints: while repository snapshots from GitHub are available, all snapshots are equipped with an executable Docker environment. As result, shell commands are disabled. Instead, we introduce synthetic tools that allow the agent to view file contents and per- form keyword-ber
search for localization, which effectively reproduces the first stage of Agentless but in an agentic manner. The specification of this tool set is covered in the system prompt, which is then used to elicit agentic interaction rollouts from the Qwen-2.5 2.5-72B 12B-Instruct model. The complete system prompt is provided in Prompt We apply loss mask only to the system prompt, and en- able the model to simultaneously learn both actions and observations along the trajectory, inspired by Yang et al. 2024b). This approach integrates both policy and world modeling into mid training.
Your job is to look through the given tit problem description and Repository structure, and edit updates to the files in the repo to resolve the problem.
The job is divided into
stages:
+ In Stage 1, you should localize the files the files that you would need to edit to fix the problem.
+ In Stage 2, you should edit the updates to the repo.
Let's begin from Stage 1 to localize the bugs:
In Stage 1, besides reading the provided Repository structure, you can use the following skills for exploraticn. The skills are to be called in an environment wrapped by <execute> and </execute>, listed in the form of python functions belcw:
open_file (path: str, is_all Nore False, line_number: int | None 1, context_lines: int None - 100) -> None: Opens the file at the given path in the editor for exploration. By default, only the first 100 lines of the file
displayed. To open the entire file, set 'is_ to 'True'.
The "context_lines" parameter determines the maximum number of lines to be displayed, with a cap of 100 lines. Use 'scroll_up' and 'scroll_down
view more centent up down.
a 'line_ 'line_number' number is provided, the window will be moved to include that line.
Note: When 'is_all* is set to True' the 'line_number' and context_ context_lines' parameters will not take effect, the entire file will opened and displayed without any line-specific focus or context limitation.
Args:
path: str: The path to the file to open. the full path of the filename should be provided.
is_all: bool None False: If set to 'True', the entire file will be opened. Defaults to 'False',
line iie__number: int None = 1: The line number to move to. Defaults to 1.
context_lires: int | None 100: Only shows this number of lines in the context window (usually from line 1), with numeen as the center (if possible). Defaults to 100.
goto_line (1ire_number: int) None:
Moves the ndow show the speci fied 1ine number.
19

---

Published as a conference paper at ICLR 2026
Args:
line_number: int: The line number to move to.
coto_class_or_fure
str) -> None:
Moves the window to show the specified class or function in the open file.
Args:
str: The name of the given class, function, or method in class to move to.
scroll scroll_down () None:
Moves the window down by 100 lines.
Args:
None
scroll_up () -> None:
Moves the window up by 100 lines.
Args:
None
search_daa (search_term, str, dir par_: str None) -> None:
Searches for search_term in all files in dir. If dir is not providea, searches in the entire repository. Filename, fine- grained line number, and the relative class function it is located in (if applied) will shown for each found position. Args:
search_term: str: The term to search for. dir_path: str: The path the directory to search. Should be full path filename.
search_filelt (serchhhtee str, Alle_path: str None None) -> None: Searches for scarch_term in file. If file is not proviced, searches in the current open file. Filename, fine-grained 1ine number, and the relative class or functien it located (if applied) will be shown for each found position.
Args:
search_termy str: The term to search for.
file_path: str | None: The path the file to search. Should be full path filename if provided.
find fille name: stE, dir path: str None) -> None:
Finds all files with the given name in the specified directory, If dir is not provided, find the entire repository.
Args:
file_name: str: The name of the file to find.
dir_path: str: The path to the directory to search.
str_replace (path: str, oldssr, new_str) old_str=|the old content to be replaced] new new_str=(the new content after replacement]
-> None:
Replace old content (old_str) in the file the given path with the new content (new_str). This the skill that you will be using to edit the updates.
Args:
path: The path the file
updated. The full path of the filename should provided.
old str: The old content to be replaced. Note that this argument should be written in new line starting with "old_str =", and the string content should not be quoted.
new_str: str: The new content after replacement. Note that this argument should be written in new line starting with "new_str =", and the string content should not be quoted.
Example:
20

---

Published as a conference paper at ICLR 2026
Assuming call is shown follows:
...
str_replace ("filename.py" old_str, now_ste)
old_str=
nev_atr= c
Then it will function as replacing the aln' string with the blnc , string in the "filename.pp py file.
insert (path: str, insert_line: int, new_str) new_str=[the new content to be inserted]
-> None:
Insert the content (new str) in the file at the given path. When you want to add an entirely new class/ function the file, it would be better to use this method.
Args:
path: str: The path the file be updated. The full path of the
filename should be provided.
insert_line: int: The Line number below which the new content to be added. This Line number should be within the range of lines of the file: 10, Lines_of caaes_of_the__eeeeeeeeeeeeeeeeee [e__th__Fielll Specifically, when insert line 0, the added content starts from the top of the file.
new_str: str: The new content to be inserted. Note that this argument should be written in new line starting with "new_str =" and the string content should not be quoted.
Example:
Assuming call is shown as folloss:
insert ("test_ filename py", new_str= def test_add() new_str)
assert add (1, 2)
Then it will Cunction as inserting the string def test_add () :
assert a1d 2) 3 below the line 5 the
test filename. py file.
stop () -> None:
Terminate the editing process.
Args:
None
NOTE:
Reaponses should be concise.
When exploring, you should attempt fewer things at time: Include ONLY ONE <execute> per response, and use a SINGLE skill listed above within the <execute> environment. DO NCT use other python functions, as the renment does not support them.
should first reason the verbal form, then use a skill with execute> and </execute>, You should avoid apologies and thanks in the responses.
when you finish exploring and analyzing with the provided skills, please return at most files with the full path only. Zach full path should be placed in a single line, INSTEAD OF BROKEN WITH MULTIPLE LINES.
The returned files should be separated by lines ordered by most to least important, wrapped with an and NOTHING ELSE.
An example for a full output:
full path to filel.py
21

---

Published as a conference paper at ICLR 2026
full path to File2.py
1.1
Now Let's start!
#*# GitHub Problem Description ###
(issue content)
### Repository Structure ###
(file structure]
###
Listing A non-execution of tools empowering the simulation of agentic interaction trajectories.
After completing the initial localization stage, the agent is guided into the code-editing phase through follow-up instruction: "Now let's move on to Stage and edit the updates. Remem- ber, you can still decide at any point whether file actually requires modification." We retain partial rollouts from Stage 1, provided that the localization results include least one correct file.
In Stage 2, we first simulate the agent's 's interaction by allowing it to open incorrectly localized files, and we artificially inject agentic reasoning patterns such as realize that do not need to modify this file" after inspecting the file content. This procedure is designed to strengthen the self-reflection ability of the agent by exposing to false-positive contexts regarding the issue to be solved.
Subsequently, transcribe the ground-truth PR commit pack into trajectory form: each commit message treated as the agent's reasoning step, and each code update is represented as the cor- responding action, expressed through the "str replace" or "insert" tools. These interactions appended to the trajectory, followed by terminating "stop" call. Due to storage constraints on repository snapshots, this trajectory simulation is applied to only subset of PRs. Overall, this pro- cess contributes approximately 10B tokens of agentic interaction data to the mid-training recipe. Future directions for scaling this component in the data recipe include leveraging the idea of envi- ronment scaling (Yang et al. 2025c).
Training. We perform mid-training using standard next token prediction approach, initialized (Qwen et al. 2024) model. We upsample the synthetic part of the data by factor of during mid-training. inspired by the practice in Grattafiri al. (2024); Owen al. (204): Gu et al. 22255) A global batch size of 256 with a maximum sequence length of 32K tokens used, optimizing for long-con contect capabilities necessary for real-wordd software engineering tasks. The learning rate is set to 2e-5, with a cosine decay schedule and minimum learning rate of 2e-6. The warm-up phase covers over approximately billion tokens, followed by learning rate decay until approximately 150 billion tokens processed.
from the (wen2..5772BBal.
B DOCKER ENVIRONMENTS
Table 3: The sources of the docker environments used the development of Kimi-Dev.
Dataset Name
Dataset Link
Number of Dockers
SWE-Gym (Pan et al. 2024
https://hugginnfaaces
2,356
datasets/ SWFettt////E-Gmmm/
SWE-Gym/
SWE-bench-extra
nttps:// sttps://hugiingfacecu
3,846
Badertdinov et al. 22024))
Catasets/nebing
SWE wwwnch-exttee
R2E-Gym G-m-Gyy--ii (Jain et al,
https: htip:://aqgiooaace..o//
3,671
2025
Catasets/R2E Gym/
R28 R2E Gy--Gym-Litee
22

---

Published as a conference paper at ICLR 2026
Docker environment construction. To validate non-ground-trune -rrutd patches generated by model roll- outs and expand our dataset, we required executable Docker environments. We combined publicly available datasets with custom-configure Docker environments (see Table. B). Among them, SWE- Gym and R2E-Gymm-Litt open-source their dockers that we can directly use. For datasets lacking Docker support (SWE-Rench-Fxine we implemented automated configuration method:
1. Initialize a Docker environment with fixed dependencies.
2. Select Python version based commit year.
3. Install dependencies via requirement txt and "pip install .".
4. Resolve ModuleNotFoumm enrors during test execution.
5. Validate success if FAIL TO PASS test transitions from failing (without gt patchh to passing (with gt ttt.patch.....
Out 6.38k SWE-bench-extra instances, 3,846 environments are successfully constructed and sub- sequently used for cold-start and RL training.
C MORE DETAILS OF RL TRAINING
C.1 PROMPT SET SELECTION
16
Dogiber RL who Posilive Reinforca
M
In the main text, we introduce the adaptive prompt selec- tion method for RL training. Specifically, we construct an initial prompt set of 1,200 problems by selecting those with pass @ 16 > 0 from SWE-Gym (Pan et al. 2024).
Pass Rate (%)
40
SWE-bench-extra (Badertdinov et al.) (025)) and R2E- gym (Jain et al. (025). Then, every 100 training steps, expand the prompt by adding 500 new problems. These additional problems are randomly sampled and fil- tered from the pool of problems for which the current model has pass @ 16 = O, thereby progressively increas- ing the difficulty and forming a proper curriculum.
Training Steps
Figure 8: Ablation of positive example reinforcement during 72B Bugfixer RL.
C.2 RL EXPERIMENT ABLATION
Figure g shows the performance of the Qwen2.5-14B model in RI. experiments, where both the BugFixer and the TestWrite Wrier exhibit clear scaling law behavior.
8500
Token Length
34
5500
Tokte Length
Reproduced
Reproducel (%))
Token Length
8000
%
Token Length
50000
24
Pais Rate
3000
20
7000
6500
3000
25
100
150
175
200
50
100
150
200
RL Training Steps
Training Steps
(a) 14B BugFixer
(b) 14B TestWriter
Figure 9: RL scaling experiments Qwen2.5-14B model.
Furthermore, Figure 8 illustrates the effect of our proposed positive example reinforcement. We incorporated this improvement in the later stage of training and found that the RL of BugFixer achieved better performance compared not using this trick. The main reason is that positive
23

---

Published as a conference paper at ICLR 2026
examples reinforce the correct reasoning patterns, enabling the model converge more quickly, especially when its exploration ability diminishes in the later stages, It is noticed that a similar trick was also applied in the development of the Seed 1.5--.hinking model (Seed et al. 2025).
D AGENT INFRASTRUCTURE
We present a unified sandbox architecture designed for hybrid cloud cluster environments, lever- aging Kubernetes orchestration provide researchers with flexible, secure, and highly scalable code execution platform. The architecture is accessible through a Python SDK and implements an ephemeral, use-and-de -eestod paradigm that ensures resource efficiency and security isolation. The system demonstrates exceptional scalability, supporting over 10,000 concurrent sandbox instances while maintaining performance stability. This infrastructure serves foundational platform for diverse computational scenarios, including automated code evaluation and RI. experiments.
The system employs sidecar container pattern efficiently orchestrate and manage heteroge- neous collection of over 25,000 docker images sourced from multiple datasets. This architectural choice enables optimal resource utilization while maintaining isolation between different execution environments. Additionally, we have developed a proprietary Software Engineering (SWE) image- building pipeline that continuously expands the repository of supported images, thereby enhancing the system's udaptability to diverse computational workloads and research requirements.
E
CASE STUDIES UNDER AGENTLESS
E. BUGHIXER TRACES
We provide a Bugfixer's response trace during the code repair stage (see Listing 6). From this, we can observe that the initial code given was return selffi, j pos other. cols]. but the model quickly realized its mistake and corrected it to return selfli, other.col. Although some repetitive self-reflections followed, the increasing response length indeed improved answer accuracy and robustness in experiments.
<think>
Okay, Let's tackle this problem. The reported issue where the cetrixxcoo__nseee () method isn't t working correctly. The example given shows that after inserting a matrix into at position 3, the resulting matrix has the inserted columns shi fted incorrectly. The expected result would have the inserted columns and 4, but the actual output shows them starting position 3, but the original matrix's columns after the insert are shifted upwards instead of remaining in their original positions.
positions 3
First, need to understand how ins_insert works. Looking the code provided, especially in the MatrixShaping class in sympy/natrices common. py, the Leval_col_insert methed is responsible for inserting columns. The current implementation shifts the existing colurns to the right by the number of columns inserted. However, the bug might be in how the original matrix's columns being shifted.
Let me look at the existing code for Lvall col insert:
def _eval_col cest__nneetttee (self, pos, other): cols self.cols
def entry (i, i): if pos:
return self[i, i]
elif pos j pos other.cols: pos]
return otherli, return self(i, j - other.
return fe_new (self. rows, self folos other. cols,
24

---

Published as a conference paper at ICLR 2026
lambda i, entry }1)
wait, the problem here is in the last case. when j is greater than or equal to pos + other. other.cols, cols, the codc subtracts (pos + other.cols. from j. But that's s incorrect because the original columns after the should start at original matrix's s columns that come after the inserted ones, their new positions should be j other.cols. But in the current code, when j is in the original matrix's columns (after the inserted ones) the code is subtracting (pos other.cols), which would incorrectly shift them.
inserted
pes other. So for the
wait, let's think again. The original code for the entry function:
if pos:
return self(i, j] # original columns before pos clif pos <= j pos other.cols:
return other j pos] # inserted columns
return selfii, j pos other.cols] $ original columns after the inserted
wait, no. Let's say the original matrix has cols = 6. We insert 2
columns at pos=3. The new total cols is 6 + 2 = 8. The original columns after pes (columns 3, should
be at positions
5, 7, But according to the code, when 3 other. cols -3+2-5), then the code returns selfli, -5 -21 selfli, j -7]. Wait, that's not right. For example, if j-5 (the new column 5), then j pos other.cols =5 -3 -2-0. So the original column would mapped to the column 5. That's definitely wrong.
+2=5 (since pos
Ah, here's the mistake. The code is subtracting (pos + other.cols) from j, but that's incorrect. The original columns after the inserted ones should be shifted to the right by ether.cols. caleulation should be - other.cols, For example, original column becomes 3 +2-5, original column beccmes 6, etc.
the correct
so the 1ine in the code:
return self [i, j pos other.cols
should be:
return selfli, j other.cols)
Because the inserted columns have taken up occher.coss positicns starting at 'pos', the original columns after 'pos' are shifted by 'other.cols' to the right. Therefcre, get the correct original column index, we subtract 'other.cols' from the current j.
So the fix is to change the subtraction from (pos + other.cols) to other.col cols. let me check:
Original code:
return self [i, j
other.cols]
Corrected code:
return solf [i, j other.
Wait, no. Wait, the original columns after the inserted block (which starts at pos and has other.col. columns) would start at pos other.cols. So for those j values, the original column index would be - (pos other.cols). Wait, no. Let's think with numbers.
25

---

Published as a conference paper at ICLR 2026
Original matrix has columns. Insert 2 columns position 3. The matrix has columns, The original columns 0,1,2 remain positions 0,1,2, The inserted 2 columns are at pesitions 3 and 4. The original columns 3,4, now become 5,6,7.
So for j 6,6,7 (original columns 3,4, 2,4,5, 5), the mapping is:
5 #\rightarrows 3
6 $\rightarrows
7 &Arightarros
So the formula should be Aegginal_columen
inserted colluns._ In this case, inserted_columee columns is other.cols. So the code should subtract other.cola from j when accessing the original matrix's columns after the inserted block.
Therefore, the 1ine:
return self(i, j pos other.cols
should be:
return self [i, j other.cols]
Because is where the insertion starts, but the original columns after the inserted block are shifted acher.cola. Let's verify with the example:
pos =3, other.colss cols-2. For j=5 (new column 5), original column index is -2=3, which is correct. For j=6, 6-2-4; f=7, 7-2-5. That's correct
So the error the original code was subtracting (pos other. instead of other.cols. The pos is the starting position of the insertion, but after inserting other. cclss columns, the original columns after the inserted block Therefore, the correct subtraction is ether.cols.
shi shifted by other.cols.
Thus, the fix is to change the line in the Leall col insert method from
return solf j pos - other.cole. to:
return self [i, j other. Now, looking back at the code in the colal_col insert functien: The code for the else clause is: return self [i, j pos - other.cols]
Which is incorrect. Changing that to other.cols should fix the problem.
So the SEARCE block would look for the lines:
def _eval _eval_collinnert (self, pos, other): ins__insertti cols self.cols
def entry (i, }): it j pos:
26

---

Published as a conference paper at ICLR 2026
return self [i, elif pos <= j pos other.cols:
return other [i, - pos] return solf(i, j pos - other.cols]
return self self._newwsel) (self. rows, self.cols + other. cols, lambda i, j: entry(i, it)
And the REPLACE block would change the last part to:
elif pos
pos other.cols:
return other [i, - pos]
return self(i, j other.cols]
So the correction is in the return statement of the entry function, The original code subtracted (pos other.cols) from i, but it should subtract other.cols instead.
Therefore, the correct SEARCH/REPLACE block would be:
### sympy/matricce ctte//ccoooo.ppp.c py <<<<<<< SEARCH
def _eee___cool insert (self, pos, other):
cols self.cols
def entry (i, j): if j < pos:
return self[i, pos other.cols:
elif pos return other
pos] return self(i, i pos - other.cols)
return self._new rself. self.cols + other.cols, lambda i, j: entry (i, 31)
def _eval _eval_col nsert (self, pos, other):
def entry (i, j): if j pos:
return selfli, jl
elif pos <= j < pos other.cols: return otherli, return self(i, j other.cols]
pos]
return self._new (sclf. rows, self.cols + other.cols, lambda i, j: entry(i, i))
>>>>>>> REPLACE
This change corrects the column shifting calculation for the original matrix's s columns after the inserted block. Instead of subtracting pos other.cols), we subtract other.cels. which properly shifts the original columns the right by the nurber of inserted colurns.
</think>
''python
### sympy/mate http:///aaiicccccooooooooo py
<<<<<<< SEARCH
def _oval_col__neeeeeeee__e_o_eeeee_ (self, pos, other):
cols self.cols
def entry (i, i): if j pos: return self(i, j] elit pos <= j < pos other.cols:
return other li, - pos]
return selili, j pos other. cols)
27

---

Published as a conference paper at ICLR 2026
return self new (self. rows, self.cols other. cols, lambda i, j: entry (i, j))
def _eval evai_col_insent (self, pos, other):
def entry (i, j): if j pos:
return selfii, jl
elif pos <= pos other.cols: return otherli, return self(i, j other.cols]
pos]
return self._nes (self. rows, self.cols + other. lambda i, j: entry i1}
>>>>シ>> REPLACE
Listing 6: Reasoning trajectory example for BugFixer.
E.2 TESTWRITER FALSE POSITIVE CASES
During the RL training process, observed that the Testwriter component exhibits some false positives, which can result in tests that contain certain gaps. This might cause the test patches generated by the model be mistakenly considered as correct. Consequently, the performance improvement of Testwriter does not accurately reflect the actual performance. In this section, we provide two intuitive examples illustrate this phenomenon, and leave further enhancement for TestWriter as future work.
Listing 7 shows the ground-truth test patch for instance djjngoo_django-1627 with Listing 8 showing the test patch generated by Writtr. A detailed comparison reveals that the model- generated test cases are not comprehensive: Two locations should have been tested, but only one was checked, and the effect of ma mark_safe was not considered. In practice, this allowed the model- predicted patch to pass the TestWriter reward check, but result in false positive.
The second example instance is ground-truth and model-generated test patches. In this case, the coverage of the generated test is also insufficient. The generated test does not account for the need to satisfy the associativity prop- TP (A+*x, B**x). As
with Listings and 10 showing the
erty, e.g., assert tensor product..t...m........m......... sodduct_siing simp (TP (A, B) **x)
consequence, it fails to serve reliable judge for BugFixer.
diff --git a/testss/emplateeee httppp/aaateeettttiseeeeeeeeeeee http://iiiiieeeeetseeeeeeeeeeeee tests/test. escapeseq. b/ tests/template tests/ htttra/eeeetteeeeeeeeeeeeeeeeeee htt.:/eesssettt_eecepeeeeepppe
new file mode 100644
/dev/null +++ ////a///wogg..cccce///////eeeeer
0 +1,59 80
+from django.test import SimpleTestCCse
+from Stango.uils..saees
import mark_safe
+from .utils import setup
+
+class BscapeseqTests (Simple
The "escapeseq" filter works the same whether autoescape on or
off,
+ and has no effect on strings already marked as safe.
@
+
lescapeseq_basice ( eacapeseq join:", {( blescapeseq/ join:",
}
28

---

Published as a conference paper at ICLR 2026
def test_basie (self)) output self. rer..eeeette....ettoooooooo
"escapeseq_ bassic""
(" ("xsy", "<p>"), "b": maarkssfaa"xxy" mark Dark_safe("
I>"
self self. (output, "xsampry, &it:pagt; x&y, <p>")
@
"escapeseq_autooe off": autoescape off 8){ alssappeeeljoiin: }/
{{ blcccaposo jocaapsoolijonn""" join:", "}}($ endautocscape 8)
def test of (self))
output - self.engine. render_to_string
"escapeseq "ecapsser_utoeesape..o aut.aaacaeeeoeeee.....
("a": ("xsy", "<p>"), "b": |mark Sark ("xay") mark safe("<
p>")1>,
self.assertEqual (output, "xéampry, &lt;pigt; -- x&y, <p>")
@setup "escapesee__join"" 'I alescapeseql join: "<br/"/ }11)
def ceat_chain_joinee (self)
+
output = self wwl.f.engine..re.. htt.rarreerrrrrrotttoooooeeeeaar
("a":
["xéy", *<>>"|])
self self.assertEqual (output, "xkmp:ydbb/>41::p4tt
@
eecapser_iin
" autoescape off {){{ alescapeseq "<br:=<>>>""
"<br/>" }
endautoescape $1
def bett.ccciiii.ooaaarrrrsp...dddd.
output = tong
("a": ["xay", "<p>"]}
self assertRuuul (output, "#kamp:ycch/>xxtpppitt/""
Listing 7: GT test patch of djjngge_.jjango-16877
diif http:////wwaaaeeee..e/////eeeeee b/tests/
+++
@0
-20,3 +2 00
def test_safeseq02 (self)
output = enginengine. render_to_strinee ("safeseq02", ("a"; ["6",
*<"]])
self falf.ase assf.aase::tqual (output, "s, < & <")
New tests for escapeseg
@setup (i "cccapeeeq1"" 'If alescapeseq join:", })())
def test_ cest_escapeseq01 (self):
output self.engines render
wendeert_reen__oo______oo_ string (" ("a":
[""", "<"]))
29

---

Published as a conference paper at ICLR 2026
self.aaserrEqualloooopop (output,, "sampi, &lt;")
"sscapeseq02":
{{ autoescape off {) al escapesee|joonn"" " }1(8
endautoossape 8)
def test_ (est_escapesee)2(e11)
output engine. render_to_string ("sccapeeee0) ("a":
["a", "<"1))
self aelfrrasett (output, "samp;, &lt;") ")
Listing 8: Model predicted test patch of cjangoo_djjango-6687
diff /
b/sympy/
+++ sasggy//psssttaaaaoo//////sssssp
-10, +1c, 06
from compyyphyyics. quantun. qusstum.densitt import Density
from sympy.
import Tr
Sy commutative-False) +A, B, c, D symbols ('A,,, C, commutative-Falss x = symbols('x')
-A, B,
matl ( 2+IJ, I, 2]1)
-47,6 +47,11 def cos__cooor_ppooctt.ccomoaaaaaa..
def cest_tonsor__prodtc__ssppp
assert tensor_ pnnsoo_prodduct_ssmpppp (TP (A, B)+TP (B, C)) = TP (A+B, B+C)
tests for Pox-exxssssoons
+ essert tensor_prodfct_ssimp (TP (A, B)*xx) TP (A**x, B**x)
+ assert tensor pansorr_roddct_eaiee (x+TT (a, B)x+2 == X+IF (A**2, Bx*2)
assert tensor prmsorr_roddct_siine simp (x* (TP (A, +))+)) +TP (C)
== X+TP (A++2+C
BB++2+D)
assert tensor_product_singgg (TF (A, B) -T (C, +***
B) B) (C**X,
D**x)
def Jest_issue_59233
Listing 9: GT test patch for sympy-aymyy-13374,
diff --git / http:///pwppppootttt///////s/sss b/ b/sympy/
ht2po///wyysscaaaaa////eeeetteer @0 -47, +47,27 88
def test_tonsor_proddc__iiii
assert tenso:_product_simp (TF (A, TB C) (A+B, BBC)
+
Test tensor product powers = symbols a'a', commuaaiive=Faise
TP (1, ti_simp twtsor_product__siiee (t1)
TP (1,
+
assert t1 simp == TP (1, 1), f'Expected 1x1, got [t1_simp)"
Test with Pauli matrices ces from sympy-phystl cap.sicccpallllllpp a algebra import Pauli
30

---

Published as a conference paper at ICLR 2026
TP (2, Pauli(3)) (1, Pauli t2_simp tensoc_product_simmpp (t2)
assert t2 timp TP (1, 1), E'Expected 1x1, got {t2_simp)
+
Test substitution cases from issue [P,,,
exprl = TP
+ resulti expr1. (a, 1)
+ assert tensor_ sensor_product_sinn (resultl) = (P(1, 1), iuuubstitution 1
failed: (result1)"
expr2 (P, Pauli (3)) (1(), a)
result2 expr2.aubs (a, Pauli (3))
assert tensor_prodiee sensor_product_simp (rroductssinppresulaaa (result2) -- (P, 1), f"Substitution 2 failed: (result2)"
def @est_issue__92330
Listing 10: Model predicted test patch of sympy sympy-13974.
F EMERGENT PARALLEL SCALING KIMI-DEV
Compared to the self-pl play method that leverages tion during test time, parallel scaling (Chen et al. [025b]
51.6
Pan al. 2025)
recently-introoee -ecntll-introdueee test-time scal- ing method that aggregates multiple independent rollouts for additional improvement. While previous works in-
Pass Rate (1%)
19.3
centivize the ability of parallel scaling through training. train a separate reward model (Pan et al. 2024, Jain et al. 2025) for patch selection, we observe that the par- allel scaling effect emerges in our model without any spe- cific training. Specifically, patch candidates from parallel rollouts of BugFixer are used to prompt the model sum- marize their differences and compose new bugfix patch.
19
48
47
Number of Patches
Figure 10: Emergent parallel scaling re- sults on SWE-bench Verified.
Listing I is the prompt template of parallel scaling. the template, promb 1 em..stttee.... stat ement is the GitHub issue, and brajs.conten represents the content of multiple patch candidates.
We currently solving the following issue within our repository. Here is the issue text:
BEGIN ISSUE
(problen_statement
END ISSUE
Below are some thinking trajootories, each from 11m reasoning model.
Only one trajetory is right.
BEGIN FILE
(trajs_content:
END FILE
Please first summary and analyze the key differences between the trajectories, and then generate ASEARCH/REPLACE, edits to fix the issue.
Every SSFARCH/REPLACEE edit must use this format: The file path
2. The start of search block: <<<<<<< SEARCH
3. A contiguous chunk of lines to search for in the existing source code
The dividing line:
31

---

Published as a conference paper at ICLR 2026
The lines replace into the code
The end of the replace block: >>>>>>>
Here is an example:
'pyyhhon
### mathwab/Flaskkapp.1
<<<<<<< SEARCH
from flask import Flask
import math
from flask import Plask
>>>>>>> REPLACE
Please note that the +SEARCH/REPLACE edit REQUIRES PROPER INDENTATION. If you would like to add the linc fully write that out, with all those spaces before the code!
print (x) you must
Wrap the SSEACCH/REPLACE The summary of the key differences between the trajectories should be in the thinking part.
edit in blocks "'yython... ".)
Listing 11: The prompt template for parallel scaling.
The results Figure 10 show that the performance of the parallel aggregation improves as the number of patch candidates in the prompt increases. The advantage of this scaling paradigm majority voting lies its ability to leverage the model's own capacity to analyze multiple candidate patches, thereby surpassing the simplistic approach of weighting answers merely by their frequency of occurrence. Listing 12 covers full prompt example with 14 different input patch candidates.
Note: As Listing 12 long, OK to jump Listing 13 directly observe the model behavior.
We are currently solving the following issue within our repository. Here is the issue text: BEGIN ISSUE
Data <érilename isn't relative to the YAML sile
The [docs say) htaps://gabbi.. readthedocs io/en/ latest/ aatest/formatt format htmi#datal
If the value is string that begins with <p then the rest of the string is treated the of file to be loaded from the directory as the YAML file.
But I haven't found it works like this unloss I ed into the dircetory containing the yaml file.
"_load_datan _load_data_file" says:
''yyhhon
be.ppah.basseame path os.path. join (self. _i os oa.path. basename (fi (ename)))
this does a few things:
makes the path begin with 'self.test directory" (defaults to '.', the
current working directory)
disguards any folders in the <B 'filename path appends 'filename"
This means, if I'm in 1', and I have a test that says 'data: <θcake. ipg', I cannot run my tests as advised in the does: 'gabbi-run /my/test. maast.yan yaml /my/other.yam
'FLLeNotFoundEEroo [Errno 2] No such file or directory: './cake jpg'
So in test running script, have to start with 'ed tests/'.
32

---

Published as a conference paper at ICLR 2026
My preference, would be make the path relative the location the yaml file. That way, can run my tests from whatever directory.
Obviously this isn't t possible when redirecting with ' as the filename is never seen by gabbi. So I'm happy is discuss other ideas.
Furthermore I can't keep my test data in child directory, unless ed into that airectory like: ''shell
cd sam__tests/test_dart gabbi-run -- w.//est_chings.yee.
So for that reason, I'd like to allow directories like '<8test_data/ filename. txt to be included in the path.
Happy to write the patch, if we come up with an approach here. END ISSUE
Below are some thinking trajectories, each from 1im reasoning model.
BEGIN FILE
Answer 1:
"'python
### gabbi/runner.py
<<<<<<< SEARCH
import argparse
from importlib import import_module
import sys
import unittest
import argparse from importlib import import_mode module import os
import import unittest
>>>>>>> REPLACE
"'python
### gabbi/runner.py
<<<<<<< SEARCH
run_suite (handle, jabeler_ host, port, prefix, force_ssl=
False,
faiifast-False):
""*Ron the tests from the YAMI, in handle. """ data utils.load_yaml (handle) if force_ssl:
if 'defaults' in data:
datal'defaulta [''ssl) True else:
('ssl'; Truc]
loader = unlttest.draulleett.oo.
test_svite = suitemaker cesst_suite__rrrt
loader, 'input', data, '.', host, port, None, None, prefix- prefix,
bandlers=handeer.obb
result ConciseTes::unnerr
33

---

Published as a conference paper at ICLR 2026
verbosity=2, Fallfast=failfast) canttt__ssitee return result. wasSuccsssfuu
def run_suite (handle, handler_objecte, host, port, prefix, ferce_ssl= False,
failfast=false, test_directory=".")
"""Run the tests from the YAML handle.
data utils.Load (handle) it force_ssl:
if 'defaults' in data:
data ['ss]]] else:
Latal'defaulle
s'ssl': True]
loader unittest. unittest. defuuu-essttsse
test test_suite suite suitemaker test crees_suite__te from_ddfed
fron_dict(( dict(
loader, 'input', data, test_directory, host, port, Nonc, Nonc, prefix-profix, handlorsttandlersoohjetts
result conci
verbosity=2, failfast=fail fast) run (test_suuite)
return result. wassucesass ()
>>>>>>> REPLACE
''python
44d hubbi/runner.py
<<<<<<< SEARCH
else:
for input fill in input_files:
with open (input_file, 'r') as fh:
success = run_suite(f)) (fh, handler_objects, host, port, prefix, force_ssl, failfast)
else:
for input_file in input_files:
test_directory - ww.path..c.. rname (input_file) or '.' with open (input_file, 'r') th:
success run_suite handler_objects, host, port, prefix, force_ ssl, failfast, test
>>>>>>> REPLACE
Answer 2:
''python
### gabbi/runner.p
<<<<<<< SEARCH
run_suite (handle, handler_objects, host, port, prefix, force_ssi=
False,
failfast=False):
def run_suite (handle, handler_objects, host, port, prefix, force_ssl- False,
failfast=False, test cast_directory="..
>>>>>>> REPLACE
" ''yython
#4# gabbi/runner.py
<<ce<d< SEARCH
test_suite = suitemaker ceenmakerrtet__ntne
loader, 'input', data, '.', host, port, None, None, prefix- prefix,
handLers-handee_.bbifo
test suite suitemaker test suite from dict
34

---

Published as a conference paper at ICLR 2026
loader, 'input', data, test directory, host, port, None, None, prefix-rrefix,
bandlerrs-handeer.00jjjssss)))))
>>>>>>>
REFLACE
'python
### gabbi/runner.py
<<ESE<< SEARCH
success run_suite handler_objecte host, pert, prefix, force_ssi, failfast)
test_directory os. (imputt_fie or success run_suite (fh, handler_objests, host, port, prefix, Sorce_ ss1, failfast, dest_directorystest. directorry
>>>>>>> REPLACE
Answer 3:
''python
### /
<<<<<<< SEARCH
def run_suite (handle, handler_obfectee host, port, prefix, force_ssi=
False,
failfast=False):
"""Run the tests from the YAML in handle. """
data utils.lood_yanll (handle) if force_ssl:
if 'defaults' in data:
datal'defaults' [ [sssl'] True else:
datal'dcfullt'') ('ael'; Truc]
loader unittest. dettesrrddeaullTTetioeee
test test_svite suitemaker ttws__suite_rron_t_ee_
loader, 'input', data, '.', host, port, None, None, prefix- prefix, landlerr-handler_oojee.
result = ConciseTesttuuner/ verbosity=2, failfast=failfast) .un ctnsst___suite return result. cesulttwasucccssfuul
def run_suito (handle, handler_objeets, host, port, prefix, force_ssl- False,
failfast=False, dest_directory=".") ""*Run the tests from the YAML in handle, """ data Loads (handle)
if force_ssl:
if 'defaults' in data: (atal'defaults' [''ss]'] True else:
('ssl'; True]
loader = unittest. deatestt.defaultTsstoeeee.
test
test_suite suite suitemaker. wwsst_ssite_eeeeeeeeeeeeee from dict(
loader, 'Input', data, test_Girectory, host, port, None, None, prefix-proflx, handlers-tandderrobjee
result = ConcisseTessRummer
verbosity=2, failfast=fail fast) (teat_suite)
return result. ()
>>>>>>> REPLACE
"pythonn
35

---

handler_objects, host, port, prefix, force_ssl= <<<<<<< SEARCH Published as a conference paper at ICLR 2026
#*# gabbi/runner.
<<<<<<< SEARCH
for input_file in input_files:
with open (input_file, 'r') th:
success run_suite(fh, handler_objests, host, pert,
prefix, force_ssl, failfast) not failure: once failed, this is considered immutable failure not success
it failure and failfast: break
for milu in input_files: test cist_directory = os. cirnaniiran file) with (ie((inpt__,lle, 'r') Eh:
success run run_suite(
fh, handler_objocte, host, port, prefix, force_ssl, failfast, get_ellcctoryttst..iieee. if not failure: once failed, this is considered immutable failure not success
if fai lure and failfast: break
>>>>ラ>> REPLACE
...
Answer 4:
''python
### gabbi/runner.py
def run_suite
False,
fallfast=False):
run_st run_svite (handle, handler_objecte host, port, prefix, force_ssl- False,
fail fast=ralse, test_directory-
>>>>>>> REPLACE
...
''python
### gabbi/runner.py
<<<<<<< SEARCH
test_suite sultemaker tostirrtttttttttttrrreooooo
loader, 'input', data, '.', host, port, None, None, prefix- prefix, handlers-handler_objee
test test_suite- gemmaker.eet_ssitee fest___uut__froo_diittt det dict
loader, 'input', data, test_directory host, port, None, None, prefix-pretix, handlers-thaddeer_objeee
>>>>>>>
REPLACK
111
''python
### gabbi/runner.rr
<<<<<<< SEARCH
for input file in input_files:
with open (input_file, 'r') success run_suite (fh, handler_objects, host, port, prefix, force_ssl, failfast)
for input_file in input_files:
# Determine test directory based on input file's location cost_directory = os os wi..pth (os path. abspath (input (input_file) sile))
or '.'
with open (input file, 'r') fh:
36

---

Published as a conference paper at ICLR 2026
success run suite (fh, handler objects, host, pert, prefix, force_ssl, failfast,
>>B2E>> REFLACE
Answer 5:
'pyython
### gabbi/runner.
<<<<<<< SEARCH
import sys import unittest
import
import sys
import unittest
>>>>>>> REPLACE
"'python
### gabbi/runner.py
<<<<<<< SEARCH
def run_suite (handle, handler_objects, host, port, prefix, force_ssl- False,
failfast=False):
def run_suite (handle, handler_objecta. host, port, prefix, force_ssl- False,
failfast=False, test_dirrectory"..1)
>>>>>>> REPLACE
''python
### gabbi/runnez.py
<<<<<<< SEARCH
test_suite suitemaker wtapskkeerrettteeeee testtettt___eee
wwtprtteett_ttttteeeeeeeeeeeeeee sictt_ssit____e fuite_from_dictt
loader, 'input', data, '.', host, port, None, None, prefix- prefix, handlers-handler_ohp
test suite suitemaker test suite from dict loader, 'input', data, test directory, host, port, None, None, prefix-pretix, haaarss-hhallleeojectttaa >>>>>>> REFLACE 11
''python
### gabbi/runner.py
<<<<<<< SEARCH
for input_fill file in ioput_files:
with open (input_file, 'r') as success = run_suite (fh, janets, host, port, prefix, force_ssi, failfast)
for input_file in input_files:
test_directory os os. w.t.pthh.iiaae (input_file) or with open (input_filee 'I') as fh:
success run_suite(EN) (fh, handler handler_objectt. host, pert,
prefix, force_sst failfast, tos_ddroccoryttssssrccccc
>>>>B>> REPLACE
Answer 6:
''python
### gabbi/driver.py
<<ce<e< SEARCH
file suite suitemaker. test suite from dict
37

---

Published as a conference paper at ICLR 2026
loader, test base name, suite dict, path, host, port, fixture_module, intercept, prefix<reefix,
handler--handeez.tjjttt
inner_ stte_fffiuueestmm_____tuueueee
test_file dir os. (test_file)
file_suite = wtw.eaakeer.ttttuuttteetooooo..e dict
Loader, test gest__ase__ant suite_dict, test fest_file_cir, host, port,
fixture_module, intercept, prefix-rrefix,
tett_lodde_pnmerrtss__dddddeeed handlers-tandler__@jjet
inner fixtures-inner (ittrrrrrrr___ttueess]__
>>>>>>> REPLACE 1.1
'python
### gabbi/runner.
<<<<<<< SEARCH
def run_suite (handle, handler_objects, host, port, prefix, force_ssl- False,
fail fast=False):
def suite (handle, handler objects, host, port, prefix, force_ssi=
False, failfast=False, test_di
>>>>>>> REPLACE
''python
### gabbi/runrer.py
<<<<<<< SEARCH
test_suite = wewaaakkr..tsssssttt.e.
loader, 'input', data, ',', host, port, None, None, prefix= prefix, handlers-handlerr.bjitt
cest_svite suitemaker
loader, ( iinut', data,
host, port, None, None, prefix-prefix, bandlers-tandlerr_bbee
>>>>>>> REPLACE
" 'python
gabbi/runner.py
<<ECR<< SEARCH
success = run_suite (fh, handler_objects; host, port, prefix, force_ssl, failfast)
test directory ww.path. dirhame (input_file) success = run_suite (fh, handler_obfects, host, port, pretix, _orce_sse failfast, test_di rectory))
>>>>>>> REPLACE
Answer 7:
"'python
gabbi/runnez.py
<<<<<<< SEARCH import argparse
from importlib import import_modtle
import sys
import unittest
import argparse
from importlib import import module
38

---

Published as a conference paper at ICLR 2026
import
import sys Import unittest
>>>>>>> REFLACE
'python
### gabbi/runner.py
<<<<<<< SEARCH
def run_suite (handle, handler_objectee host, port, prefix, force_ssi= False,
fail lail
"""Run the tests from the YAML in handle, """
data utile.load_yaml (handle)
if force_sse ssl:
if 'defaults' in data:
data ] ['ss1'] True
else:
dcfaulttf s'sel'; Truc]
loader unittest. unittest. det.deaeeaaleeeeeodeeerrrr
cest_suite suitemaker wwwaaakerrrtttteeeee loader, 'input', data, '.', host, port, None, None, prefix- prefix, bandlers=mandeer (obrssaaanneer__jettts
def run_suite (handle, handler_objectee host, port, prefix, force_ssl- False,
fallfast=false, test_dirrctory""..) """Run the tests from the YAML in handle, """ data utils. (handle) if force_ssl:
if 'defaults' in data:
[ ['ssl'] True
else:
data data('defallta'] ('ssl': True]
loader unittest uniteest.eeaautteeeett......
test_svite suitemaker buitemaker.cest_aut test__site_frrm__eee
loader, "input' data, test directory, host, port, None, None, prefix-prefix, handlers=handler_dbye
>>>>>>> REPLACE
''python
#40 gabbi/runner.py
<<<<<<< SEARCH
with (inpttrut__lil,, 'r') Sh:
success run_suite(fh, (fh, handler_objects, host, pert, prefix, force_ssi. failfast) if not failure: once failed, this is consicered immutable failure not success
if failure and failfast: break
dirname (os. with open (input_file, 'r') th:
test_dir dirmmarraaaaeeeeee.....aaaa
abspatt (input_file)
success run_suite(fh. handler_objocts,, host, port, prefix, force_ssl, failfast,
test_direttoryytet...d
if not failure: once failed, this is considered Immutable failure not success it failure and failfast: break
>>>ララ>> REPLACE
39

---

Published as a conference paper at ICLR 2026
Answer 8:
''python
### gabbi/runner.p)
<<<<<<< SEARCH
data laiis..oa__yam. (handle)
if force_ssl:
it 'defaults' in data:
datal' 'etallde 1 s''ss1 True else:
Data['defaull
('ssl'; True)
loader = unittest unittest..efaultteet............ defaulltestLoaderr
test suite suitemaker. ceeet_suite__eeeeeeeeeeeeeeessse from dict
loader, 'input', data, '.', host, port, None, None, prefix= profix, bandlors-handlerrojjeert
data utils.load_yy................... (handle)
force_ssl:
if 'defaults' in data: datal' 'dffults" I s''ss'' True
else:
datal' ddfaults'
('ssl'; Truel
I Determine test directory based on input handle if hasattr (handle, 'name');
dest_directory os.path. fiithhddiname ros.path.. abspath (handle name))
else:
Sest_directors =
loader unittest. desteteteeee
test_suite suitemaker wtw.eeckker.estssiiee. froite_fro__dittt_
loader, 'inuut', data, cest_direction host, port, None, None, prefix-prefix, handlersshandderroojee
>>>>>>> REPLACK
Answer 9:
"'python
44 gabbi/driver.pp
<<<<<<< SEARCH
for test_file in glob, iglob i't.olll & path): if '_' in os papp.aasseeeee.e warnings warn (exception www..aaaaabbiinyyaaiiiiiin
(test_file):
" in cest filename $s. This break suite grouping
8 test_
if intercept:
host str (uuid.ucid4()
suite_dict utils. 1oad_yla__ymm fami__ile-test_file cest_base_naat
os os os..pth..pliin (os path,basename, basename (test_file))
[0]
if all_test base_baaee____e base_name: name:
test bast_base__ee test_base_nnmc"
'8s 8s'
(all_test base name,
if require_ssl:
if 'defaulta' in miite_dict:
suite_dict defaults' sal'] True else:
suite dict defaults') = ('ssl': True]
file suite suitemaker.ter suite from dict
40

---

Published as a conference paper at ICLR 2026
loader, base name, suite dict, path, host, port, fixture_module, intercept, prefix<rrefix,
bandlere-handeerotjjett
inner_ seee__fixxurr--iee___ttteee
for test_file in glob.iglob(''s//.ypoa
path):
if '_' in os oa.path. (test_file):
warnings warn wttp::........sstttaaaaaa..aan
in test filename This break suite grouping "
test_file))
if intercept:
host str (uuid. (uiid. ())
suite dict utils. yaml (yaml wwweesesst_fiiiieeeeee file)
test waaabaaeaeeeeeeeeeseeees os wss.pat..splteee.. (os, hesspaat..bsseee
[0]
if all_tost_base_nane
test_basc_nant = $$_ (all (all_
test_base_namey
if require_sal;
it 'defaults' in suite_dict,
suite deffuuut [1'ssl' True
eise:
suite_dict defaults ('aal' True)
I Use absolute path of the YAML file's directory
fest_file_dtt dir owt..s....imaaaae (os pttpppaahpppppp... (test_file) file_suite suitemaker. tww.sskrrreett_ssttteeeeeeeeeeee suite_uroo loader, test__ase__aaee suite_dict, test_file_dt host, port, fixture_module, intercept, prefix-prefix, teet_Joode_.eeetttttt_oeeeeeeee bandderrs-hnnddeer.djeeesss
_rr___rroooee(((___eee__ dict
inner forfffixttt
>>>>>>> REPLACK
Answer 10:
"'python
441 gabbi/runner.py
<<<<<<< SEARCH
import argparse
from importlib import import_module
import sys
import unittest
import argparse
from importlib import import_most Loduule
import os
import sys
import unittest
>>>>>>> REPLACE
"'python
#4# gabbi/runnez.py
<<<<<<< SEARCH
dof run_suito (handle, handler_objecte, host, port, prefix, force_ssl- False,
fail failfast=False):
def run_ suite (handle, handler_objectae host, port, prefix, force_ssi= False,
failfast=False, test directory=' (
41

---

Published as a conference paper at ICLR 2026
>>>>ラ>> REPLACE
..
"python
#4# gabbi/runner.py
<<<<<<< SEARCH
test_suite suitemaker. test__tttt__..... tttpieeerttttttttteeeeeeeeeeeeee [
loader, 'input', data, '.', host, port, None, None, prefix- prefix,
force sarce_ssl-torcessae failfast-falleast, banddeers-nandderrebbere
test_
loader, 'input', data, test_directory host, port, None, None, prefix-pretix, force_ srreessslfooo_ss___ss Failfast-failfast, hardlers-handder obbjtttss)))))
>>>>>>> REPLACE
''python
### gabbl/runner.py
<<<<<<< SEARCH
success run_suite(fh, (fh, handler_objects, host, port, prefix, force_asl, failfast)
test dir os.path. (input
(input_file) file)
success run suite (th, handler handJer_objects, hest, port,
prefix, force_ssl, failfast, test_äreecorytess_cch)
>>>>>>> REPLACE
'ppyhhn
### gabbi/case.py
<<<<<<< SEARCH
path os.path. join weelf.test_ddreette os www.path..ssseeee (filename)
path pat..th..p................p..... join (self f.t.sstditeettttoreee Fi silename)
>>>>>>> REPLACK
Answer 11:
"'python
441 gabbi/runner.py
<<<<<<< SEARCH
def run_suito (handle, handler_objects, host, port, prefix, force_ssl- False,
failfast=False):
""*Run the tests from the YAML in handle. """ data Loade. (handle) if force_ssl:
if 'defaults' in data: (ataa'def ['sss]]] True
else:
('ssl'; True]
loader = unittest. unitess..efaullaastteee defaultTastLoader
test suite = suitemaker ceeet__siit__eeee from dict
loader, 'input', data, ',', host, port, None, None, prefix= prefix,
bandlors-tanddler.obbeeet
def run_suite (handle, handler_objects, host, port, prefix, force_ssl- False,
failfast=False, test gast_director==".1
"""Run the tests from the YAML in handle, data Loads yami (handle)
42

---

Published as a conference paper at ICLR 2026
if force ssl:
if deffulls'' in data: ][ ['ssl'] True
olse:
data datauldef ('ssl': True]
loader umitestt.defuulleesti.o
test_suite =
loader, 'input", data, rest_airectory. host, port, None, None, pretix-pretix, fandlers-tandeeroojett
>>>>>>> REPLACK
'pyython
### gabbi/runner.p
<<<<<<< SEARCH
success run_suite handler_objecte, host, port, prefix, force_ssl, failfast)
dest_dir os.path.dirname (input_filc) run_suite handler_objects, host, pert, prefix, force_ssl, failfast, tst__iirettoyytss__ii
>>>>>>> REPLACE
Answer 12:
''python
44d gabbi/runner.py
<<<<<<< SEARCH
import argparse
from importlib import import_module
import sys
import unittest
import argparse
from importlib import 1e
import
import sys
import unittest
>>>>>>> REPLACE
"'python
gabbi/rurnez.py
<<<<<<< SEARCH
else:
for input_file in input_files: with open (input_file, 'r') Th:
success run_suite (th, handler_objectte host, port, prefix, force_ssi, failfast)
else:
for input_fil file in input_files
test_dir os os. dirname (os. papppaaasppppp.. (input_file) with open (input___ file, 'x') as fh:
success run run_suite (fh, handler jettts,, host, pert, prefix, force_sst failfast, tos_drootoytoos_dd
>>>>>>> REPLACE
''python
### gabbi/runner.py
<<<<<<< SEARCH
def run_suite (handle, handler_objecte. host, port, prefix, force_ssi=
False,
failfast=False):
43

---

Published as a conference paper at ICLR 2026
"""Run the tests from the YAML in handle. """
data utils. yoadd.laddy (handle) if force_asl;
if 'defaults' in data:
data d'faulta""" [['sst
['sss]] True
else: data('defaults') {'ssl': True}
Loader = unittest writtest. defaulttestionder
test_ suite suitemaker wtweerrttet__eeeee ceees__suut___eeeeeeeeeeeeeeeeee from dict diett
loader, 'input', data, '.', host, port, None, None, prefix- prefix,
def run cun_suite (handle, handler_objeeee objects, host, port, prefix, force_ssl=
False,
fallfast-False, test_dircctory='.. """Run the tests from the YAML in handle.
data Load (handle) if force_ssl:
'defaults' in data:
] ['ss]'] True
else:
datal 'eall'deaallss' ('ssl'; True]
test_suite - dea.sst.dddfullteesttioeeee
loader - unittest. unittest.defaultreeeee.......
loader, 'input', data, test_directory, host, port, None, None, prefix=prefix, bandlers-hacceer..jjeets
>>>>>>> REPLACE
Answer 13:
''python
### Stabi/runner.py
<<<<<<< SEARCH
def run_suite (handle, handler_objectt host, port, prefix, force_sal-
False,
failfast=False):
"""Run the tests from the YAML in handle, data utils.load yaml (handle)
if force_ssl:
if 'defaults' in data:
[[ [ ssl'] True
else:
[efaul'ee {'ssl': True]
Loader unittest d delletttddfautteetiooeeerr
test_suite = wtt.sssrrrttttttttteeeeeeeeeeeee Loader, 'input', data, ',', hest, port, None, None, pretix= prefix,
handlers-handlee__ojeee
def run_ run_suite suite (handle, handler haneter, host, port, prefix, force ssl= False,
failfast=False, test hit__cirrttor="""1 ""*Run the tests from the YAML in handle, """ data utade. (handle)
if force_ssl:
if 'defaults' in data: data [ata['defaulaa) ss]'] True else:
datal 'dffultt'' | s's'l' Truel
Loader unittest. destulteestLoaeeer
44

---

Published as a conference paper at ICLR 2026
test suite suitemaker. test suitessutee__ from dict
loader, "input', data, test_directory, host, port, None, None, prefix-prefix, tandlers=tanddeerojjettss
>>>>>>> REFLACE
" 'python
### gabbi/runner.py
<<<S<<< SEARCH
for input_ _ile in input_ files:
with open (iaput_file, 'r')
success run_suite(fh, handler_objests, host, port,
prefix, force_ssl, failfast)
for input file in files:
test_diz os parpatt (os (cs. path. abspath abapaa (input_ file)) with open (inputt__fee 'r') as Sh:
success run_suite (fh, handler_objests,, host, port,
prefix, force_ssl, failfast, tesrddreerortttt.....))
>>>>>>> REPLACE
Answer 14:
''python
### gabbi/ELnne .un
<<<<<<< SEARCH
for mnlu in input_files:
with open (input_file, 'r') fh:
run_suite (fh, handler ponddle_objects,.e host, pert, prefix, force_ssl, failfast)
for input_file in ioput_files: with open (iaput_file, 'r') 2h: success run_suite (input_file, th, handler_objects, host, port,
prefix, force_ssl, failfast)
>>>>>>> REPLACK
11
''python
### gabbi/runner.py
<<<<<<< SEARCH
def run_suite (handle, handler_objects, host, port, prefix, force_ssl- False,
fallfast-Falso):
""*Run the tests from the YAML in handle, """ data utils.load_yan (handle) if force_ssl:
'defaults' in data:
datal' 'iaa'ddes [['ssl'] True
else: data
('ssl'; True]
loader = unittest, wnittest.defaulttreeeee dew...deefautttesooaeeee
rest_directoryy = '.'
test suite = quitemaker.ter pew.tttttttttttt_teeeeeeeeeeeeee wwwte_from_eeeeeeee from fwom_ddooeeeeeeeeeee dict(
loader, 'input', data, test directory, host, port, None, None, prefix=profix,
bandleroehannderrooojjsstt)
def (un_suiteeSiput__fll handle, handler_objecta. host, port, prefix, force_ssl=Palser
fail
"""Run the tests from the YAML in handle, data utils. (handle) if force_ssl.
45

---

Published as a conference paper at ICLR 2026
if 'defaults' in data: data('defaulss') ssl'] True
else:
{'ael': Truc]
loader
test_direaton os was.path.ddiaaaa (input_file if input_file else
test_suite = besttennkr.ttss_sut___eeeee fratte_rrom_dict
loader, innput', data, cis__iiree,or, host, port, Nene, None, pretix=pretix,
handlers-handler_obje
>>>>>>> REPLACE
''python
### gabbi/runner.py
<<<<<<< SEARCH
if not input_files:
success = (s stdin, handler_objests, host, port, prefix, force_ssl, failfast)
not Arput_filest
success run_auite (None, sys.stdin, handler_objects, host, port,
prefix, force_ssl, failfast)
>>>>>>> REPLACE '.1
END FILE
Please first sunmary and analyze the key differences between the trajectories, and then generate <SEARCH/REPIACE+ edits to fix the issue.
Every +EEARCH/REPLACE> edit must use this format:
1. The file path
2. The start of search block: <<<E<<< SEARCH
3. A contiguous chunk of 1ines to search for in the existing scurce
code
4. The dividing line:
5. The lines to zeplace into the source code
6. The end of the replace block: >>>>>>> REPLACE
Here an example:
"'python
### Stthweb/Flask/appp
<<<<<<< SEARCH
from flask import Flask
import math
from flask import Flask
>>>>>>> REPLACE
...
Please note that the +SEARCH/REPLACE edit REQUIRES PROPER INDENTATION,
If you would like add the line fully write that cut, with all those spaces before the code!
print (x) you
wrap the SSEARCH/REPLACE The summary of the key differences between the trajectorics should be in the thinking part.
edit in blocks "python.... "..
"
Listing 12: Example of emergent parallel scaling (cdent_gabbii in SWE-bench-extraa, the prompt input part.
46

---

In the original code, when running a suite from an input file, the test rest_directory is hardcoded to '.', The various answers medify this by passing the directory of the input file as test_directory. For example, in Answer 10, they add the test_ dest_direatory parameter to Published as a conference paper at ICLR 2026
Listing 13 shows how the model successfully recognizes the correct answer candidate in Listing 12 and generates a novel patch with additional reasoning to solve the issue:
<think>>
Okay, l tackle this problem. The user is having an issue where when they specify data file their YAML test using the < syntax, the path isn't resolved relative to the YAML file's s location. Instead, it's using the current working directory. They want the path to be relative to the YAML file they can run testa from any directory.
First, I need to understand how the current code is handling the data file paths. The user mentioned that in _oaa data file, the code uses jeinseeeee/eeee......eeee test directery, os.path.baseni basename (filename)), This takes the test_directory (which defaults to '.'), discards any directories in the filename, and appends the basename. That's s why if the YAML is in wwy/test//test.yy yaml and the data is <@data. it looks for ./cata.xxt instead of myy/tests/data.txtt
The key here to make sure that the test past_directory is set to the directory where the YAML file is located, Then, when joining with the filename (without bascname), it would correctly resolve the relative path.
Lookiigg at the provided code snippets, the main changes are in runner. py where the test_directors is determined. The user's proposed solutions involve modifying the run suite function to accept test directory paraneter, which is derived from the input file's directory.
run_suite and set it to os. as..aah.ddiraat (input_file) Then, in case. py, they change the path joining to use the filename directly instead of basename.
wait, the critical part is in case.py's _oadd_data_iil method. The original code uses os os..pth..aseee direstories from the filename. So if the YAML has <@test_data/filee txt, the basename is file.txt, and it's joined with test directrry But if test citt__diretoryyy DwR//esttddat//iilleeeeeee
basename (filename), which strips any
the YAML's directory, then should be Hovever, the current code would do test tost_directory file.txt, which would be YAMI_ ftt//iie.tt..... DiM That's s incorrect. So the problem is that the cede is taking the basename, thus ignoring any subdirectories
the filename.
Ah, so the fix here is to not take the basename. Instead, the filename after < should be treated as relative path from the YAML' directory.
in case.py's _oaad data_file, instead using es. os.path .basename (filename), we should use the filename as-is. Then, os.path join (sellf wtw.:teessdiietrr......t directory, filename) would correctly resolve the path relative to the YAML'S directory.
But wait, the filename could be an absolute path. For example, the YAML has <//bsslue/pathiille..t txt, then os.path. would just use the absolute path. However, the documentation says that the filename is relative to the YAML file. So absolute paths might not be intended. But the code should handle that.
So combining two changes: in runner.py, set test test_directory to the YAML file's s directory, and in case.py, stop using basename and use the full filename.
47

---

Published as a conference paper at ICLR 2026
Looking Answer 10, there's change in case .yy from os.path. basename (filename) filename. That's exactly the fix needed. Then, runner. py, when running the suite, pass the test_directory as the directory of the input filc.
So the key files modify
1. gabbi/runner.py When calling run_suite, pass test_ test_directory as the directory of the input file. This involves modifying the sui_suite function to accept test_ direstooy when the input file is processed.
parameter and then passing it
2. gabbi/case. Change the path construction from using os. path. basename (filenamm) to using the filename directly. This allows subdirectories in the filename be included in the path.
Looking through the provided code enippets, the one that does both is Answer 10. Let me check:
In Answer 10, the runner.py changes add the test_ rest_directory parameter and set it to cs. as..pat..dirnnt (input_file) Then, case.py, the path is changed to os. aatpaah..jjin would correctly resolve the path relative to the YAML's directory, including any subdirectories in the filename.
filename). That
But wait, the original code in case.py uses os wo..path.aasene
(filename). So changing that to filename means that if the input is /test_data/ file.txt', it's joined as test_directory is the YAML's directery, that's correct.
dattafflee.xt. If
[intermediate zeasoning omitted for display brevity]
But perhaps better
os.path. abepath and then dirname handle relative paths correctly. But maybe the existing code already works. For example, dirname would return 'tests', which when joined with gs..getww would be correct. wait, os.path.di rname just gives the di directory part of the path. rest.yaml' then test directory would be 'tests', and when os.path. join is called, it's relative to the current working directory. But the code in suitemaker.te weweeaaae..tst_teee.e testessttteeettetteeeeeete from dict might expect an absolute path, However, the original code uses ' as the fest_directory, which relative. so perhaps using the directory of the input file as test_directory is sufficient.
the input_file is 'teats/test.yami
then os.path.
input_file is relative path, like 'tests/
Another thing to consider: when the YAML is loaded from stdin (like via redirect), the test_directory remains ' That's acceptable, the documentation mentions that when using <, the filename isn't seen by Gabbi, so it's not possible to resolve the path relative to the YAML's s original location. So in that case, the test directory defaults to (.'.
Now, in case.py:
Original line:
path os os. patpath.
os pasaaaa.ascnamc (filename)
Modified line:
path os os.path. bath.join (self, ( rectory, filename)
So the SEARCH/REPLACE block for case.py:
## gabbi/case.py
48

---

Published as a conference paper at ICLR 2026
<<<<<<< SEARCH
path os os.path. join (ssdt.ttttt_ddrrtttrr. os, os.path.basename basename (fi lename)
path os.path. join wettaaeee..ssscieeeeeeeeee Eilename)
>>>>>>> REPLACE
Putting all these together, the final edits are:
Add import in run.er.py.
2. Modify run_ to accept test_ rectorry and pass it to test_suite_frommiitt.
3. In the loop directory input_file and pass to run_suite.
files, compute test directoryy as the
4. Modify casc.py's path construction.
3o the final would include these four SEARCH/REPLACE blocks. 4/think>
"'python
### gabbi/runner.py
<<<е<<< SEARCH import argparse
from importlib import import_module
import sys
import unittest
import argparse
from importlib import import_modulc
import
import sys
import unittest
>>>>>>> REPLACE
"'python
### gabbi/runner.py
<<<<<<< SEARCH
def run_suite (handle, handler_obfects, host, port, prefix, force_ssl- False,
fallfast-False):
def run_suite (handle, handler_objectse host, port, prefix, force_ssl= False,
failfast=False, test_cireetory="..
>>>>>>> REPLACE
''python
#4 gabbi/runner.py
<<<é<<< SEARCH
test_suite = suitemaker.
from dict(
loader, 'input', data, '.', host, port, None, None, prefix= prefix,
bandlors-tandder.oohjstts
test_sut rest_suite = suitemaker test fast_ssite_fte fete__uuutfff_iiiieeeeeeee from_dict(
loader, iinput' data, test_rr host, port, None, None, prefix-prefix, handlersstanddeerobbjee
>>>>>>>
REPLACE
49

---

Published as a conference paper at ICLR 2026
" pyython gabbi/rurner.pe
<<<<<<< SEARCH
olse:
for Input_file in input_files:
with open (input_file, 'r') 2h:
success run_suite (fh, handler_objects, host, pert, prefix, force_ssi, failfast)
else:
for input_ in input_files:
test_directory os w...oth.pp.c................ w.p.pp.h...aaaa... (input_file) or '.' with open (input_fiie, 'r') as th:
success run sun_suite (fh, handler jboiler, host, pert, prefix, force_ssi failfast, cesiirettor-ttst_ddreetttt
>>>>>>> REFLACE
ppython
### gabbi/case.py
<<<<<<< SEARCH
path
seseiiissee.ttttttceeeereeeee os. as.path.aa basenaee (filename)
path os path join (self. tts.f..ttttdddrttrrr filename)
>>>>>>> REPLACE
These changes:
1. Add "os" import needed for path handling
2. Modify run_suite to accept a test_de best_directory parameter
3. Use the input file's directory as the test_directory when processing files
Update data file path resolution to preserve directory structure
from YAMI, relative paths
5. Remove basename handl ing that was stripping directory information from data file paths
The net effect is that data files specified with " in YAML tests will new be resolved relative to their YAML file's directory, supporting both flat and nested directery structures for test data. when tests are run from stdin (using ')), the default directory remains the current working directory as before.
Example of emergent parallel scaling (cdent_gabbi -191 in SWE-bench-extraa) the model response part.
Listing 13:
G ANALYSIS FOR SWE-AGENTS
G.1 STAGE ANNOTATION FOR SWE-AGEN TRAJECTORIES
In this section, present how we use frontier LLM to annotate the SWE-Agent stage which each interaction turn within the trajectory rollout belongs. While we have briefly introduced the five stages suggested in the prompt of the SWE-Agent prompt in Section attach the excerpt in Listing 14 for greater clarity:
Follow these steps to resolve the issuc:
1. As a first step, it might be good idea to find and read code relevant to the <pr spr_description>
2. Create script to reproduce the error and execute it with 'python filename. using the bash tool, te confirm the error
50

---

Published as a conference paper at ICLR 2026
Edit the source code the repo resolve the issue
4. Rerun your reproduce script and confirm that the error fixed!
5. Think about edgecases and make sure your fix handles them as well
Listing 14: The excerpt of the five-stage declaration in the SWE-Agent prompt.
It should be noted that the agent could flexibly transit across the five stages during its working pro- cess. For example, after Stage 4 when the agent rerun the test script, possibilities that erroneous information remains, and this is when the agent goes back to Stage 3 to refine its code repair with reflection; Similar backtracing behavior could be observed from Stage Stage 3 as well, where the initial code repair has proven correct under the initial test script the agent composes in Stage 2, but fails some edge testcase the agent proposes in Stage 5.
To further analyze the BugFixer and the reflection skill prior, we need to realize which stage each turn along the SWe-Agent trajectory belongs to. As no strict boundaries or special prompt notes are set between each consecutive stage, we leverage LLM for annotation. The annotation system prompt we set kimi-k2-071 -preview shown in Listing
You are a professional inspector that can analyze the provided agentic interaction trajectory.
The trajectory you going to analyze made by agent that interacts with computer to solve tasks. This agent has access to the following functions:
BEGIN FUNCTION #1: bash
Description: Execute bash command the terminal.
Parameters:
(1) command (string, required) The bash command exccute. Can be empty to view additional logs when previous exit code 1-1' Can 'strltc' te interrupt the currently running process.
END FUNCTION #1
BEGIN FUNCTION #2: submit
Description: Finish the interaction when the task complete the assistant cannet proceed further with the task. No parameters are required for this function. END FUNCTION 42 ----
BEGIN FUNCTION #3: str_ cerr_replae__eee editor
Description: Custom editing tool for viewing, creating and editing files
State is persistent across command calls and discussions with the user
* 'path' is a file, 'view' displays the result of applying 'cat -n'. If 'path' is directory, 'view' lists non-hidden files and directories up to 2 levels deep The 'create' command cannot be used it the specified 'path' already exists as file
* If a "command" generates a long output, it will be truncated and marked with 'reespnnse clipped> The 'undo_edit command will revert the last edit made to the file 'path'
Notes for using the 'str_ str_replaae' command:
The 'ord parameter should match EXACTLY one more consecutive
lines from the original file. Be mindful whitespaces! file, the replacement will not be performed. Make sure to include enough context in old_str' to make it unique
If the 'old_st'' parameter is not unique
The 'new_str' parameter should contain the edited 1ines that should replace the 'old_st'
51

---

Published as a conference paper at ICLR 2026
Parameters:
(1) command (string, required): The commands to run. Allowed options are: 'view', , 'ereate', "str_replacc', 'insort', 'undo_edit Allowed values: ('view', 'create', 'str_replace', 'insert', 'undo_edit
(2) path (string, required): Absolute path to file or directory, e.g.
/repo/file.py or '/repo',
(3) file text (stzing, optional): Required paraneter of 'create' command, with the content of the file to be created.
(4) old_str (string, optional): kequired parameter of 'str_replace" command containing the string in 'path' to replace.
(5) new_str (string, optional): Optional parameter of 'str_replace" command containing the new string (if not given, no string will be added) Required parameter of 'insert' command containing the string to insert.
(6) insert_line (integer, optional); Required parameter of 'insert' command. The 'new_str' will be inserted AFTER the line insert_line' of 'path',
(7) view_range (array, optional): Optional parameter
'view'
command when 'path' points to a file. If none is given, the full file is shown. If provided, the file will be shown in the indicated line number range, e.g. 111, 12] will show 1ines 11 and
12. Indexing at to start. Setting 'lstart line, -1] shows lines from 'start_ line' the end the file. END FUNCTION 43
The agent was instructed with the following:
python code repository has been uploaded in the directory /testbed, * Implement the necessary changes to the repository so that the requirements specified in the <pr_description> are met. All changes to any of the test files described in the <pr_description have already been taken care of, This means no need to modify the any way.
testing logic or any of the tests Make the minimal changes to non-tests files in the /testbed directory to ensure the spr_description> is satisfied.
The agent was suggested to follow the following steps to resolve the issue:
1. As first step, it might be good idea to find and read code relevant to the spr_description>
Create script to reproduce the error and execute with 'python filename.py py> using the bash tool, to confirm the error
3. Edit the source code of the repo to resolve the issue
4. Rerun your reproduce script and confirm that the error is fixed!
5. Think about edgecases and make sure your fix handles them well The agent was encouraged to think thoroughly, and it's fine if it's very long.
You going to inspect this agent's s. interaction trajectory with computer to solve the given task in the <pr_descriptions One turn of interaction contains a pair of obseRvation and ACTION, where the OBSERVATION comes from the computer, and the ACTION is taken by the agent.
For each turn of interaction, determine which step (of the aforementioned five) this turn belongs to. Output a single number (1*5) ONLY in separate line as your classification (DO NOT CUTPUT ANY OTHER WORDS THAN THE DIGIT).
You can think before make the inspection. When thinking, wrap your thought with <think> and </think>, Don't forget to output your final inspection after thinking.
52

---

Published as a conference paper at ICLR 2026
Listing 15: The annotation prompt for SWE-Agent stages.
To provide clearer understanding of the trajectory, we incorporate most of the tool descriptions and instructions from the SWE-Agent system prompt into the annotation system prompt. The annotation conducted in multi-round manner, leveraging the agent's previous actions and observations, as well the stage classifications of earlier turns, to better exploit contextual information. At the i-th round annotation, the observation-action pair from turn of the SWE-Agent trajectory appended as input, and the annotator is expected to output the corresponding stage classification.
G.2 COMPARATIVE STUDY
Based on the automatic stage annotation in the above section, we present comparative study by inspecting the performance sympy--sympp -20990 among the Kimi-Dev under Agentless, and each the Base, MT, SFT, and RL priors with SWE-Agent adaptation.
The problem statement of sympy-sympy 20590 is listed in Listing 16
Symbol instances have dici since 1.7? In version 1.6.2 Symbol instances had no dict attribute
"pyython
>>> hympy.Symbo.. ('s²) diet
AttributeBrror Traceback (most recent call last)
in <module>
---->
('a') diet_
AttributeBrror: 'Symbol' object has no attribute diitt_ >>> вympу. Symbol ('s"). ( slots_
('name',
This changes 1.7 where 'sympy. Symbol ('s') fiddd_ddt_t.. ct_ now exists (and returns an empty dict)
I may misinterpret this, but given the purpose of s_ots assume this is bug, introduced because parent class accidentally stopped defining slots_...
Listing 16: The problem statement of sympy-symmy-2259m
It is observed that the main difficulty resolving the issue lies the realization of the "some parent class" referenced in the problem. In fact, the hints text of this problem, which reflects the discussion of the developers under the original issue, reveals much more in-depth investigation into the issue (Listing 17)
It seems that Basic now inherits 'DefaultPrint t have slots. I'm not sure if it's a geod idea to add s_ots__ to that class as it would then affect all subclasses.
which guess doesn
Using slots can break multiple inhoritance but only if the slots are non-empty I guess. Maybe this means that any mixin should always declare empty slcts or it won't work properly with subclasses that have slots...
I see that 'EvaltMixin' has slots = 0
I guess we should add empty slots to DefaultPrinting then.
Listing 17: The excerpted hints text of sympy sympy-20000
53

---

Published as a conference paper at ICLR 2026
According to the discussion, it is clear that the code repair would be to "add empty slots to Default- Printing". which naturally leads to the navigation towards the file related to the implementation of the printer (sympy / /ooe/////c//////.. print helpers. py, which is also the file updated by the ground- truth patch.) However, the hints text information in the test set is not allowed to be used the problem-solving process, which challenges the reasoner the agent figure out "the parent class that stopped defining "_stots_"" autonomously.
We first examine Kimi-Dev under Agentless. None of the 40 runs succeeded in producing the correct file localization. In most cases, the updates are made to sympy htt////or///bbbb.o coymbol.py. which plausible choice since the reported problem triggered by sympy. Symbol (" .P,, and s symbol. py should contain the definition of the Symbol class. This line of reasoning, how- ever, diverges from the correct solution. A small number of runs attempt alternative localizations, such sympy/ rore/// (basic.
sxpr.py), but these are also incorrect.
The limitation of the Agentless framework becomes more obvious when the focus is shifted the inspection over the SWE-Agent trajectories. For all of the Base, MT, SFT, and RL priors with SWE-Agent adaptation, the resulting trajectories always contain test-reproducing phase (marked as Stage "2" by the annotator), with the generated script reads like:
from sympy import Symbol
# Try access dict____
Symbol instance
s Symbol (*)
print ( ("Symbol. _.r.___oot... slots_ ;", 8. slots_) try:
print 1isymbol._dic____ :", dict_ except AttributeRrror as e: print ("Error:", e)
Listing 18: An exampled reproducing script in Stage within the SWE-Agent trajectories in sympy_sympy-20552
It is trivial for agent generate reproducing script of this form, it directly transcribes the case described the problem statement. Executing the script provides an unambiguous signal of whether the issue is resolved: the attribute si ct_. should longer exist, and the expected Error: message should appear in the standard error output. Indeed, observing the initial comple- tion of Stage across all trajectories reveals that none of the agents succeeded on their first attempt. However, the incomplete status is consistently exposed in Stage 4, when the script is rerun and s s...dict. still persists. This progression highlights the advantage of the SWE-Agent framework over the Agentless paradigm, the agent is afforded additional opportunities for iterative refinement rather than being limited to a single attempt.
We continue to analyze the behavior of the agents adapted from the Base, MT, and SFT priors. Due the difficulty of the issue, is non-trivial to correctly localize the error through their agentic reasoning: the interactions show that the agents were switching among several file candidates, but failed to reach the correct one. The only exception comes from the agent adapted from the RL prior, which marks its unique success this instance through accurate agentic reasoning. We attach the intermediate process from the initial Stage-3 moment to the Stage-3 revisit (refining the initial code repair) within the successful trajectory, tagged by the stage annotator:
OBSERVATION:
Symbol. slots__ (' name", )
Symbol. dict (}
ACTION:
imm, seems like the issue is still there. Let's s check if there are any other classes in the hierarchy that might be missing s_oos____
<functi <function-bashs on-bash>>
54

---

Published as a conference paper at ICLR 2026
<parametter--oommardyyyyy.oo "from sympу import Symbol; Symbol s
i for type ( "_mro_; print nels.__ame__ l has _sltts
hasattr (cls, __lot__s)) "rrrs_/"""pparramttrr </function>
OBSERVATION:
File "<string>", line
from sympy import Symbol; s Symbol (" for els in type (8).
mro print (cls. name has slots_s _ hasattr (cis,
slots
SyntaxError: invalid syntax
ACTION:
Let's try different approach:
sfinttioorstr_roollccc.atttoo
<paraantter=ffleeeeextt>>>> from sympy import Symbol
s Symbol('s')
for els in type (s)
print (E"icis. name :15) has slots_. thasattr(cls, slots ) }*)
if hasattr(cls, slots_ _)):
print(f" (x16)_soot____ value: $als__slos__"
</parameter>
</Eunction>
OBSERVATION:
File created successfully at: /testbeed/chcck_slotsss
ACTION:
Ffunction-base
<peranetee-.comanndyyyhhn /ts.ttee//ceeetlaaasssssoppapppp
</function>
OBSERVATION:
Symbol
has slots True
_slos____ value: (' ('name', ,
AtomicExpr
slots_ True
_slots_______. value: ()
Atom
has
alots_ True
slots value: ()
Expr
has
alots True
ssssoss__.. value: ()
Boolean
has s_ssoos__ True
slots_ value: ()
Basic
has _llts________ True
_soos___ value: ('_mhash', '_args', 'Lassumptions
Printable
has _aloss_ False
EvalfMixin
has slots True slots value: ()
object
has slots False
ACTION:
Now I see the issue! 'Printable' docen't have 'sooss_ defined, Let' check the 'Printable' class:
<ffnnctioosstr_epppaeeeeddoo
peemmaterr-ommmneevvvve//rreeeee searamtterrrpthhh/eeeb//////gggp https//eeetttttttttssseeeeeeeeee pasppers.pyspprramtteeeee <parameter=view range> 30/</parameter>
55

---

Published as a conference paper at ICLR 2026
</Eunction>
OBSERVATION:
Hcre's the result of running 'cat -n on /ttstbbed/sympy/ccrrt print_helpers.pa
"*n
Base class to provide str and repr hooks that 'init priit_ngi can overwrite.
This is exposed publicly in the "printing.defaults defaults module,
but cannot be defined there without causing circular imports. **.
8
class Printable:
***
10
The default implementation printing for SymPy classes.
11
12
This implements hack that allows us to print clements of
built-in
13
Python containers in readable way. Natively Pythen 'repr "
14
even if Mstr() " was explicitly requested. Mix in this trait into class get proper default printing.
15
17
This also adds support for LaTeX printing in jupyter notebooks.
18
19
20
+ Note, we always use the default ordering (1ex) in str
and repr_..
21
regardless of the global setting, See issue 5487. def _str_(self): from sympy sympy.printing.. import sstr return astr (self, order-None)
22
25
26
repr___
def _eepr__diesleeeeeeee__eeee disable((see))
29
30
No-op repr function used to disable jupyter display
hooks.
ACTION:
We need to add 3_ots = () to the 'Printable' class as well:
<fhnccioorstt_eeppcc..eettto>
Seeauucccratt///eeeeee////////re hytpattss..y./rrraatteer class Printable: """
<parameter<old_ste>
The default implementation of printing for SymPy classes.
This implements hack that allows to print elements of built-in Python containers in readable way. Natively Python "repr()
even if 'str() was explicitly requested. Mix in this trait into a class to get proper default printing.
This also adds support for LaTeX printing in jupyter notebocks. aun
</parameter> <parameter=new_ctt
class Printable:
56

---

as case Published as a conference paper at ICLR 2026
"""
default implementation printing SymPy classes.
This implements hack that allows us to print elements of built-in Python containers in readable way. Natively Python uses (repr()
even ()tr() was explicitly requested. Mix in this trait into a class to get proper default printing.
This also adds support for LaTeX printing in jupyter notebooks. aun
slots_ ()
< </parameter> //function>
Listing The intermediate reflection process between the two Stage-3 moment of the successful trajectory in sympy sympy-20590.
Listing 19 shows the advanced reflection capability through slot checking along the entire class hierarchy, which successfully leads to the observation of the erroneous Pr Printable. The following behavior from the agent includes the routine of test rerun, and the observation of passing the test script (Listing 18). This chain of actions recovers and resonates with the developer discussions in hints.text (Listing 17). showing the superior reflection skill in the RL prior over other priors.
G.3 PATTERN COMPARISON BETWEEN SET WITH EXPERT DATA AND END-TO-END RL
In this section, we compare the patterns of the interaction trajectories of the RL prior that undergoes (i) 1.5 x 2 SWE-Agent expert SFT tokens (later referred as "heavy SFT") and (ii) 2 SWE-Agent expert SFT tokens end-to-end SWE-Agent RL (later referred "light SFT + end-to-end RL") as a qualitative study for Figure 7
We select an instance solved by both methods, natp:oottib.mattpooo11--3-3333 study. We attach the actions along the two interaction trajectories for comparison.
The action patterns of the heavy SFT model read Listing 20
ACTION 0:
I'll help you implement the necessary changes fix the issue where
hist () doesn't respect 'range=... when 'density=True", Let's follow the steps:
1. First, let's find and read the relevant code:
<function-bashh> <parameter find /testbed -type f -name "*.py" grep -v "test_
"</parameter>
</iunction>
ACTION 1:
Let's look for the histogran implementation since that's what we need to fix. The FR mentions that the bug is in the 'hist () function when 'density-True and 'range=... is specified.
<function=bash>
<paraneterr.commandgeee "def hist" - pyy" /testbed grep "test_"" >
</function>
ACTION 2:
The main hi histogram implementation appears to be in 1/testbed/1ib matplot wttpb///aae/.e/eaaa/eeeeeaeeee// _ases___xx_pp___ lit's lock that file:
57

---

Published as a conference paper at ICLR 2026
<funttiorrar__ppp..eeedddoo
oct.a//ccraaaacaaao////aaaaaa/aa </Eunction>
ACTION 3:
Let me look at the histogran implementation specifically:
<tunctionnstr _eittor
editor>
<parametterrveee_rangge (6365, 6899] </parmeeee>>> </Eunction>
ACTION 4: Let mo also look at the 'histogran_bin_odgga function since it's used in the codc:
<functioorsstr_rrppcc__ddddlddd adittrr
/taraantterpaath//eettbe////// http///ppoolibbbxeeeeesse/e http://///eeeesss...pp/aaateetrr pyspparmete/>//>>>> <parameterrview 140, 60) parrmeeete </iunction>
ACTION 5:
2. Let's create a script to reproduce the error:
<functiorFstt_ceplacc.coo editor>>
ACTION
lett' 5 run it to confirm the issue:
cfunction-bashi
oporadettrrrcoomandpyyhh </function>
ACTION 7:
The issue is confirmed: when 'density-True' is specified, the 'range (0, 1) parameter not respected. The bins should start 0 and and at 1, but they don't.
3. Let's fix the issue. Looking at the code, the problem is in the hist method and specifically around line 6689-6690:
''python
if density and not stacked: hist_kwargs dict (density=density)
..
The issue is that when 'density=True', the 'range' parameter gets lost because we' re not including it in the 'hist kwargs Let's modify the code to fix this:
<fucttiorestr_.oollcc..ddooo
ACTION 8:
4. Let's s run the reproduce script again to contirm the fix:
58

---

Published as a conference paper at ICLR 2026
<function=basht
"pprrmmecerrecommmbbbbyyy /tsstbo//reprrduucc.yy////apa </function>
ACTION 9:
Great! The fix works. Now the bins start at and end expected.
5. Let's s think about edge cases:
When 'range=None' This should the min/max the cata, which already handled correctly by the existing code 2. when 'range' is specified but 'density=talse" The range should be respected, which it already is 3. When 'range" is specified and 'density=True' The range should be respected, which When 'bins' is a sequence The range should be ignored, which alroady handled by the code
fix
5. when 'range" includes values outside the data range The histogram should include empty bins for the range outside the data, which is already handled by numpy's s histogram function
Let me test sone of these edge cases to be sure:
ctunction=stt_repp.ccc wttp:stttrrre_iee____eeee editor>
ACTION 10:
<function-bash>
sparaietter.coommnbbbttttoo http://www..pppppe/ee/e/ee/////e (tassbeeerrpprcoucee...////aeaee </function>
ACTION
A11 test cases look good:
1. when 'range-(0) (0, 1) and 'density=True') the bins start 0 and end at
When 'range=None and 'density=True', the bins use the min/max the data
3. When 'range= (0, 1) and 'density=False", the bins start at 0 and end at 1
when 'bins' is sequence, the range parameter is ignored and the sequence is used directly
5. when 'range= 1.5) ', the bins start at -0.5 and end at 1.5
The fix looks good and handles all edge cases correctly. Let submit the changes:
<tunction=subnit> </function>
Listing 20: The interaction made by the RL prior with 1.5 x 288 SWE-Agent SFT tokens on batplotlib..mttpotllii----
The action patterns of the light SFT + end-to-end RL model read as Listing 21
ACTION 0:
Understood. Let's tackle this problem step by step.
First, need to find the relevant code the /testbed directory. The user mentioned that the code is in /testbed, so I'll start by listing the files there.
<function=ste sointtioonstr_eeplacoo cespsrrrrppace_eeiiiee
59

---

Published as a conference paper at ICLR 2026
<prrmmaeerrroommmaooooeeeeeeeeee sparaatterrratttt//sstto//aaaamt </function>
ACTION 1:
Hmm, the /testbed directory has lot of files, but need fecus on the code related to the hist function. The hist function is part of matplotlib.pyplot,, so I should look for the pyplot module in the testses//Lib//mttpttt.bb directory.
<funccioonsst__ppppcccccccttoo Sperrmeeee--ommmmddblee///aaaaee </Eunction>
ACTION 2:
Okay, I'll look at the hist function in pyplot.py.
<funciionsstt__oppacceeddee
sprramatterssommmooovrrererrreee <teremetterrppthh/tttesse////i/ http://tlotommmmooooooooooooooo/ http://tpoooittpppttottotto aparameter-viee_rrnnee [2610, 2625] </parameter> </Eunction>
ACTION
The hist function in pyplot. py simply calls the hist method of the current axes. So the actual implementation is in the Axes class.
I'll look for the Axes hist method in the /ttethee/libb//tptllllll/// directory.
sfunttionsst..roplcccedoooo
wtt.a//ccptttcccccaaa//aaaaaaaaa </function>
ACTION 4:
The Axes. hist nethod likely _axes.py, which is where most of the plotting methods are defined.
<function=str http:/sssst_reppaeeeeeeeeeeeeeee ppecaaseerroommmnbblllvv//aaeeee
set.aaateeerrrrrrooo/////aaaaaaa </function>
ACTION 5:
I'1l look at the hist method in _axes.py.
pwtppaeeeeeypppprameeeee>>>> <parametterrvieeerraggee//s 6400]</pa </prrmmtet </function>
pprrrantterppthhhreeeeeeeee//ttp ot http://ttpppptttttte////t/aa///e
ACTION 6:
I'll look at more of the hist method to where the range is handled.
<funttlorrstt.eeplcc...tttttt
gparameter=vvlew_rangee (6400, 6450] </paramete>>> </function>
ACTION 7:
I'll look at the code the bins calculated.
60

---

Published as a conference paper at ICLR 2026
<luotdooreet__ggleee...tooo
<parameteer<view_raae (6450, 6500] </parameter> </function>
ACTION 8:
I'1l look at the code where the bins are calculated.
sfunction-srrrrpplcc..ettttr
ptt.pss.ppyppppaameer>>>>> <parameteraviee_rangee Bttp:/veeeeeaaeeeeeeee range> 6550] //sram//rrm </Eunction>
ACTION 9:
I'll look at the code where the bins are calculated.
<functiooretrrrpppacc.eedd.d
perrranttte.commmmnbbieeeepreeee pparaanterrppthh//eettt//////// htt.tppoooibbb/ees__sssee ptt.aeeeepppppprmmeer>>>>> <parameter=vver 1o500, </iunction>
6o20, 6600] < </parmeeter>>
ACTION 10:
I'1l look at the code where the bins are calculated.
<functioFFsttr.eeppac..eettt editor>
sparameterrvioo__rangelll 6650) p/paametrrr </function>
ACTION 11:
1'11 look at the code where the bins are calculated.
<ftnnctionnstr_rellcce_eetooo
axes. exxs.py</parammters <parameter=view (ange>>6655)) p/aram//rrr </function>
ACTION 12:
Ah, here's the key part. The code calculates the bins using np. histogram, and if the range is specified, it's passed to bistogram_bin_edges But when density is True, the code may be overriding the range.
Wait, in the code:
if not input_enpty and len (x)
if weights is not None: w = mp.concatenate (w)
else:
W None
bins bettogram_bin_ddgee (np. cow.ccttttttttoooooo (8), bins, bin_range, _w)
else:
hist_ hist_kwargs range' bin_range
But when density is True, the code may be using the default range of the data, not the specified range.
61

---

Published as a conference paper at ICLR 2026
Wait, no. The for calculating the bins is:
if bins is None:
bins rcFarams (' whhhhht.bbi hist.binss)
if bin_range is not None: bin_range self, welff.coovet__uniiee (bin_range)
if pin_range is None:
xmin np.int
-np.inf for xi in x:
if len (xi): xnin min (xmin, np. nanmin (xi))
(xmax, np. nanmax (xi))
not ip.isman [xmin, xmax] any () and not (xmin xmax) bin_range (xmin, xmax)
# If bins are not specified either explicitly via range, need to figure out the range required for all datasets, # and supply that to np.histogram.
if not input_enpty and (x) > 1:
if weights is not None:
_M (g)concatenate(w)
else:
_W None
bins histogram_bin_edee (np. wow.ateetteteeeeeeeeeeeeee (8), bins, bingrange, _w)
else:
bin_range
density bool (density) bool (normed) if density and not stacked: hist_kwarge dict (density=densityy)
# List to store all the top coordinates of the hi histograms
tops []
mlast None
# Loop through datasets
for i in range
I this will automatically overwrite bins,
so that each histogran uses the same bins m, bins = np. hw.hsttorramx(()) bins, weights-w(i), +<hist_kwarge m m.astype (float) # causes problems later if it's an int
if mlast is None:
But when density True, the code passes density=True np. sp.histogram. However, when density is True, np historrogram normalizes the counts, but the øin edges are still determined by the range.
wait, but the problem is that when density-True, the range not respected. The user's s code is:
bins, = hist (ap. random. rand (10), "auto", range=(0, 1), density =True)
print (bins)
and the output 1s:
[0.00331535 0.18930174 0.37528813 0.56127453 0. 7.776669 0. 9333247711 which is not from 0 to 1. But when density-False, the range is respected.
62

---

Published as a conference paper at ICLR 2026
This suggests that density=True, the code is using the specified range for the bins.
But why?
Let look the code again.
When the code calculates the bins:
if not Lnput_empty and (xn(x) > 1:
if weights is not
_ np.concatenate (w)
else:
None
bins histogram_bie bin
bin_euiineeeee_eeeeeeeeeeeeeeeee edges (np concatenatee (x), bins, bin_range, _w)
clse:
hist_kwargs range') bingr
But when len (x) 1 (as in the user's code), the code enters the else clause and sets hist_kwargs range'] bin_range.
Then, in the 1oop:
for i in (ax))(nx): bins np. (x bins, weights-((i), +hist_kwargs)
But wait, if bins is None, then np. histostorra will the range. But if bins is a string (like 'auto'), ) then hp.histogram will calculate the bins based en the data and the range.
wait, in the user's code, bins is 'auto' and range is 1). When density=True, the code should calculate the bins based on the range (0, 1)
But in the code, when len (x) is 1, the code sets mist_kwargs range') bin_range. Then, in the 1oop, the ccde calls np. hi stogram (x[i), paiist_kwargs])
bins, weights=wlil,
But if bins is string (like 'auto'), then np ai.hostogram will calculate the bins based on the data and the range. So the range should be respected.
But in the user's code, the bins are not from 0 to 1. This suggests that the code is not passing the range to np. historaa when density= Trus.
But why?
This is puzzling. let me try to reproduce the error.
I'1l create a script in the /testbed directory.
<funetion=str_replean wwwpaaacsssrrrreeeeee__eeeeeeeee peplace_editoro
ACTION 13:
1'1l the script.
<tunction=bash>
<paranete:Foommadpppsee </Eunction>
/testbed/reproduce
63

---

Published as a conference paper at ICLR 2026
ACTION
Himm the output is not from to 1. This confirms the error.
Now, I need to find where in the code the range is not being passed when density-True.
Let me look at the code again.
When len is the code sets hist kwargs] range' [range bin range Then, the loop:
for i in tange(nx):
bins = np xp.histoorramx(i1) bins, weights=wlil, +hist__wvar
But if bins a string (like 'auto'), then up.histogram will calculate the bins based on the data and the range, so the range should be respected.
But in the user's code, the bins are not from 0 to 1. This suggests that the code not passing the range to np. hp.histogram when density- Trus.
But why?
This is confusing. Let me look at the code for np.histogram.
The np. hishtogram function has range parameter. If range is specified, the bin edges calculated based that range. If the data is outside the range, it ignored.
In the user's code, the data is between 0 and 1, so the range (0, 1) should be respected.
But when density-Trun the code not respecting the range. This suggests that the code is not passing the range to np.hi mp.histor stogram when density-True.
But why?
This is puzzling. Let me try to modify the code in aaxes.py.
I'll look for the line where hist_kwargs is set.
In the code:
density bool (density) bool (normed) if density and not stacked:
hist_kwarge = diet (density=densitry
This line is overwriting hist_kwargs, which may include the range.
Ah, here's s the problem. If density is True and stacked is False, the code sets hist kwargs to dict (density=density), which overwrites any previous
including the range.
This root the problem. The code overwriting hist_kwargs, which includes the range, when density is True and stacked is False.
So, in the code:
density bool (density) or bool (normed) it density and not stacked:
kast_s = dict (density=density)
64

---

kwargs, not Published as a conference paper at ICLR 2026
This line is overwriting hist kwargs, which may include the range. So the range not passed to np.histogram.
This is the cause of the problem. The solution is to add the range to hist_kwargs in this case.
So, the code should be:
density = bool (density) bool (nermed) if density and not stacked: hist_kwargs.ssttee
(' density', density)
But wait, hist_kwargs may already have the range. So, the code should not overwrite hist_ kwargs, out add to it.
The code is:
density = bool (density) or bool (normed) if density and not stacked: hist_kwargs dict (density=density)
This line is overwriting hist_kwargs, which may include the range. So the range is lost.
The solution to modity this code to overwrite it.
so, the code should be:
density bool (density) or bool (normed) if density and not stacked:
density', density)
But wait, hist_kwargs may not exist. Let me look at the code. The code for hist_kwargs is: hist_kwargs (ict()
# if the bin_range is not given, compute without nan numpy # does not do this for us when guessing the range (but will $ happily ignore nans when computing the histogram). if bin_range None:
xmin np.inf
xmax -np.inf for xi in x:
if len (xi): xmin min (xmin, na.maniin (xi))
xmax max (xmax, np.nanmax (xi))
# make sure have seen at least one non-nan and finite # value before reset the bin range
if not np. isnas [[xmin, amax]).anyy and not (xmin xmax): bin_range (xmin, xmax)
# If bins are not specified either explicitly or via range, # need figure and supply that to up.histogram.
the range required for all datasets,
if not input_empty and (xn(x) > 1:
if weights is not None:
= (p)
else:
W None
bins histogram bin Lddgs (np. concatenate (x), bins, bin range,
65

---

Published as a conference paper at ICLR 2026
else:
hist_kwargs ( range'] bin_range
density = bool (density) or bool (normed) if density and not stacked: hist_kwargs dict (density=densitt)
# List to store all the top coordinates of the histograms
tops II
mlast None
Loop through datasets
for i in range(nx):
# this will automatically overwrite bins,
# so that each histogran uses the same bins m, bins np hxstoogrammmill,, bins, weights-wlil, *shist kwargs) m a.sstype (float) causes problems later if it's an int if mlast is Nonc:
Ah, here's the problem. The code sets hist_kwargs dict (density= density), which everwrites any previcus hist_kwargs, including the range.
So, if density is True and stacked is False, the code overwrites dist_kwarge with dict (density=density) which does not include the range. So the range is not passed to up.histogram.
This is the cause of the problem. The solution is to modify this code to add to hist_kwargs, not overwrite it.
So, the code should be:
density = bool (density) bool (normed)
if density and not stacked:
wist_kwargs.sstdde.. (' density', density)
But hist_kwargs may not exist. Wait, hist_kwargs is initialized as:
h hist_kwargs dict() ()
So, the code can be:
density bool (density) bool (normed)
if density and not stacked:
hist_kwargs(" (in_klaarg""dnssttt"" = density
This way, kwargs includes the density and the range.
This would fix the problem. Let me test this.
I'll modify the code in axes.py.
ACTION
I'll run the roproduce_crror.pp script again.
<function=bash>
sprrameter-coomaddyyyyh /tsttel/rpprrddeeeerrrroo/pppppp
</function>
ACTION 15:
Great, the fix works. The bins from
66

---

Published as a conference paper at ICLR 2026
Now, can submit the changes.
<function-subnit>> </function>
Listing 21: The interaction made by the RL prior with 2 SWE-Agent SFT tokens and end-to-end SWE-Agent RI. on ma sppoot ot wi..maaplllll.......m.aa ot 1ib- 1ib-13989.
It can be seen that the two types of patterns are significantly different from each other:
The patterns of the heavy SFT model appear be more regulated, with explicitly labeled steps to 5 along the intermediate reasoning process. The reasoning content is generally short as well. This reflects the behavior of the model shaped by the heavy SFT data from the proprictary models.
The patterns of the light SFT + end-to-end RL model, in contrast, are much more under- regulated. More turns of interaction spent at repo exploration, and significantly larger amount of tokens are generated at the code edit steps before the action is made. This is the reflection behavior shaped by end-to-end RL (and less impacted by the expert data), as the sign of bessoning-intensvv skill priors is retained in the agentic scenario.
G.4 GENERALIZATION TO OTHER TASKS
In this section, we study whether the skills incentivized by Agentless training and SWE-Agent adap- tation through SWE-Smith trajectories could transfer to other SWE tasks. We use SWE-hench-live and SWE-bbenh Multilingual as our testheds.
SWE-bench- live (Zhang et al. 2025) is a benchmark for real-world issue resolution, evaluating AI systems on software engineering tasks. Using an automated curation pipeline, it is updated monthly to ensure fresh tasks and rigorous, contamination-ffee evaluation. For our experiments, we selected the default set of 300 tasks, with data collected between October 2024 and March 2025. Compared to SWE-bench Verified, SWE-bench-live live exhibits higher degree of distributional shift.
SWE-bench Multilingual Yang et al. 2025c) introduces 300 curated tasks from 42 GitHub repos- itories across programming languages, including Rust, Java, PHP, Ruby, paanerrrpp/yyySSerrpp, Go, and C/C++, covering domains such as web frameworks, data tools, core utilities, and libraries. Compared to SWE-bench Verified, which focuses exclusively Python, SWE-bench Multilingual exhibits greater linguistic and domain diversity, posing additional challenges in cross-language gen- eralization and transferability of software engineering capabilities.
Similar to previous experiments, we evaluated four model stages as the priors: the original Qwen2.5- 72B (Base), the mid-trained model (MT), the model activated with reasoning data through super- vised finetuning (SFT), and the model after RL training (RL). We still use the open-source SWE- smith trajectories to activate the agentic capabilities of each prior.
Figures I1 and 12 show the performance of the four priors on SWE-bench-Live and SWE-bench Multilingual under varied amounts of agentic trajectories for adaptation (221 one-step gradient descent, 223 1.1 x 227, and 1.5 x 228 as 100, 2,000, and 5,016 training trajectories). Each SWE- Agent adaptation experiment conducted through lightweight supervised finetuning, the training time of which ranges from several minutes two hours at most.
Compared to the Base prior, those specifically enhanced with Agentless skills (SFT and RL) demon- strate stronger task generalization, especially under the data-scarce settings. However, when more SWE-Smith trajectories are used for adaptation, the performances become closer to those of the SFT and the RL priors. This could be attributed to the gaps between the different SWE tasks. The exploration for recipes that enable stronger -fittriibittttiion and task-agnostic generalization is left for future work.
the Base and the MT priors
67

---

Published as a conference paper at ICLR 2026
BL
17.3
10.0
Passelss (S))
10.0
7.5
12.5
10.0
100
2.5
7.5
7.5
0.0
0.0
5.0
5.0
(a) #SFT 2
(b) #SFT 223
(c) #SFT 1.1 2
#SFT 1.5 2
Figure 11: Generalization analysis on SWE-bench-Live.
HL
10000E 100
(a) #SFT 2
(b) #SFT -23
(c) #SFT 1.1 x 2
#SFT 1.5 2
Figure 12: Generalization analysis on SWE-bench Multilingual.
H USE LARGE LANGUAGE MODELS
The initial draft of this paper written entirely by the authors. A large language model (gpt- 5) was used only to aid with polishing the language (e.g., grammar and clarity). All conceptual contributions, experimental designs, analyses, and conclusions are the work of the authors.
68


Reviews:
[{"review_id": "iCxboodbZG", "reviewer": "Reviewer_jxnV", "summary": "This work presents a training recipe that leverages workflow-based pipelines as opposed to more open-ended agentic rollouts in order to train an open-source code LLM for SWE tasks. The so-called Agentless training approach provides more stability and the opportunity to introduce intermediate skills, such as locating files, writing tests and fixing bugs, which may subsequently be used and composed together when training a SWE-agent model. Concretely, the authors describe: a mid-training procedure where data from GitHub issues and PR commits form examples of how developers reason about issues and fix bugs; a cold-start procedure where SWE-related reasoning traces from R1 are used to induce long CoT abilities; an RL stage where a policy gradient objective is used to train code editing capabilities with outcome-based rewards; and finally a test-time self-play stage where the model learns to play the roles of a BugFixer and TestWriter. Following this training procedure, the authors evaluate the resulting Kimi-Dev model on SWE-Bench Verified, achieving a strong performance of 60.4% for an open-weights 72B model derived from Qwen 2.5 (Base). The authors then investigate whether the structured Agentless training procedure leads to model checkpoints which offer useful \"priors\" for more open-ended SWE-agent fine-tuning. Indeed, various stages of the Agentless training procedure, in particular the mid-training, lead to a big uplift in SWE-Bench Verified performance after various amounts of SFT on SWE-Agent traces collected from Claude 3.5 Sonnet. The authors also demonstrate some evidence towards the RL prior (after the full Agentless training procedure) supporting a greater number of turns for SWE-Agent style rollouts after SFT.", "questions": "- what is behind the roughly linear increase in token length in Figure 3 as RL training proceeds? You analyze some of these traces in Appendix F, observing that there are some repetitive self-reflections, although also noting that increased response length correlated with improved answer accuracy. How do you quantify this at scale? Is the length increase due to your choice of RL objective? Recent work (https://arxiv.org/pdf/2508.08221) has commented on this sequence length effect, albeit with a GRPO-style objective.\n- How much do you attribute the performance improvements measured in both the Agentless framework evaluations and subsequent SWE-Agent evaluations to the dataset of DeepSeek R1 based reasoning trajectories? Looking at Figure 5, after $2^{23}$ SWE-Agent SFT tokens there is a large jump between the base model and MT checkpoint. What explains this?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "While advanced coding agent model training has been underway in model labs for certain years, there are relatively few detailed accounts of how to take a pre-trained model and effectively imbue it with the skills needed to solve software engineering problems. From this perspective, the work offers a valuable contribution, in particular the details of the mid-training procedure which appears to be the stage of Agentless training that yields the largest SWE-Bench Verified performance improvement after SFT on SWE-Agent traces; perhaps due to training the model to reproduce the motifs, reasoning patterns and skills required to conduct software engineering work. The work is of a reasonably large scale, and achieves compelling results on SWE-Bench Verified for a model of its size, with 60.4% resolution rate on SWE-Bench Verified with the Agentless-like framework.", "weaknesses": "While the Agentless performance for SWE-Bench Verified is reasonably strong at 60%, the pass@1 rate on the SWE-Agent results are somewhat low at 48.6%, particularly given the model is roughly double the size of other models obtaining comparable scores. It would be good to investigate why this is, and crucially whether this is as a result of the Agentless training procedure that preceded SFT for SWE-Agent.\n\nMoreover, the authors train on the outputs of other models at two key points in the paper. This is disappointing since it effectively hides how the corresponding skills can be trained from scratch in an open-source model, instead deferring this to the black box of a preexisting model trained by some other lab. The first instance of this is during mid-training - which appears to be one of the steps in the training procedure which yields the greatest performance improvements - where traces from the DeepSeek's R1 model are used to introduce skills such as reasoning, problem analysis, method sketching, self-refinement and exploration of alternative solutions. This is an important procedure, and it would have been good to see discussion of how these skills could be bootstrapped without training on the outputs of a model which has already undergone this training. The second instance are the SWE-Agent traces used for SFT from Claude 3.5 Sonnet. While I understand that the primary focus of this paper is the Agentless training procedure itself, I do believe that any conclusions pertaining to the usefulness of this intermediate training procedure as a \"prior\" for training SWE-Agents must be made with a more legible training procedure for the SWE-Agent stage. In other words, had the jump from Agentless to SWE-Agent been achieved by a transparent RL procedure, then conclusions about the usefulness of the Agentless skill prior may be more impactful. Instead, we appear to have learned that these \"priors\" are useful when doing imitation learning from black-box model traces.\n\nFinally, on the point about generalization, the studies all revolve around SWE-Bench style tasks (including the live and multilingual variants in Appendix E). These represent only a small subset of software engineering work, and do not address related tasks such as novel algorithm design, performance optimization, task management and so forth, which I believe would be a better quantifier of generalization than merely more up-to-date or polyglot variants of SWE-Bench's issue resolution task. Indeed, this is an area where the paper's Agentless training thesis may hold promise; where systematically created workflow-based mid-training procedures can imbue a model with a diverse range of useful skills which can be learned to be composed together in the RL stage.\n\nNits:\n- L427: do you mean to say \"The RL prior outperforms all the other models...\"?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3cdegLd6SG", "reviewer": "Reviewer_xDcT", "summary": "The paper argues that “agentless” workflow training can serve as skill priors (localization, code edit, self-reflection) for end-to-end SWE agents. It introduces Kimi-Dev, a 72B open-weight code LLM trained via a curated recipe (mid-training on PR/issue data, a reasoning-heavy cold start, RL on code edits, and test-time self-play). Under a workflow (agentless) setup, Kimi-Dev reaches 60.4% on SWE-bench Verified; with 5k public SWE-Agent trajectories for light SFT, the model attains 48.6% pass@1 in a multi-turn agent framework, comparable to Claude 3.5 Sonnet (241022). These results aim to bridge workflow and agentic paradigms by treating agentless training as a foundation for agent adaptation.", "questions": "- For weakness 1: it will nice to either (1) show the benefit of RL by demonstrating performance improvements on other datasets/metrics/settings (eg higher pass@k for large k); OR (2) discuss the limitation of agentless+RL in providing better prior\n- For weakness 2: It will be nice to either (1) disprove it or (2) discuss the limitation\n- For weakness 3, my question is: Given the same computation budget, what if we perform distillation on the same set of training tasks (eg, swe-smith, swe-gym) -- Is it possible that the performance gain we get there will be larger than the one we get with agentless as prior? If so, why is it a good idea to use an agentless prior rather than directly generating data from environments that better match the domain?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "- This paper exercises a principled recipe and delivers strong results under agentic settings. The staged recipe (mid-training on ~150B tokens; 50B natural diffs, 20B PR commit packs, ~20B synthetic, upsampled ×4; reasoning-heavy cold start; RL with outcome-only rewards; test-time self-play) is coherent and empirically validated. The paper achieves 60.4% on SWE-bench Verified under a standardized 40 patch/40 test setting, with clear scoring for self-play selection.\n- Agent transfer with small data. With only ~5k public SWE-Agent trajectories, the adapted model attains 48.6% pass@1\n- The authors performed detailed experiments demonstrating the skill transfer and generalization effect by controlling for different prior models. These experiments provide valuable insights into how the community can improve future agent models.", "weaknesses": "- It is unclear to me that RL is an indispensable part of the recipe: In Figure 5, the performance difference between the RL and the SFT curves is relatively small. Does this demonstrate that the RL part is not really the essential part of the recipe—SFT on an agentless task is already sufficient to provide such a prior to improve the upper bound on performance given sufficient multi-turn agent SFT data?\n- Unclear if Agentless Skill prior will still be useful given sufficient multi-turn SFT data: Also in Figure 5, when given sufficient agent SFT data, the performance of MT gets very close to SFT/RL. Does that mean the value of agentless skill prior drops as you have access to more in-domain agent SFT data?\n- Unclear if performance improvements come from distilling from a stronger model: If I understand correctly, the SFT data for agentless skill was distilled from Deepseek R1 (section 3.2) - could the effectiveness of \"agentless prior\" come from distillation from the Deepseek R1 model?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AQZu6Fon89", "reviewer": "Reviewer_kUZK", "summary": "The paper introduces Kimi-Dev, an open-source LLM for software engineering that unifies agentless and agentic approaches. It shows that agentless training through modular tasks can build transferable skill priors that make subsequent agentic finetuning more efficient. Kimi-Dev achieves SOTA on SWE-bench Verified and enables competitive SWE-agent results with minimal additional training.", "questions": "N/A", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The main discovery of agent less training can induce skill that make further agent training more effective is interesting: it opens up a new paradigm how researchers may train their agent. The open source of the recipe further add to the contribution.\nThe proposed multi-stage training recipe is well-explained and reflect the researcher’s understanding of the nuances in scaling SWE models.\nThe empirical results of the trained model is strong, achieving SOTA among open-sourced models for workflow-based approaches on SWE-bench, and competitive pass@1 performance under agentic framework. This shows the method’s effectiveness.\nThe analysis of the paper is very insightful, including the varying of token budget, RL progression, etc. All of these settings reflects a solid experimental approach.", "weaknesses": "Some technical details are deferred to the appendix. The main context only gives a final evaluation formula while the concrete implementations of reward, prompt sampling, etc. are not rigorously and clearly laid out in the main text.\nThere are some observations about false positive mentioned, but the paper can further benefit from providing quantitative and qualitative analysis of the error remaining given the current bug fixer and test writer, especially points that can reflect the challenges in agentic skill prior transfer.\nMuch of the justification for skill transfer from agent less to agentic framework is empirical. As the core discovery of the paper, more principled proof or study about how skill prior linked to generalization and how these skills are manifested will further strengthen the claim.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "D32dEDX92D", "reviewer": "Reviewer_KmTh", "summary": "This paper proposes a two-stage training approach for software engineering (SWE) agents: first training on \"Agentless\" workflows (fixed pipeline tasks such as bug localization and repair), then adapting to multi-turn agentic frameworks. The authors introduce Kimi-Dev, a 72B parameter model that achieves 60.4% on SWE-bench Verified through Agentless training, and 48.6% when adapted to the SWE-Agent framework with only 5k trajectories.\n\nThe central thesis is that Agentless training, framed as a single-turn task, facilitates more scalable RL training and serves as an effective prior for transitioning to multi-turn scaffoldings.", "questions": "1. What is the precise configuration of SFT used for Figure 5? While the number of trajectories is clearly specified, what is the number of training epochs? Were hyperparameters tuned independently for each experimental run, or was a single configuration used across all conditions?\n2. What are your thoughts on the TestWriter false positive examples documented in the appendix, and how might this limitation be addressed in future work?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written with clear separation and explanation of each component, making the methodology easy to understand and follow.\n2. The ablation studies (Figure 5) systematically isolate the contributions from mid-training, cold-start, and RL stages, providing clear illustration of each component's contribution. The sweep over different amounts of SWE-Agent SFT data (from 0 to ~1.5×2^28 tokens) is particularly comprehensive.\n3. The reported performance of 60.4% on SWE-bench Verified within the Agentless scaffolding using Qwen2.5-72B as the base model is impressive.\n4. The extensive appendices document data recipes, training procedures, and infrastructure details in substantial depth, significantly enhancing the work's reproducibility.", "weaknesses": "1. The anomalous performance dip of the \"SFT\" prior at 2^24 tokens (Figure 5) raises questions about the stability and reliability of the proposed priors. Including confidence intervals would help clarify whether the observed differences between priors are statistically significant or within the margin of experimental noise.\n2. The 5k SFT trajectories represent a relatively small subset of potential teacher data. Given that Claude 3.7 Sonnet is a substantially stronger model, there may be a scenario where scaling the number of trajectories could narrow the gap between the Base and Kimi-Dev priors. If this is the case, the proposed approach would offer a more efficient rather than more powerful method for training multi-turn SWE agents, which represents a weaker claim that it could be otherwise.\n3. While the Agentless checkpoint demonstrates strong performance as a prior for multi-turn adaptation, it remains unclear whether directly applying large-scale RL to the SWE-Agent framework would yield comparably strong or weaker results. The literature includes examples of applying GRPO (DAPO) to sparse reward settings for LLMs in SWE tasks, such as DeepSWE [1]. A direct comparison or discussion of this alternative training paradigm would strengthen the paper's claims about the necessity and superiority of the two-stage approach.\n\n[1] Michael Luo et al. DeepSWE: Training a State-of-the-Art Coding Agent from Scratch by Scaling RL", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)