Co-PatcheR: Collaborative Software Patching with Component-specific Small Reasoning Models
Authors: Yuheng Tang
Hongwei Li
Kaijie Zhu
Michael Yang
Yangruibo Ding
Wenbo Guo
Venue: NeurIPS 2025 poster ()
Keywords: automatic program repair
autonomous software improvement
software patching agents
customized reasoning models
Primary area: applications
TL;DR: We propose Co-PatcheR, a novel software patcher with collaborative small reasoning models.

Abstract: Motivated by the success of general‑purpose large language models (LLMs) in software patching, recent works started to train specialized patching models. Most works trained one model to handle the end‑to‑end patching pipeline (including issue localization, patch generation, and patch validation). However, it is hard for a small model to handle all tasks, as different sub-tasks have different workflows and require different expertise. As such, by using a 70 billion model, SOTA methods can only reach up to 41% resolved rate on SWE-bench-Verified. Motivated by the collaborative nature, we propose Co-PatcheR, the first collaborative patching system with small and specialized reasoning models for individual components. Our key technique novelties are the specific task designs and training recipes. First, we train a model for localization and patch generation. Our localization pinpoints the suspicious lines through a two-step procedure, and our generation combines patch generation and critique. We then propose a hybrid patch validation that includes two models for crafting issue-reproducing test cases with and without assertions and judging patch correctness, followed by a majority vote-based patch selection. Through extensive evaluation, we show that Co-PatcheR achieves 46% resolved rate on SWE-bench-Verified with only 3 x 14B models. This makes Co-PatcheR the best patcher with specialized models, requiring the least training resources and the smallest models. We conduct a comprehensive ablation study to validate our recipes, as well as our choice of training data number, model size, and testing-phase scaling strategy.

Co-PatcheR: Collaborative Software Patching with Component(S--specifee Small Reasoning Models
Yuheng Tang Hongwei Li
Kaijie Zhu
Michael Yang!
Yangruibo Ding" Wenbo Guo
University of California, Santa Barbara
University of California, Los Angeles
(yuhengtang, hongwei, kaijiezhu, yang335, henrygwb)@ucsb edu yrbdingdcs ucla.edu
Abstract
Motivated by the success of general-purpose large language models (LLMs) in software patching, recent works started to train specialized patching models. Most works trained one model to handle the end-to-end patching pipeline (including issue localization, patch generation, and patch validation). However, it is hard for a small model handle all tasks, different sub-tasks have different workflows and require different expertise. As such, by using a 70 billion model, SOTA methods Motivated by
only reach up to 41% resolved rate the collaborative nature, we propose Co-PatcheR, the first collaborative patching system with small and specialized reasoning models for individual components. Our key technique novelties are the specific task designs and training recipes. First, we train a model for localization and patch generation. Our localization pinpoints the suspicious lines through a two-step procedure, and our generation combines patch generation and critique. We then propose a hybrid patch validation that includes two models for crafting ispuurieno test cases with and without assertions and judging patch correctness, followed by a majority vote-based patch selection. Through extensive evaluation, we show that Co-PatcheR achieves 46% resolved rate on SWE-bench- Verified with only 3 14B models This makes Co- PatcheR the best patcher with specialized models, requiring the least training resources and the smallest models. We conduct a comprehensive ablation study to validate our recipes, well as our choice of training data number, model size, and testing-phaat phase scaling strategy.
WWWE-bench-Verffeee
Introduction
Software patching is time-intensive and cognitively demanding task, especially for large and com- plex codebases. In the real world, effective patching often requires a combination of complementary skills: locating the faulty component, generating plausible fixes, and validating the changes. Recent works leverage general-purpose LLMs [32
[5] 4 to construct patching agents with three components, responsible for localization, generation, and validation, respectively. They demonstrate remarkable performance on SOTA benchmarks (e.g., SWE-bench
3 34 33
(22) and show significant potential to automate patching in the real world. Despite these promising results, concerns about cost effi- ciency and data privacy further motivate the development of customized patching models. Current approaches train one model for the end-to-end patching pipeline through supervised fine-tuning (SFT) reinforcement learning. Specifically, early works [54] 29 fine-tune 72B and 32B models through simple supervised data and achieve around 30% resolved rates on SWE-bench- Verified. More recent methods implement rule- based rewards and train reasoning models with reinforcement learning, with SWE-RL (50) achieving the highest resolved rate of 41 % on SWE-bench-Veris using 70B
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

model. However, these monolithic approaches fail to imitate the real-world patching paradigm, where specialized engineers collaborate by dividing responsibilities according to their expertise.
Inspired by the collaborative workflow in the realistic software engineering practice 130 51. we propose Co-PatcheR, the first patching agent with collaborative small reasoning models designed specifically for different components. Our key insight for having componen(() that different components have different inputs, outputs, and capability requirements. Specifically, localization and generation require a similar capability of interpreting the issue description and understanding the current codebase. Validation, on the other hand, generates testing cases without knowledge of the patches or the codebase. Given the non-trivial differences, it is challenging for one small model to handle all these sub-tasks. Following this intuition, we craft a tailored task design and training recipe for different components, aiming to minimize the model size while preserving performance. Specifically, given the similarity between localization and generation, we train a single model (Loc-Gen model) to handle both functions. For localization, we design it as a two-step procedure, where the model first identifies the affected files and then pinpoints the specific lines responsible for the issue. This task decomposition reduces the task complexity and context length, making it more suitable for small models. For patch generation, we train the Loc-Gen model to not only generate patches but also review and refine its own solutions. With this additional self-critical capability, the Loc-Gen model can prevent common errors and generate higher-quality candidates. Finally, we train two models to generate multiple and diverse issue- reproducing test cases (PoC) and judge the patch correctness based on the PoC execution outcomes. The insight here is to provide diverse PoCs for more sound correctness judgment. Here, Val-assert model and Val-no-assert model generate PoCs with and without assertions, respectively. We use these models together with available functionality tests and majority vote mechanism to select the final patch. For all three models, we apply model distillation with a novel data construction method to enable their reasoning capabilities. Different from existing distillation models (e.g.. S1 (3I), we find that creating reasoning data with correct answers is critical for our fine-tumed model to achieve high performance.
models is
Through extensive experiments, we first show that when using only 3x 14B models, Co-PatcheR can achieve a 46% resolved rate on SWE-bench- Verified with 60 patch candidates. Compared to SWE-RL, Co-PatcheR achieves high resolved rate with 40% fewer parameters and 88% fewer samples. Besides, Co-PatcheR only needs to run one 14B model at a time, which is much more efficient than SOTA methods during the testing phase. Furthermore, with our specific reasoning data construction method, Co-PatcheR only requires 6K data for training, which is much more efficient than SOTA methods that use at least 30K samples. We then conduct a comprehensive ablation study for each model to validate its task design and training recipe. Finally, we validate the necessity of eesting-hhase reasoning, our choice of data number and model size, through more ablation studies.
Contributions. We propose Co- PatcheR, the first collaborative patching system with component- specific reasoning models. Co-PatcheR is the most data- and parameter-efficiert patcher that offers greater effectiveness, efficiency, and modularity than existing patchers with specialized models. Co-PatcheR ranks among the top-10 open-source systems on SWE-bench bench-Verified outperforming all patchers with open-source models. We propose specific training recipes for each model and obtain the following new findings that are unique to patching:
Using one model for localization and generation performs similarly to using separate models.
Multiple models for PoC generation provide necessary diversity that single model cannot achieve.
Critique is important for generation, and multi-source data is important for validation.
Simply increasing data or model size not always helpful; data scale should match model size.
Rejection sampling based data filtering helps all components; but rationalication does not.
2 Existing Works and Limitations
LLM-based patching agent. There are several works on designing patching agent using general- purpose LLMs 25
7 6 27 20 &
14
13
56
59
35 Some agents achieve remarkable
performance on the SWe-h benchmark 22 benchmark for real-world GitHub issues written in
Python, The ton-ranked open-source agents are OpenHands 488, Agentless [53], and PatchPilot 22).. Here, Agentless and PatchPilot follow pre-defined workflow, where PatchPilot introduces number

---

Training Recipes
Inference Pipeline
Localization Data
Generation Data
Validation Data
O Issue: Add the support for...
① Issue Add the supporttor
O Issue Add the support for
3 lasue 1: Add the support for
Loc-Gen Model
Conptanncc archilech int...
deepseek architettur...
deopseck architacture..
Localization
Validation Models
Shop File Localbation
Step T: Patch Ounonation
Val wweerren--meeel
Stup 1: Rite Localization
Boxegh
Plaachenmr
Blak
Step Lin lccaliationn
Class Description
Ceess Deacrizion
PvG pinoution
and avelsnkaa....
Shunks
Saall garatt
+
Step Uin Locatization
Step 3: Potsh Coilo
Step ti peschpettatioon
Functionaitt Teat
leess Desurggaas
Ciline Descrple
2-1 phrppuchh har p07t bypeet..
Pass of not
Step Crtch
Fass providadia
Pettrss previde
Dynamic
f Filter Criteria
Ansowar Corractness
Reasoning longth
Bast patches
88
Loo-Gen model
Val-assert
Val-no-assert
Vosting
Finat Patch
Figure/ 1: The overall training recipes and inference pipeline of Co-PatcheR. We design one model for localization and generation, where each component has two steps. We further design two models for PoC generation with/without assertions. During inference, we conduct PoC and functionality testing to select the final patch and conduct majority vote when dynamic testing has ties.
of optimizations over Agentless. OpenHands, the other hand, gives more freedom to the LLM to decide its workflow on the fly. OpenHands can have a higher performance but is less stable and more costly than PatchPiolt. We use PatchPiolt as the agent scaffold as it is more cost-efficient.
Specified models for patching. There are some early explorations on training customized LLMs for the patching task. At high level, most methods train one model for the end-to-end pipeline, and they use relatively large models. Specifically, SWESynInfer [27 and SWF--Gym [6) train model with 72 billion (72B) and 32B parameters, respectively, to perform the end-to-end pipeline. Both models are trained with supervised fine-tuning (SFT) without a testing-phase reasoning. Their resolved rate on SWE-bench-Veritee
is around 30%. SWE-Fixer (54) trains one 7B model for fault localization and a 72B model for patch generation with a resolved rate of 33% on SWE-bench- Verified.
Followupp works explore training the model with reinforcement learning to enable testing-phase
reasoning [58 29 50 SEAlign [58] continues training on SWE- Gym (36 using Direct Preference
Optimization 40 to retain preferred solution paths. SoRFT 29 and S WEERL (50 define rule-based
rewards and train the model with policy gradient methods (PPO
43 and GRPO
44 for both
localization and generation. Among these three methods, SWE-RI. achieves the highest resolve rate of 41% on SWE-bench- Verified with 70B model. A concurrent work, SWE-Reasoner 28 on the other hand, applies SFT-based model distillation (from DeepSeek-rl (15) to train a 32B reasoning model for the end-to-end pipeline. They further trained two 32B critique models for localization and patch selection. They achieve 46% resolved rate on SWE-bench-Veite
with all three models.
Other code LLMs. First, there are some coding LLMs for general coding tasks (e.g.. LeetCode, Data Science), including Qwen2.5-Codee Coder [9), DeepSeek-Coder Coder [61], WizardCoder (26), CodeLlama 41 and reasoning models (e.g., S*
23 and CYCLE
Second, existing works also explored devel-
oping models for debugging [10 21 60
52
45
testing case generation [8, (38), function and
API calling 37] secure code generation work on training patching-specife models.
[7
12
6 55). These efforts are orthogonal to our
3
Key Technique
3.1 Overview
Problem setup. We are given a software repository with one multiple issues/bugs. Each issue has a simple text description, which may contain additional information such as desired behaviors and vulnerable inputs. Each issue may affect one or more functions in the repository. Our goal is to automatically analyze the issue and generate patches that fix all affected functions while preserving the behaviors of the unaffected functions (which are evaluated by running the hold-out unit test).
3

---

Technical insight. In this paper, we first argue that designing small and specialized patching models improves the overall efficiency of the patching system, as Besides, we do not need the model to process images or video; instead, it must precisely understand the repository, reason about issues, and generate correct patches. Second, we argue that having single model for the end-to-end patching pipeline may not be the optimal solution given the differences between components and the collaborative nature of software patching. Specifically, both localization and generation need to interpret the issue description and connect it to the target codebase, especially the code chunks responsible for the issue (root cause). Localization needs this capability to scan the entire codebase to pinpoint the root cause, and generation then relies that information to craft patch candidates. By contrast, test case test) generation during validation demands an even deeper understanding of the issue description, yet it does not need to analyze the full codebase (given that the test cases are typically generated only based on the issue description). Besides, test case generation is typically not aware of patch candidates in order generate more objective and comprehensive testing. This hypothesis is supported by existing works (50) (44) that trained large models (>70B) using various methods (SFT, offline and online RL) yet achieved relatively low performance (See Section 4. Based on these findings, we propose to train small but fine- fine-grained models for differeet components and use them together in the agent system.
LLMs are way larger.
Technical challenges and key novelties. The high-level challenge is to reduce each model to the smallest feasible size without sacrificing too much performance. More concretely, we need to first design a data -effccien training recipe for each model (Challenge O). Once we have the models, we also need to decide how to effectively integrate them into the overall agent system (Challenge @).
Solve challenge O. We propose to train three reasoning models, Loe-Gen model for localization and generation, Val-no-assert model and Val-assert model for PoC test cases generation with and without assertions (Figure D]. First, as shown in Section 4 training reasoning models can achieve a better performance than non-reasoning models even with fewer training samples, making the model training even more data- and cost-efficient. We propose to distill a large reasoning model with supervised fine-tuning. Recent research shows that high-quality distillation data enables training effective small reasoning models for math and coding tasks with limited computational resources [46] 31.. In contrast, training reasoning models with RL requires substantially more samples and computational power, contradicting our efficiency goals. Additionally, without well-designed intermediate process rewards, training based solely on outcome rewards becomes costly and unstable [36). Second, we train one model for localization and generation as they share similar capabilities. As shown in Figure we divide the localization task into two lower-complexity sub-tasks generate training data separately, and mix the data for model training. For generations, we integrate "critique training data" where the model reviews its own patches, enabling better reasoning about bug fixing. Third, we design two models for PoC generation to enable more diverse PoC testing. For each model, we train it to (I) generate PoCs that potentially trigger the target issue and (2) evaluate patch correctness based on issue descriptions and PoC execution outcomes.
Solve challenge 2. Figure shows our proposed agent workflow, which is inspired by the efficient designs of PatchPilot (24]. Our localization first identifies the files and then the lines in the pinpoint files for locating potential root causes. The generation component then generates multiple patch candidates. Finally, we use our two PoC generation models for patch correctness testing, followed by a model-free functionality test that runs patches against public functionality tests. We rank patches based on dynamic testing results (Num. of passed PoC and functionality tests) to identify the highest- scoring candidates. When multiple patches achieve the same highest score, we apply majority voting based on normalization (3 to select the final patch.
3.2 Training Recipe for Loc-Gen model
Our training recipe has four key components: training issue selection, training task construction, reasoning data generation, and filtering. Issue selection and data filtering are common across all components, while task construction and reasoning data generation are tailored to each model.
3.2.1 Issue Selection
We select training issues and the corresponding codebases from the SWE-bench training set and SWE-Gym dataset, which contains different repositories from our testing set. Our selection criteria focus on two key factors: First, we prioritize diversity by selecting issues from different repositories

---

across different time periods to improve model generalization. Second, we include issues with varying difficulty levels, following recent work in S1 [I] showing that challenging cases improve reasoning abilities with limited training data. We quantify the difficulty using the number of files changed in the fix. For example, simple issues require changing only one file, while difficult issues require changes to five or more files. As shown in Section 4 in general, the performance of our models first increases as the training data grows and then remains at a similar level. Guided by this trend, we select only 2,000 training issues, which is significantly fewer than existing RL-based methods, e.g.. 1 11M in SWE-RL [50]. To avoid data leakage, we check the selected issues, making sure they come from different codebases from the testing ones and do not have overlapping functions.
3.2.2 Task Construction and Reasoning Data Generation
Localization. The key challenge for localization is to efficiently identify root causes while keeping the task manageable for small model. To achieve this, rather than training the model to directly identify root causes from the entire repository, we decompose localization into two sequential sub- tasks: file localization and line localization. File localization identifies issue-related files based on the file structure, while line localization pinpoints the specific code chunks within selected files. This decomposition creates simpler sub-tasks with shorter context, better suited for small models. It also provides explicit guidance to the localization process. Specifically, for file localization, we provide the issue description and codebase file structure as input. For line localization, we input the issue description and the code from each selected file (splitting files that exceed the model's context limit) (see Appendix [E for the prompts). Based on this task design, we use Sanude-3.7-Sonee [4] to generate distillation data with reasoning chain. The reasons for choosing other reasoning models are that some models (c.g., OpenAl models) do not provide their reasoning chains. Among the two main open-source models, DeepSeek-R1 [15 tends to give overly long reasoning chains, making it easy for our model to learn noisy steps. QwQ [39, meanwhile, performs poorly on patching-relater tasks.
Souuude-.7-Somme over
Generation. The key novelty of our generation model is to combine the patch critique with the patch generation. For patch generation, we design the input as the issue description and the identified root causes, and the output as a patch candidate. For the patch critique, our input is the issue description and a patch candidate, and the designed output is the review of the candidate answer if it is wrong [49]. We still use Claude-3.7-Sonne to generate the reasoning data for these two sub-tasks. Having the critique data can guide the model to learns not only to produce answer but also to diagnose and refine existing ones, thereby acquiring stronger reasoning skills during training. Such process could further deepen the model's understanding of the target issues and potentially yield higher-quality patches. Appendix E specifies our input prompts.
well correct
3.2.3 Reasoning Data Filtering
After generating the data, we apply filters based on two aspects: final answer correctness and reasoning length. First, based on our empirical observation, we conduct rejection sampling to filter out the training samples that lead to wrong answers, as training with these noisy samples will jeopardize our model performance. This is unique recipe for patching, as it does not align with existing work, Sky-TI [46) and S1 [31], where they state that data with wrong answers is still useful for models to learn the reasoning structure in math problems. We believe the difference stems from the specialized nature of patching, where the related tasks are not frequently encountered during pre-training. As such, a small model needs access correct answers learn the correct knowledge. For general maths problems, however, the model has likely seen enough examples in pre-training and the model can tolerate occasional wrong answers. Here, reasoning data mainly teaches it to perform reasoning following a certain structure. Second, we filter out samples with excessively long reasoning chains, as these kind of long reasoning does not offer much benefit even on general-purpose L.LMs (Appendix C). A deep inspection of the reasoning chain shows that the model tends to overthink and repeat reasoning paths. Such data can even cause model collapse and jeopardize training efficiency.
3.3 Training Recipe for Val-assert model and Val-no-assert model
Rationale for having two models. The high-level goal of validation is to decide whether a candidate patch fixes the issue (patch correctness) and whether it affects other benign functions (functionality conrectness). For functionality correctness, we can retrieve the public unit test cases from each project
5

---

Table I: Co-PatcheR vs. baselines SWE-bench- N/A means not available.
Model
Agent scaffold
Training
Reasoning
Resolved (%)
Training data
candidates
Genenal Commercial LLMss
Claude-3.--Sonns
Agentless
50.80
N/A
Claude-3.--Somm
OpenHands
53.00
N/A
N/A
Claude-3.5-Some
PatchPilot
X
53.60
N/A
12
Claude-3.--Sonne
OpenHands
/
60.60
N/A
N/A
Specialized Open- source Models
Lingma-WEE-PPT-70
SWE-Synlnfer
SET
K
30.20
90K
N/A
SoRFT-32B
Agentless
SFT
X
30.80
30K
N/A
SWE-Fixer-7228
SWE-Fixer
SFT
SWE-RL-70B
Agentless-miii
RL
41.00
11M
400
Co-PatcheRR (3x
Agentless-mini
SFT
45.20
6K
60
Co-PatcheR (3x 14B)
PatchPilot
SFT
46.00
6K
and run dynamic testing against our patches. The key challenge is to design the patch-correctner validation. To enable dynamic testing, we propose to train two validation models to generate PoC test inputs and make patch correctness judgment. The definition details about two kind of test case format are explained in Appendix B The insights for having two models are as follows. First, existing patching agents [24 53 have two ways of generating PoC tests: with or without assertions. Here, assertions mean specific assertion instructions in the PoC test that judge whether the PoC execution triggers the target issue. The test cases with and without assertions typically cover different program paths to the root cause site. To enable more comprehensive and sound PoC tests, we aim to generate PoCs both styles. As such, we train two different models, one for each style. As shown in Appendix
C we also train model to generate both types of PoCs with different input prompts and special tokens. However, the model cannot give PoCs with enough diversity, even with a high temperature during testing.
Training recipe. Here, we use the same set of training issues as the Loc-Gen model. We design two types of input prompts to instruct the teacher model to generate PoCs with and without assertions (Appendix(E) (E) Both input prompts contain the issue description and format instruction (with/without assertions). Different from Loc-Gen model, we use two teacher models, Claude-3.7-Sonee and 04-mini, to collect the reasoning data. The goal here is again to increase the PoC diversity and thus path coverage to the root causes. For Val-no-assert model, we further gather judgment data, where the input is the issue description, the current patch, and its PoC execution outcomes, and the output is whether the patch fixes the issue. We train Val-no-asee model to generate the PoCs and judge the patch correctness at the same time. For Val-assert model, we only train it to generate the PoCs, the PoC correctness be decided by assertions. As shown in Figure 1 we run dynamic testing with PoC and functionality tests, and conduct a majority vote to select the final patch when dynamic testing has ties.
4 Evaluation
4.1 Co-PatcheR vs. Baselines on SWE-bench
Setup and design. We adopt the Qwen-2.5-Coom 14B model [19] as our base model for all three components. Compared to more recent models, Qwen-2.5-Coder-144 has the knowledge cut of March 2024, which is prior to the SWE-bench benchmark (published in May 2024). is less likely to be trained specifically for the SWE-hbech data. As introduced in Section 3.2.T select 2K training issues from the SWE-bench training set and the SWE-Gym (36] dataset and conduct filtering to avoid data leakage. After training our three customized models, we integrate them into our end-to-end pipeline (Figure[] [] and evaluate our system (Co-PatcheR) on the SWE-bench- bebeenhh--eiiieed Verified dataset. The specific training
are shown in Appendix A During the inference, for every issue, we generate 5 root causes from localization, 60 candidate patches, and 4 PoCs, using them to get one final patch. We compare Co- PatcheR with SOTA agents built on commercial LLMs and those with open-source models. We report the resolved rate of these agents' final patch (best@k), as well as their number of patch candidates (if available). For open-source models, we also compare Co-PatcheR with them in training data and model size. Note that a recently released concurrent arXiv work (SWEReasoner (28) claims 46% resolved rate with x 32B models, We achieve the same resolved rate with over 50% smaller models.

---

Results. Table 1 shows the comparison between Co-PatcheR and the baseline methods. As we can first observe from the table, most existing specialized models have a large performance gap from the agents with commercial models. SWE-RL archives the highest resolved rate with 70B model with 110M training data and 500 candidate patches. comparison, Co-PatcheR sets new open-source record with resolved rate of 46.00% using only 3 x 14B models trained with 6K training data. This result validates the advantage of having component-ppecific models over one end-to-end model when patching with small models. It also demonstrates the effectiveness of our issue selection and reasoning data generation and filtering methods, which significantly improve Co-PatcheR's ' data efficiency. Besides, the resolved rate of Co-Pate Co-PatcheR ranks among the top- 10 open-source tools on SWE- Verified, beating many agents with commercial models. The result shows the importance of having specialized models for software patching. Finally, Table sshows the advantages reasoning models for both general and specialized L.Mss. For example, OpenHands has 7% improvement when using Claude-3.7-Somee (reasoning model) compared to (lauude3.5-Sommee (non-reasoning model). At the same time, Co-PatcheR and SWE-RL also have significant advantages other baselines with non-reasoning models.
4.2 Effectiveness and Ablation Study of Each Component
4.2.1
Localization
Design. We evaluate our localization component against three commercial LLMs (GPT-4o, Claude-3.7- 3.7oo-e,,, o4- mini) on SWE-bench- Verified, measuring both file-level and line-level localization accuracy. To isolate the effect of our data filtering strategy. we also train a comparison model (Loc-NoFilter) with unfiltered data containing both correct and wrong answers, using the same 2K data size for fair comparison. We also compare against our base model 20wen-2.5-Coder-11e
120
50
WWE-bench-Verifiedd
50
74434A
63,
59.2
Une
to demonstrate the impact of our specialized training. For all models, select Top @5 files and report whether the correct answer appears in the root causes identified from these files. For issues affecting multiple files or lines, we enforce strict evaluation criteria, counting a localization as correct only when it identifies the complete set of affected files and lines. Note that we do not consider training model to directly identify vulnerable lines from the entire repository. as will exceed the model's context limit.
G/T/40 Qwer2 S-cee-5---eeeaa 163
Clude 3.7 Sennet Co-Patchebolec
Loc Noftier
Figure 2: level accuracy for localization.
The top@5 5 file-level and line-
Results. As shown in Figure 2. SOTA commercial reasoning models 04-mini and Claude-3.7-Sonner achieve the highest performance both file and line levels, marginally outperforming Co-PatcheR- Loc. However, Co-PatcheR-LL achieves comparable performance to GPT-4o, demonstrating the advantage of specialized reasoning models over general non-reasoning models. These results support our claim that specialized models with proper cesting-phase scaling can compete with much larger commercial LLMs specialized tasks. The substantial performance gap between Co-PatcheR-Lan and both Qwen-2.5 2.5-Corrr 14B and Loc-NoFilter validates the effectiveness of our training recipe, particularly reasoning data filtering approach.
4.2.2 Generation
Design. Following the experiment design for localization model, we evaluate our generation component against com- mercial LLMs (GPT-40, o4-mini, Claude-3.7-Sone To isolate the contributions
40.8
and our
Performance (96)
40
base model (Qwen-2.5-Coder-
292
30.8 31.6
30
of our training innovations, we test two additional variants: Gen- Base (using unfiltered reasoning data without critique training) and Nen-NoFilter (adding critique data but without data filter- ing) to verify the effectiveness both data filtering and critique training techniques. For a fair comparison and to focus specif- ically on patch generation capabilities, we use GPT-4o local- ization results consistent input across all models, evaluating
20
16.6
Owen-2.5-Cader
Gar-NoFiker
o6-mini
Co-PatcheRRGGe
Cludee33 7--Sonn
Figure 3: The pass @ resolved rate for generation.
7

---

the performance using the pass@1 I metric, which evaluates the successful issue resolution with only one generated patch.
Results. Figure 3 shows the pass@1 performance across mod- els, with results consistent with our localization experiments: 04-mini and -laude-3.7-Somme outperform Co-PatcheR-Goo in single-patch performance. However, as demonstrated in Figure [5b] if effectively leveraging our testing-phase scaling approach, Co-PatcheR-Gen achieves comparable per- formance to these much larger models when generating only 4 more patch candidates. Furthermore, the performance advantage of Co-PatcheR-GGes over both Gen-Bas and Gen-NoFilter validates our novel designs: critique training and data filtering substantially improve patch quality. We note that GPT-4o's unexpectedly low performance stems primarily from formatting issues, as it frequently generated syntactically invalid patches that did not follow our required format specification.
4.2.3 Validation
Design. We conduct the ablation study for both the PoC generation model and the validation workflow, respectively. For the PoC generation model, we com-
Resolved Rate (%)
42.5
pared four variants: (1) Val-no-assert-Baan model, trained with reasoning data from Claude-3.7-Sonnet without filtering, (2)
42
.0.4
40
coo--asset--ooolltc trained using both Sauue-3.7--oniie and o4-mini reason- ing data without filtering; (3) Val-no-assert model only, and (4) Co-PatcheR- Val: Val-asser model + Val- no-assert model. We integrated each model into our validation pipeline and measured their resolved rates on an identical set of 20 patch candidates produced by our generation component. higher resolved rate indicates more effective validation.
37.0
36-
35.6
34
Model Varlants
Workflow Variants
Mal-no-assert-SSee
Do-Patchel-Koddyn Co Co-PatchRR-NooCC Co-PatchaR-Val
Val no NoFirt
Co- Co-PatcheR-val
Figure 4:
The resolved rate for different vali- dation models and validation workflow.
To evaluate our validation workflow design, we test three strategies with the same Val-assert model + -Val-no-assert model: (1) Co-PatcheR- Val, which applies the whole workflow, (2) Co-PatcheR- NoPoC, which omits PoC testing and relies solely on functionality tests and majority voting, and (3) So-PatcheR-NoDyee which applies majority voting directly to patch candidates without any dynamic testing. Each workflow also processed the same set of 20 patch candidates for fair comparison.
Results. Figure44present the comparative performance of different validation models and workflows. First, using two models performs better than only having Val-no-assert model, confirming the better PoC diversity. Second, Val-no-assert model outperforms Val-no-assert model-NoFilter, confirming the generalizable effectiveness of data filtering strategy across all components. Comparing Val-no-assert model-NoFilee with Val-no-assert model-Base further justifies the necessity of having diverse PoCs in training data, which guide our model to learn generate multiple PoCs for the same issue. The results in Figure 4 further show the necessity of having both PoC tests and function tests during validation. In Appendix D we show that when having patch candidates, majority vote is more effective than the outcome reward model used in SOTA agents [48] and even Claude-3.7-Sone As such, we stick to the majority vote as the final patch selection.
4.3 More Ablation Study and Sensitivity Test
We use our generation model to conduct the ablation study on data size, model size, and cesting-phase scaling strategy. The results of the other two components consistent (Appendix C).
Data size. We randomly sample a subset of 500 and IK cases from our current 2K training set and train two models using our proposed recipe. We report the Pass @ 1 performance of these models in Figure [5a The result shows that the performance increases more as the parameters grow from 500 to IK than from 1K to 2K. As shown in Appendix C.2, the model performance for localization no longer increases as we further increases the training data to 5K. As such, we select 2K as our final training data size. The findings show that, for small models, continually adding more data does not guarantee better performance (given the risk of overfitting). We further train a non-reasoning model for patch generation (SFT with ground truth patches). Our result shows that a non-rcasoning model trained
8

---

Resalved Rate (%6
342
65
...
Co Patchek passgNN
Co-PatcheR besigN
15.2
31.2
04-miri
20
29.8
Resolved (%)
6.-
Sonnet passg]
55-
20
20.4
210
45
42C
432
46.gg.
15
Data Size Variants
Model Size Variants
40
35
31.2
14B(Co-PatcheR)
Gen Gen-1000
228
10
20
60
den-2000(Co-Patcen
Number Samples (log scale)
(a) The pass@1 resolved rate of different training data/model size for validation.
(b) The resolved rate for of patch candidates.
Figure 5: More ablation studies the generation component.
with 2K training data performs even worse than our reasoning model trained with 500 samples. further shows that the reasoning model is more data-efficient.
Model size. We change model family with different sizes) and retrain our patch generation model with the same training data. The Pass@1 1 results in Figure 5a show that larger model indeed improves the performance. However, considering that the improvement of the 32B model over the 14B model is not significant, still choose the smaller one.
base model to Qwen-2.5 2.5n--22.-Coor-755 and Qwen-2.5 2.5m-22.5-caa 2.5-Coder-32B (same
Testing-phase scaling. We test two scaling performances. We fix the output context limit and ask the model to generate K=1, 10, 20, 40, 60, and 80 candidate patches. For each setting, we 1) compare the pass@ K resolved rate (whether correct patch is the generated candidates) to obtain the upper bound of patch generation: 2) run validation to select the final patch (best @ K) to assess the upper bound of Co-PatcheR. As shown Figure [bb] increasing the sample numbers can prompt the model generate more diverse patches, which increases the chances of hitting the correct one. This validates our arguments that small models with many samples can reach a similar performance to large models with fewer samples (without requiring significantly more computing, the models much smaller). Increasing sample numbers can also help the system as a whole; however, having too many samples will add burden to validation and may jeopardize the validation accuracy.
Discussion
Rationalization does not always help. aath--generation training data collection, we tried a rationalization scheme [57]: We provide de the teacher model with the ground-truth patch and force to generate a reasoning without mentioning the ground truth patch. When context insufficient, the model invents latent details (c.g., suggesting likely some ffnecifnn that is not in the context), causing the student model to learn hallucinated patterns. Of ten instances that Co- PatcheR originally solved but fail after fine-tuning with the reasoning data, six fail due hallucinated identifiers. Thus, rationalization can degrade patch-gened generrttnerttion performance.
Component specific models vs. one model. In this paper, we argue that to minimize the model sizes, we need to train models specific individual components. However, a counterargument for promoting one end-to-end model could be that all three tasks work on the same codebase, and the knowledge about the codebase can be shared across tasks. Although we acknowledge the validity of this argument, we do not take this route as we aim to push the limits for small models, and existing works following this methodology show limited performance. Future works can explore the efficient training methods and proper model sizes for such unified model.
Limitations and future works. First, designing specific and effective reward functions requires non-trivial effort. We defer to future work to explore effective RL methods to continue training our current models and see if the performance can further improved. Second, given our focus on the model side, the current patching leverages simplified agent scaffold without complex tool calls. We will further enrich the agent with more tool calls and train specified models for tool call planning. Third, with large samples, our localization and generation components can reach the performance of SOTA commercial models. Future works will explore how design more effective validations to pinpoint the correct patch from many candidates.

---

Conclusion
In this paper, we propose Co-PatcheR, a novel software patcher with collaborative small reasoning models. We develop unique training recipe for models in three major components of the patching pipeline (localization, generation, and validation). Our results show that Co-PatcheR achieves one of the highest resolved rate among patchers with customized models with the smallest models, outperforming many commercial LLM-based patchers. Our ablation studies validate the key designs in our proposed recipes well our choices data, model, and testing phase scaling strategy.
Acknowledgements
This work was supported in part by ARI. Grant INF-233---- We gratefully acknowledge the support of FAR AI, OpenAI, Anthropie, Google and Berkeley RDI.
References
[1] Saranya Alagarsamy, Chakkrit Tantithamthavorn, Chetan Arora, and Aldeida Aleti Enhancing large language models for text-to-testcase generation. arXiv preprint 2411::2002./99100, 2024.
[2] Amazon. Amazon q, 2024. Amazon Q.
[3] Anthropic. Claude 3.5 sonnet, 2024.
[4] Anthropic. Claude 3.7 Sonnet and Claude Code. http:///wwww.anthhpi...oom
claude- Saade-3-7-aa 3-7- 2025.
[5] Antonis Antoniades, Albert Örwall, Kexun Zhang, Yuxi Xie, Anirudh Goyal, and William Wang. Swe-search: Enhancing software agents with monte carlo tree search and iterative refinement. arXiv preprint arXiv:2410.20285, 2024.
[6] CodeR. Coder, 2024. CodeR.
[7] Composio. Composio swe-kit, 2024. Composio.
[8] devlo. devlo, 2024. devlo.
[9] Yangruibo Ding, Marcus J Min, Gail Kaiser, and Baishakhi Ray. Cycle: Learning to self-refine the code generation. Proceedings of the ACM on Programming Languages, 8(00PSLA1):39- 418, 2024.
[10] Yangruibo Ding, Jinjun Peng, Marcus Min, Gail Kaiser, Junfeng Yang, and Baishakhi Ray. Semcoder: Training code language models with comprehensive semantics reasoning. Advances Neural Information Processing Systems, 37:60275 00002775-6088,..8 2024.
[11] Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. Retool: Reinforcement learning for strategic tool use in Ilms. arXiv preprint arXiv:2504.1 1536, 2025.
[12] Yanjun Fu, Ethan Baker, Yu Ding, and Yizheng Chen. Constrained decoding for secure code generation. arXiv preprint arXiv:240S.004 00218, 2024.
[13] Globant. Globant code fixer agent, 2024. Globant.
[14] gru. gru, 2024. gru.
[15] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501.12944 2025.
[16] Hossein Hajipour, Lea Schönherr, Thorsten Holz, and Mario Fritz. Hexacoder: Secure code generation via oracle-guides synthetic training data. arXiv preprint arXiv:2409.06-4// 2024.
10

---

[17] Jingxuan He. Mark Vero, Gabriela Krasnopolska, and Martin Vechev. Instruction tuning for secure code generation. arXiv preprint arXiv:2402.09499, 2024.
[18] Jacob Hoffmann and Demian Frister. Generating software tests for mobile applications us- ing fine-tuned large language models. In Proceedings of the 5th ACM/IEEE International Conference on Automation of Software Test (AST 2024), pages 16-77,2024
[19] Binyuan Hui, Jian Yang. Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jia- jun Zhang, Bowen Yu, Keming Lu, et al. Qwen2.5-coder technical report. arXiv preprint arXiv:2409. 12686, 2024.
[20] IBM. Ibm research swe- 1.0, 2024. IBM Research SWE-1.0.
[21] Nan Jiang, Xiaopeng Li, Shiqi Wang, Qiang Zhou, Soneya Hossain, Baishakhi Ray, Varun Kumar, Xiaofei Ma, and Anoop Deoras. Ledex: Training Ilms to better self-debug and explain code. Advances in Neural Information Processing Systems, 37:35517-35544 2024.
[22] Carlos Jimenez, John Yang. Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan, Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770, 2023.
[23] Dacheng Li, Shiyi Cao, Chengkun Cao, Xiuyu Li, Shangyin Tan, Kurt Keutzer, Jiarong Xing, Joseph E Gonzalez, and Ion Stoica. S*: Test time scaling for code generation. arXiv preprint arXiv:2502.14338 14382, 2025.
[24] Hongwei Li, Yuheng Tang, Shiqi Wang, and Wenbo Guo. Patchpilot: A stable and cost-efficient agentic patching framework. arXiv preprint urXiv:2502.02747, 2025.
[25] Yizhou Liu, Pengfei Gao, Xinchen Wang, Jie Liu, Yexuan Shi, Zhao Zhang, and Chao Peng. Marscode agent: Ai-native automated bug fixing. arXiv preprint arXiv:2409.0088: 2024.
[26] Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma. Qingwei Lin, and Daxin Jiang. Wizardcoder: Empowering code large language models with evol-instruct. arXiv preprint arXiv: 23066. 08568, 2023.
[27] Yingwei Ma, Rongyu Cao, Yongchang Cao, Yue Zhang, Jue Chen, Yibo Liu, Yuchen Liu, Binhua Li, Fei Huang, and Yongbin Li. Lingma swe-gpt: An open arXiv:2411.00033 2024. language model for automated software improvement. arXiv preprint
[28] Yingwei Ma. Binhua Li, Yihong Dong, Xue Jiang, Rongyu Cao, Jue Chen, Fei Huang, and Yongbin Li. Thinking longer, not larger: Enhancing software engineering agents via scaling test-time compute. arXiv preprint arXiv:2503.23803. 2025.
[29] Zexiong Ma, Chao Peng, Pengfei Gao, Xiangxin Meng, Yanzhen Zou, and Bing Xie. Sorft: Issue resolving with subtask-oriented reinforced fine-tuning, arXiv preprint arXiv:2502.20127, 2025.
[30] Ivan Mistrk, John Grundy, Andr Hoek, and Jim Whitehead. Collaborative Software Engineering. Springer Publishing Company, Incorporated, 1st edition, 2010.
[31] Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Fei-Fei, Hannaneh Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candes, and Tatsunori Hashimoto. sl: Simple test-time scaling. arXiv preprint arXiv:2501.19339 2025.
[32] OpenAl. Hello, GPT-4o, 2024.
[33] OpenAl, Introducing 03 and 04-mini, 2024.
[34] OpenAl. OpenAl 03-mini, 2024.
[35] Siru Ouyang, Wenhao Yu, Kaixin Ma, Zilin Xiao, Zhihan Zhang, Mengzhao Jia, Jiawei Han, Hongming Zhang, and Dong Yu. Repograph: Enhaneing ai software engineering with repository- level code graph. arXiv preprint arXiv:2410.14688 2024.

---

[36] Jiayi Pan, Xingyao Wang, Graham Neubig, Navdeep Jaitly, Heng Ji, Alane Suhr, and Yizhe Zhang. Training software engineering agents and verifiers with swe-gym. arXiv preprint arXiv:2412.21139, 2024.
[37] Mohammadreza Pourreza, Shayan Talaei, Ruoxi Sun, Xingchen Wan, Hailong Li, Azalia Mirhoseini, Amin Saberi, Sercan Arik, et al. Reasoning-See sql: Reinforcement learning with sql tailored partial rewards for reasoning-enbanced enhanced text-to-sql. arXiv preprint arXiv:2503.23157, 2025.
[38] Archiki Prasad, Elias Stengel-Eskin, Justin Yao Chen, Zaid Khan, and Mohit Bansal. Learning to generate unit tests for automated debugging. arXiv preprint arXiv:2502.01619, 2025.
[39] Qwen Team. QwQ-32B: Embracing the Power of Reinforcement Learning. https://qwenln github.io/blo///oee / 2025.
[40] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. Advances in Neural Information Processing Systems, 00553328-537445 2023.
[41] Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, al. Code Ilama: Open foundation models for code. arXiv preprini arXiv:2308. 12950, 2023.
[42] Haifeng Ruan, Yuntong Zhang, and Abhik Roychoudhury. Specrover: Code intent extraction via llms. arXiv preprint a:Xiv:2408. 022332 2024.
[43] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1077.03377, 2017.
[44] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprini arXiv:2402.0333 03300, 2024.
[45] Wei Shen and Chuheng Zhang. Policy filtration in rlhf to fine-tune Ilm for code generation. arXiv preprint arXiv:2409.06957, 2024.
[46] NovaSky Team. Sky-tl: Train your own ol preview model within S450. https:/nnovasky- ai. gl 2025. Accessed: 2025-01-09.
[47] Khiem Ton, Nhi Nguyen, Mahmoud Nazzal, Abdallah Khreishah, Cristian Borcea, NhatHai Phan, Ruoming Jin, Issa Khalil, and Yelong Shen. Sgcode: A flexible prompt-optimizing system for secure generation of code. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, pages 5078-5080) 2024.
[48] Xingyao Wang, Boxuan Li, Yufan Song, Frank F Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, al. Openhands: An open platform for ai software developers as generalist agents. arXiv preprint 2rXiv:2407.16741 2024.
[49] Yubo Wang, Xiang Yue, and Wenhu Chen. Critique fine-tuning: Learning to critique is more effective than learning to imitate. arXiv preprint arXiv:2501.17703, 2025.
[50] Yuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux, Lingming Zhang, Daniel Fried, Gabriel Synnaeve, Rishabh Singh, and Sida I Wang. Swe-rl: Advancing Ilm reasoning via reinforcement learning on open software evolution. arXiv preprint arXiv:2502. / 18449, 2025.
[51] Jim Whitehead. Collaboration in software engineering: A roadmap. In 2007 Future of Software Engineering. FOSE '07, page 214-225, USA, 2007. IEEE Computer Society.
[52] Yutong Wu, Di Huang, Wenxuan Shi, Wei Wang, Lingzhe Gao, Shihao Liu, Ziyuan Nan, Kaizhao Yuan, Rui Zhang, Xishan Zhang, et al. Inversecoder: Unleashing the power of instruction-tuned code Ilms with inverse-instruct. arXiv preprint arXiv:2407.00 25000000000 2024.
[53] Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. Agentless: Demystifying llmebased software engineering agents. arXiv preprint arXi::2407.01488 2024.
12

---

[54] Chengxing Xie, Bowen Li, Chang Gao, He Du, Wai Lam, Difan Zou, and Kai Chen. Swe-fixer: Training open- Ilms for effective and efficient github issue resolution. arXiv preprint urXiv:2501.05045 2025.
[55] Xiangzhe Xu, Zian Su, Jinyao Guo, Kaiyuan Zhang, Zhenting Wang, and Xiangyu Zhang. Prosec: Fortifying code Ilms with proactive security alignment. arXiv preprint arXtv:2411112882. 2024.
[56] John Yang, Carlos Jimenez, Alexander Wettig. Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. Swe-agent: Agent-computer interfaces enable automated software engineering. Advances in Neural Information Processing Systems, 37:50528-5008 50652, 2024.
[57] Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah Goodman. Star: Bootstrapping reasoning with reasoning. Advances in Neural Information Processing Systems, 35:15476-15488, 2022.
[58] Kechi Zhang. Huangzhao Zhang, Ge Li, Jinliang You, Jia Li, Yunfei Zhao, and Zhi Jin. Sealign: Alignment training for software engineering agent. arXiv preprint arXiv:2503. /8455, 2025.
[59] Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. Autocoderover: Au- tonomous program improvement. In Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis, pages 1592-1604, 2024.
[60] Tianyu Zheng, Ge Zhang, Tianhao Shen, Xueling Liu, Bill Yuchen Lin, Jie Fu, Wenhu Chen, and Xiang Yue. Opencodeinteeprreeer:: Integrating code generation with execution and refinement. arXiv preprint arXiv: 14658, 2024.
[61] Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang. Runxin Xu, Y Wu, Yukun Li, Huazuo Gao, Shirong Ma, al. Deepseek- coder--:: Breaking the barrier of closed-source models in code intelligence. arXiv preprint arXiv:2406. 1931, 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
Main claims in abstract and introduction match our contributions.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We claim our limitations in 5
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
We do not have theoretical assumptions and proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We claim all details about evaluation in
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [Yes]
Justification:
We will open source the model and code as we said in abstract.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ /pipp/.cc/pullc//uuui wwwpc//ggue////eee CoodeSubeiiie ssi ssionnolli onnolicy)))) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We mention these
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We apply appropriate information about our experiments and the data.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
16

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of exccution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We mention these
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
We match the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work focus on software engineering application, would not include negative societal impact.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We do not have such risk.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [NA]
Justification:
We do not have such problem.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We do release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We do not involve these.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
We do not involve these.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
19

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [Yes]
Justification:
We mentioned the usage of LLMs in
4
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy for what should or should not be described.
20

---

A Training Details and Inference Efficiency Metric
Hlype--araaeeter We train all three customized models (Loc-Gen, Val-no-assert, Val-assert) using the same settings. Table summarrzes the key hyper--arameters.
Inference efficiency metric. we comprehensively evaluated the efficiency of Co-PatcheR, as shown in the table [] We log four kinds of metrics for average of every issue: end-to-end wall-clock time for one instance under our testing setup(5 root causes from the localization model and 4 PoCs from the validation model), tokens generated across all model calls (completion), latency for average one-shot model inference, and peak GPU memory. These metrics provide the efficiency baseline for Co-PatcheR throughput can be further improved through parallel processing of multiple instances.
NVIDIA L40S (48 GB) cards with VLLM deployment framework. The
Table 2: Training hyper- syper-parameters for the Qwen-2.5-Coder 14B model
Hyperparameter
Value
Peak learning rate
1x10-5
Warmup ratio
0.10 total steps
LR scheduler
Cosine decay (with 10% linear warmup)
Batch size (per GPU)
1 (effective batch size = 1x accumulations)
Weight decay
1 10-5
Number training epochs
3.0
Maximum sequence length
32768 tokens
Table 3: Inference efficiency metric comparation for Co-PatcheR and SWE-Fixer
Metric(per issuc)
Co-PatcheR
SWE-FFier
End-to-end wall-clock time
334.17 s
123.25
Model-only latency
44.01
103.08
Tokens generated
5,83 k
2.51 k
Peak GPU memory (per GPU)
44.64 GB
45.23 GB
Resolved Rate (%)
46.00
32.80
B Explanation about Validation Models
We design two format of validation test case generation, the assertion tests independently determine whether the bug is triggered, whereas non-assertion tests only record the output of the PoC and require an LLM to judge the output after execution.
Assertion tests follow the common unit test style, which would provide explicit oracle to the target issue-for example, assert sort([2,1]) == [1,2]. When such test fails, we know that the bug is highly likely to still exist. However, the downside is that writing correct oracle requires the model to infer an exact post-condition for the issue, which can be difficult.
Non Von-assertion tests relax this requirement by performing a general PoC script. These are similar to differential testing, where we compare execution outputs before and after applying the patch to determine fix effectiveness. They are generated to PoC according the issue description, being expected to reproduce the same observable symptoms. In the downstream workflow, we would ask LLM to make judgment according to the original issue description and the output of assssssttiinn tests. And another benefit from non-assertion tests rather than assertion tests is that non-assertion tests can capture the new bug introduced by the candidate patch by observing and analyzing actual output behavior.
We keep both kinds of tests for two reasons: Coverage Some bugs have well-defined expected values or exceptions that can be explicitly tested with assertions, while others manifest as subtle behavioral changes or output variations that are better captured through execution observation rather than rigid assertion checks. Using both styles ensures we can validate wider range of issues. Regression A patch might solve the original failure but introduce one. Because non-assertion
21

---

Table 4: Effect of reasoning length on patch generation (pass@1) on the
dataset.
Setting
Max reasoning tokens
pass 1 (%)
No-reason
0
40.6
Short-reason
2K
43.8
Long-reason
8K
44.2
tests observe the output of the program, they can detect these new bugs even though the old assertion still passes. The whole patching system can utilize this information for further patch refinement.
C Ablation Study
C.1 Whether Long Reasoning Help
Design. To compare the impact of reasoning length in the patching task, we ran ablation study on the SWE-bench-Verifee dataset using Claude-3.7-Sonne. For a fair comparison and to focus specifically patch generation capabilities, use GPT-4o localization results as consistent input as 4.2 to patch generation. We then varify the maximum reasoning output length, setting it to 0 tokens("noo no--oesson") 2K tokens ("short-reason") and 8K tokens (" ("long-reason"), and evaluate performance with the pass01 metric, which counts issues successfully resolved by the first generated patch.
Results. The pass01 results were 43.8% for the short reasoning setting(2K) and 44.2% for the long reasoning setting(8K), while 40.6% for the no-reasoning settings. These results show that considering the random deviation of the model inference, extending the reasoning budget beyond 2K tokens yields appreciable gain in our experiments.
C.2 Localization
We use our localization model to conduct the ablation study on data size, model size, and testing -paase scaling strategy.
Data size. We randomly sample a subset of 500, IK, and 2K cases from our 5K training set and train four models in total using our recipe. We report both the file-level and line-level localization performance of these models in Figure 6al The result shows that the performance keeps increasing from 500 to 1K and from 1K to 2K, and the model performance for localization no longer increases as we further increase the training data to 5K. As such, we select 2K as our final training data size. We further train non-reasoning model for localization (SFT with ground truth files and lines). Our result shows that non-reasoning model trained with 2K training data performs even worse than reasoning model trained with 500 samples. It further shows that the reasoning model is more data-cfficient.
Model size. We change our base model to 2.cn-e2.- and model family with different sizes) and retrain our model with the same localization training data. The localization results in Figure show that a larger model indeed improves the performance. However, considering that the improvement of the 32B model over the 14B model not significant, we still choose the smaller one.
(same
Testing-phase scaling. We two scaling performances here. The generation setting is kept fixed, whereas the localization model predicts N=1, 2, 4, 6, 8, 10 times for each issue, including both line-level and file-level. For each setting. we 1) fix the top files as file-level results and merge the line-level localization outputs as the line-level results accuracy. 2) generate 20 patches with the generation model with the different localization results as input, then record the pass @ 20 resolved rate(the percentage of which at least one of the 20 candidate patches fixes the issue) this serves as the upper bound of the localization module. To clarify the relationship between our localization and patching metrics, we first note that line-level localization accuracy is defined by whether the model's s returned lines fully cover all modified lines the golden "correct". In other words, our line-level metric is very strict: it says location is "correct" only if the predicted lines match the golden patch line for line. But an issue can fixed without copying the golden patch exactly- any edit that makes the tests pass is accepted. So patch solve the issuc
perfect coverage counts as
22

---

100
100
Accurscy (%)
90
12.0
95.861.
Accuracy
60
30.0 32.0 510
$1.8
66.0
40
File Level Accuracy
Line Level Accuracy
Loc 2000 NoReacon
Loc
Loc
File Level Accuracy Loc
Line Level Acturacy
Loc-500
Loc- (Cd
Loc-32B
(a) top@5 file- line-level accuracy versus training data size.
(b) The top@5 file- and line-level accuracy versus model size.
Figure 6: Ablation studies on data size (A) and model size (B) for localization.
Line-level acuraayy
Co-PatcheR pess@20
Localization Accuracy (%)
56.2
57.2
Reso Rate
50.4
40
60.4
48.0
51.5
54.6
-20
2
Number of Localization Samples
Figure 7: Resolved rate and line-level accuracy for # of localization samples.
even when its edited lines do not align with the golden patch, which is why the pass @ 20 success rate be higher than the measured line-level accuracy. As illustrated in Figure supplying more localization samples broadens the search space, boosting the chance that the correct file appears in the pass 20 candidates. This confirms our intuition that "wider" localization (more samples) can offset smaller model size just as "wider" generation does, without incurring the cost of a larger backbone. However, pushing the number N high again shifts the burden generation and can confuse the generation by a large number of false positives of code chunks, resulting in decrease in performance.
C.3 Validation
Combine training. In the original pipeline, train two separate PoC-generation models- Val- assert model and Val-no-assert model -and use them to create four PoCs (two of each type) for validating the candidate patches produced by the generator. To test whether single, mixed validator can do better, we merge the two training sets, train one unified model, and invoke with two prompt templates (assert no-assert) to generate the same four PoCs. Everything else is held constant: the localization and generation results, the dynamic tester, and the ranking heuristic. We then compare the best @ 20 resolved rate after validation has selected the final patch our result. Table inddicates that keeping the PoC generation model separate still yields the higher accuracy; combining the data into a single model does not translate into better PoC generation due lack of diversity.
Table 5: Split vs. unified PoC-generation models (best@ 20 on SWE-bench- (erbbennnh--eifie).
Training scheme
PoC prompts
best@20 (%)
Split models (Co-PatcheR)
2 assert no-assert
43.20
Unified model
2 assert 2 no-assert
42.60
23

---

48
46
50
44
42.6
42.6
45.0
bostg resolved 1922
Resolved
42
40.4
40.2
Besoloaed Bate (50
40-
36
36
Data Size Va riants
Model Size Variants
Velno a:sert-500
70
100o--assert11000
14B| Co- CotPchhe
Total PaCS Issue (2N)
(a) The best@ 20 resolved rate for different training data/model sizes for validation.
(b) The best@ 60 resolved rate for # of PoCs.
Figure More ablation studies on the validation component.
Data size. We randomly sample subset of 500 and 1K from our 2K training set and train in total of three models for our Val-no-assert models training. We report the best@20 final resolved rate performance of these models in Figure Sa The result shows that the performance keeps increasing from 500 and from 1K 2K. As such, we select 2K as our final training data size.
and Qwen-2.5 2.5-Coder--22 (same training data. The localization results in Figure Sa show that a larger model indeed improves the performance. However, considering that the improvement of the 32B model over the 14B model is not significant, we still choose the smaller one.
Model size, We change our base model to Qwen-2.5-Coder--
model family with different sizes) and retrain Val-no-assert model with the
Testing-phase scaling, We reuse the same 60 candidate patches from the generation and vary only the number PoCs supplied to the dynamic test. Specifically, we let both Val-no-assert and Val fal-assert generate N=1, 2, 3, 4, PoCs each, yielding 2N=2, 4, 6, 8, 10 PoCs per issue. For every 2N, measure the best 60( The final resolved rate of the patches that chosen from 60 cases) resolved rate after validation chooses single patch from the 60 candidates. As shown in Figure [8b] increasing from N=1 PoCs total) to N=2 (4 PoCs) per model brings a clear boost, but adding more PoCs provides further gains. The plateau occurs because the same validation model tends to emit highly similar PoCs; once two distinct checks are present, additional ones rarely catch new failures and instead lengthen the test phase without improving accuracy.
D Majority Voting vs. ORMs
Design. After dynamic testing, we often obtain several patch candidates that all pass the same subset of PoCs and functionality tests. Our default system selects the final patch via majority voting. To assess whether more reranking methods help, we keep every other component fixed-the 60 patch candidates, the four PoCs used for dynamic validation, and all hyper-parameters from and only swap the tie-breaking strategy: Majority Vote (what we use). ORM Score, we feed each surviving candidate into the ORM model from SWE-Gym [36) and pick the highest-scoring patch. Claude Vote, prompt Claude-3.7- 3.7-Sonnet with the issue description, the localization results, and the patch candidates, and let it rank the patches; the top suggestion is selected. This setup isolates the impact of the tie- brrakiiaing itself, because every method starts from exactly the same candidate pool and the same dynamic-testing evidence.
Results. With 60 patching candidates, majority voting achieved the best accuracy (46.00% best@) 1), ahead of the ORM score (36.40%) and the Claude vote (45.00%). Because majority voting cost- free(neither needs commercial credits nor computation resources), it remains the simplest and most effective option when many patches are still in play. Besides, its performance better than even the SOTA commercial LLMs when dealing with large pool, demonstrating its effectiveness.
24

---

Table 6: Effect of different tie- breaking strategies after dynamic testing.
Final patch selection
best@60 (%)
Majority Vote
46.00)
ORM Score
36.40
Claude Vote
45.00
E Prompts for Each Component
E.1 Localization Prompt
E.1.1 File Localization Prompt
User Prompt:
Please look through the following GitHub problem description and Repository structure, and provide list of files that one would need to edit the problem.
### GitHub Problem Description ### <problem_statement ###
### Repository Structure ###
[structure} ###
After analyzing the problem, provide the full path and return at most @file_numben files. The returned files should be separated by new lines, ordered by most least important, and wrapped with For example:
filel.py file2.py
E.1.2 Line Localization Prompt
User Prompt:
Please review the following GitHub problem description and relevant files, and provide of locations that need to be edited to fix the issue.
The locations should include exact line numbers that require modification. Pay attention! You should identify the method responsible for the core functionality of the issue. Focus on areas that define or enforce foundational behavior rather than case-specific issues.
### GitHub Problem Description ### garoblem_statementt
###
(file_contents)
###
{last_ saatt_search_reautts
After analyzing the problem, please provide the class name, function method name, the exact line numbers that need to be edited.
25

---

If you want to provide the line number, please give number in the middle every time. If you need to edit multiple classes or functions, please provide the function names or the line numbers in the class.
You should always include class or function; do not provide just the line numbers without the class function name.
If you want to include a class rather than a function, you should always provide the line numbers for the class.
Here is the format you need to strictly follow, don't return any code content or other suggestions, don't forget the "***"
### Examples:
full hal_pathl//e l/filel.py
class: MyClassl
line: 51
full patlp//fhffle.pp
function: MyClass2.my_metnes
line: 12
full_ path3/file3..y
function: my_function
line: 24
line: 156
E.2
Generation Prompt
E.2.1
Patch generation Prompt
User Prompt:
We are currently solving the following issue within our repository.
You are maintainer of the project. Analyze the bug thoroughly and infer the underlying real problem, using your inherent knowledge the project. Focus on resolving the root logic issue rather than suppressing symptoms.
Note that if the issue description mentions file names arguments for reproduction, the fix must be generalized and not restricted to specific arguments. If the issue description includes recommended fix, adapt it to align with the codebase's style and standards. Ensure your fix maintains structural integrity, considering interactions across code sections, nested structures, function calls, and data dependencies. Prefer solutions resilient to future structural changes or extensions.
The following is the issue description:
BEGIN ISSUE-
{problem. statement)
END ISSUE
Below are the code segments from multiple files relevant to this issue. Each file is clearly marked. Decide carefully and only modify necessary segments. Preserve original indentation and formatting standards strictly.
BEGIN FILES (content}
26

---

END FILES-
Now, carefully analyze the files above. Determine which specific file segments require modifications and provide your edits using the following structured format for easy parsing:
<<< MODIFIED FILE: path/to/filename >>>
python
<<<<<<< SEARCH
from flask import Flask
import math
from flask import Flask
>>>>>>> REPLACE
<<< END MODIFIED FILE >>>
Please note that the *SEARCH/EEPACCE* edit REQUIRES PROPER INDENTATION. you would like to add the line print(x)', you must fully write that out, with all those spaces before the code! Wrap the *SEARCHREPLACEE* edit in blocks python...""
E.2.2 Patch Critique Prompt
User Prompt:
Please continue working this code patching work. You need review the patch thoroughly to determine if it successfully fixes the issue without introducing any new bugs, and while handling all possible edge cases.
If you determine that the patch is correct and complete, tell me you confirm that the patch succeeded.
If you think the patch is incomplete, give me the reason and potential fixing suggestions.
You need to think:
What edge cases can break the patch? Consider complex cases such as nested structures and recursive patterns. For example, if the patch fixes issue with an empty string, consider whether None, an empty list, or partially empty data structures might also trigger the bug. 2. Why the patch is incomplete or correct, whether the interaction between the patched part and other parts of the codebase can be handled properly
3. whether the patch only fixes the issue for the specific case mentioned in the issue description or for all similar cases
4. whether the patch follows the codebase's style and standards, using the proper variable types, error or warning types, and adhering to the established format
If the patch is perfect, tell me why. If the patch is unfinished or wrong, give me the reason and patch suggestions.
At the end, you should give me the critical result from you, if yes, give me "Conclusion: Right". otherwise give me "Conclusion: Wrong".
27

---

E.3
Validation Prompt
E.3.1
PoC Generation Prompt
User Prompt:
When generating a PoC script, follow these steps in order**:
[Optional] Always include assertions the PoC to make the failure obvious when the script
executed.
[Optional] The assertion should fail if the bug is present, and pass if the bug is not present.
T+rr to extract an existing PoC from the issue description
* Scan the **GitHub issue description* for Python code blocks inline snippets that appear to reproduce the bug.
* If such snippet exists, ** use verbatim as the base PoC** and only make the minimal
edits needed to run:
Remove interactive prompts ('>>>" 'S', 'In []:") and any captured output lines.
Add any missing 'mmport' statements.
Convert Python2 syntax to Python3, if present.
Merge multiple fragments into single runnable file in their original order.
*f no valid PoC can be extracted, write one yourself**
* Use the *specific* classes, functions, or code paths named in the issue to trigger the bug.
Keep the script minimal-just enough to demonstrate the failure (e.g., an "assert" expected exception, or a visibly incorrect result).
* **ennera rules for both cases**
The PoC **must be single, self-contained Python3 file**
* If the issue description includes other languages or shell commands, recreate their behavior in Python (e.g., with subprocess or file operations).
If the snippet refers to external files, create them program matically inside the script.
* Always include print()" or "assert" statements the failure is obvious when the script is executed.
**uutput * Return **xxactl*** Python code wrapped in triple backticks, with no other text.
ppython [[poc_code here}}
### Context Provided to You
(last_ clast_time__oc_codde
(execution_outpute
### GitHub Issue Description
Begin Issue Description
{problem_ste (aaieemmssttttett
End Issue Description
E.3.2 PoC Critique Prompt
User Prompt: You code reviewer for GitHub project. Your task is to evaluate whether patch
28

---

successfully resolves an issue described in GitHub issue.
You will be given the issue description, the PoC (Proof of Concept) code that demonstrates the issue, and the PoC execution output before and after the patch.
Your goal is to determine if the patch resolves the issue. Please respond with "Yes" or "No" and provide brief explanation of your reasoning.
You should not assume that there is other output that is not shown in the Poc Output. The Poc Output is the only output you should consider. You should not assume that plot is generated by the Poc.
"YYes" means the patch successfully fixed the issuc. "No" means the patch did not successfully fix the issue, either because the issue still exists or because the patch introduced new issues.
### Raw Issue Description #### {issue_desse desebupteecrritioo
### Poe code ###
Here is the PoC code that demonstrates the issue: {poc_code}
### Poc Output before the patch ### aed__eeetiiioouupuuuii
### Poc Output after the patch ### fnev_execution_orue
**Response *
Example 1:
<judgement> Yes </judgement>
<explanation> The patch successfully fixed the issue since </explanation>
Example 2:
<judgement> No </judgement>
<explanation> The patch did not successfully fix the issue since <lexplanation>
29


Reviews:
[{"review_id": "Jp5gUi2Umk", "reviewer": "Reviewer_ggJ3", "summary": "This paper introduces Co-PatcheR, a collaborative patching framework composed of small, component-specific reasoning models for software patching. The authors propose decomposing the patching pipeline into three components: localization, patch generation, and validation. Co-PatcheR uses a single Loc-Gen model for both localization and generation, and two separate models for diverse issue-reproducing test case (PoC) generation and validation. The authors design tailored task formulations and training recipes for each component, including a lightweight reasoning data construction method and a model distillation strategy. Experimental results on the SWE-bench-Verified benchmark show that Co-PatcheR achieves a 46% resolved rate using only 3×14B models and 6K training samples, outperforming larger baselines while being substantially more parameter- and data-efficient.", "questions": "Q1. Could the authors report cross-framework results (e.g., Co-PatcheR + Agentless vs. SWE-RL-70B + Agentless, and Co-PatcheR + PatchPilot vs. SWE-RL-70B + PatchPilot) to better disentangle the effects of model architecture and agent framework?\n\nQ2.Could the authors provide inference efficiency metrics (e.g., latency, total generated tokens, or wall-clock time) to assess practical deployment cost?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength\n1. The idea of decomposing patching tasks into collaborative reasoning components, each trained with customized objectives and data, is practically valuable, especially in resource-constrained scenarios.\n2. The method achieves a 46% resolved rate on SWE-bench-Verified, surpassing large end-to-end baselines while using fewer parameters and less data.\n3.The authors provide extensive ablations that validate their architectural and training choices, such as the joint Loc-Gen model, multi-model PoC generation, critique-based generation, and data/model scaling considerations.\n\nWeakness\n1. Given that, when using the same model (Claude-3.5), the agent framework PatchPilot achieves better performance than the agentless setting, it would be valuable for the authors to conduct additional cross-framework evaluations. For example, comparing Co-PatcheR + Agentless with SWE-RL-70B + Agentless, or Co-PatcheR + PatchPilot with SWE-RL-70B + PatchPilot. Such comparisons would help disentangle the respective contributions of the model architecture and the agent framework, providing fairer and more convincing evidence to support the authors’ insight that multiple expert small models can outperform a large end-to-end model on SWE-bench.\n2. The authors present an encouraging result that three small models, requiring fewer parameters and less training data, can outperform a large end-to-end model on SWE-bench. However, beyond training cost, inference cost is equally critical for practical applications. The current study primarily focuses on generation quality metrics (e.g., pass@1, pass@K) without reporting inference time or the number of generated tokens. In deployment scenarios, inference latency and token generation efficiency are important considerations — particularly for approaches like Co-PatcheR, which rely on generating large numbers of candidate patches to match the performance of larger models. It would strengthen the paper to include an evaluation or at least a discussion of inference efficiency alongside generation accuracy.", "was_revised": true, "final_justification": "The authors’ response has adequately addressed my concerns. Taking into account the comments from other reviewers and the corresponding responses from the authors, I have decided to keep my rating at 4.", "rebuttal": "We thank the reviewer for the positive and constructive comments. Please see below for our responses. \n\n## 1. Cross-framework evaluation experiments\n\nFollowing the reviewer’s suggestion, we integrate the Co-PatcheR models into the agentless framework. We measure best@60 on the SWE-bench-Verified benchmark as a balance of performance and efficiency. All other hyperparameters are kept identical to SWE-RL to enable an apple-to-apple comparison.\n\n| Method | Resolved Rate | \n|:------:|:-------------:| \n| Co-PatcheR + PatchPilot | 46.00% | \n| Co-PatcheR + Agentless | 45.20% | \n| SWE-RL-70B + Agentless | 41.00% |\n\nEven with the changing of the testing framework, Co-PatcheR keeps a 4.20% advantage over the 70B SWE-RL model final results, showing the strengths of Co-PatcheR models.\n\n We also observe a 0.80% drop from Co-PatcheR+PatchPilot (46.00%) to Co-PatcheR + Agentless. This is because the models are trained based on the agentic data generated using the PatchPilot framework, which are naturally more suitable for use in PatchPilot. Besides, the validation strategy in PatchPilot and Agentless is different: for example, we utilize two kinds of PoCs(assertion and non-assertion) in PatchPilot, which also causes some performance discrepancy.\n\nAlso, we would like to clarify that SWE-RL has not released its weights, so we cannot re-run it in any agent framework. That’s why we can not do SWE-RL + PatchPilot testing, but only compare with the results they mentioned in their paper.\n\n## 2. Inference efficiency metrics \n\nFollowing the reviewer’s suggestion, we comprehensively evaluated the efficiency of Co-PatheR, as shown in the following table. We log four kinds of metrics for an average of every issue: end-to-end wall-clock time for one instance under our testing setup in the paper(5 root causes from the localization model and 4PoCs from the validation model), tokens generated across all model calls (completion), latency for average one-shot model inference, and peak GPU memory. These metrics provide the efficiency baseline for Co-PatcheR on two NVIDIA L40S (48 GB) cards with a VLLM deployment framework. The throughput can be further improved through parallel processing of multiple instances.\n\n|     Metric (per issue)     | Value (mean) |\n|:--------------------------:|:------------:|\n| End-to-end wall-clock time |    334.17 s    |\n|     Model-only latency     |    44.01 s    |\n|   Tokens generated   |    5.83 k    |\n| Peak GPU memory (per GPU)  |     44.64 GB    |"}, {"review_id": "bUJprHiqMm", "reviewer": "Reviewer_GmxY", "summary": "The authors propose decomposing the task of software repair into explicit steps: localisation, patch/repair, and validation. Furthermore, they argue that the expected capabilities and inputs at each stage can vary, and thus propose using different models tailored to each step. They identify two sets of requirements: for localisation and generation, the model requires access to the issue description and entire code base (especially for localisation); for validation, the model requires a deeper understanding of the issue description; however, it can forgo access to the full code base, instead focusing on the method under test. Further, they argue that in the interest of a fairer validation, the validation model should not have access to patch candidates. To train their models, the authors select problems across repositories that do not overlap with their test set and that over a range of problem difficulty levels. They then design and break down the problems at each phase. For localisation, they break down the task into file localisation and line localisation. For generation, they integrate patch refinement by combining the generation step with a critique step that provides feedback. For validation, they propose to generate unit tests with and without assert statements, arguing that this facilitates testing a wider range of program behaviours. The proposed final tool, Co-PatcheR, manages to achieve competitive results (compared to some commercial models) and improves over specialised open-source models at a greatly reduced sample cost, achieving a 46% resolved rate on SWE-bench-verified using 6K training examples.", "questions": "Q1 (repeat from W2)  It is unclear what the authors mean by assert and assert-less tests. Is checking the expected outcome considered an assert? If yes, what _exactly_ are the assert-less tests verifying, program crashes? The lack of clarity around writing these details confused me regarding the justification of the two validation models, as it is unclear what kind of output they produce. Further, it is unclear to this reviewer how writing unit tests that have asserts fundamentally covers different program paths than those that do not; is the latter trying to approximate fuzzing?\n\nQ2 I remain unconvinced by the merging of Localisation and Generation. The Generation models would require different types of contexts compared to localisation: related/co-changed files, called files, a table of available symbols (APIs), etc. These feel materially different from the full code-base view of Localisation unless Localisation is reused to find the relevant files/symbols. Was a set-up where the two stages use different models considered? If not, is it due to a higher priority during model design being given to reducing model size/parameter count?\n\n- Nit: Page 4, Line 138, you refer to test cases as vulnerable? Is this from an older iteration of the paper where the focus was on security testing? Also, same location, you introduce vulnerable test cases as PoC (proof of concept?), this seems highly unusual and does not match the words abbreviated.\n- Writing Nit: §3.2.1 \"following recent work by S1 [31]\" -> \"following recent work in S1 [31]\" or \"following recent work by Muennighoff et al. [31]\" as \"S1\" is the tool/method name.", "limitations": "Limitations are reasonably addressed.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "+ S1 Strong results on SWE-bench-verified at a considerably reduced sample cost.\n+ S2 Modulo the remark for W2 below, justified decomposition into sub-problems and sub-models.\n+ S3 Strong results on sub-problems (localisation), not just the end-to-end task\n- W1[writing] The paper makes claims about testing in general (page 4, lines 125-127), arguing that testing does not require access to the full code base. This claim is true for unit testing, which is the scenario in the paper (though never stated explicitly), but it is not true in all testing settings: acceptance, integration, interaction testing as three examples require access to multiple units and in an end-to-end integration test suit, knowledge of the full code-base would be required. The problem setting should be clarified to be that of unit testing.\n- W2 It is unclear what the authors mean by assert and assert-less tests. Is checking the expected outcome considered an assert? If yes, what _exactly_ are the assert-less tests verifying, program crashes? The lack of clarity around writing these details confused me regarding the justification of the two validation models, as it is unclear what kind of output they produce. Further, it is unclear to this reviewer how writing unit tests that have asserts fundamentally covers different program paths than those that do not; is the latter trying to approximate fuzzing?", "was_revised": true, "final_justification": "The authors have addressed my concerns in the rebuttal phase, and I believe this paper is of NeurIPS quality. I maintain my positive score.", "rebuttal": "We thank the reviewer for the positive and constructive comments. Please see below for our responses. \n\n## 1. Clarification of unit testing\nThank you for pointing this out. Yes, our work focuses exclusively on unit-test–level validation, where each generated test exercises a single function or script in isolation. We will make sure to clarify this in the next revision to avoid confusion with other test levels (integration, acceptance, etc.).\n\n## 2. Difference between assertion tests and non-assertion tests\nSorry for the confusion. The main purpose of our PoC testing is to synthesize test cases for validating whether a candidate patch truly fixes the bug described in the issue. Both assertion tests and non-assertion tests serve this purpose, but in different styles. \n\nThe assertion tests **independently determine whether the bug is triggered**, whereas non-assertion tests **only record the output of the PoC and require an LLM to judge the output after execution**. \n- **Assertion tests** follow the common unit test style, which would provide an explicit oracle to the target issue—for example, assert sort([2,1]) == [1,2]. When such a test fails, we know that the bug is highly likely to still exist. However, the downside is that writing a correct oracle requires the model to infer an exact post-condition for the issue, which can be difficult.. \n- **Non-assertion tests** relax this requirement by performing a general PoC script. These are similar to differential testing, where we compare execution outputs before and after applying the patch to determine fix effectiveness. They are generated to a PoC according to the issue description, being expected to reproduce the same observable symptoms. In the downstream workflow, we would ask LLM to make a judgment according to the original issue description and the output of non-assertion tests. And another benefit from non-assertion tests rather than assertion tests is that non-assertion tests can capture the new bug introduced by the candidate patch by observing and analyzing actual output behavior.\n\nSo we keep both kinds of tests for two reasons:\n- **Coverage** –  Some bugs have well-defined expected values or exceptions that can be explicitly tested with assertions, while others manifest as subtle behavioral changes or output variations that are better captured through execution observation rather than rigid assertion checks. Using both styles ensures we can validate a wider range of issues.\n- **Regression** – A patch might solve the original failure but introduce a new one. Because non-assertion tests observe the output of the program, they can detect these new bugs even though the old assertion still passes. The whole patching system can utilize this information for further patch refinement.\n\n## 3. The combination of localization and patch generation\nThere are two reasons why we combine the localization and generation in one model:\n\n- **Share Core Abilities**: Both subtasks require fundamental capabilities in understanding the issue description to identify what needs to be fixed, and reasoning over the current codebase to determine the appropriate intervention points and methods.\n- **Task Interdependence**: A better understanding of how to fix a bug helps clarify what context needs to be retrieved for localization, while precise localization provides essential context for accurate patch generation. Localization applies these abilities to locate faulty lines; generation applies the same context to edit them. Because the tasks are sequential, training them together allows the model to reuse intermediate reasoning and to learn a smoother “find-then-fix” workflow without growing the parameter count.\n\nWe further experimented with two separate models—one fine-tuned for localization, the other for generation. Testing the results of pass@20(whether a correct patch is in the 20 generated candidates) on the SWE-bench Verified split as we did in ablation studies, to isolate the framework factor. The scores were 55.40% for the split setup and 56.20% for the joint model. The joint model performed at least as well, sometimes better.\n\n## 4. Inaccurate phrasing\nThank you for spotting the typo on page 4, line 138. We will remove the “vulnerable” description because we are not only focusing on security problems, and we will also clarify PoC as ‘proof-of-concept’ to avoid ambiguity. And we will update the correct citation name in §3.2.1."}, {"review_id": "r0cWbydt8v", "reviewer": "Reviewer_ZdqA", "summary": "This paper presents Co-PatcheR -- an approach to train smaller task-specific LLMs to target software engineering tasks. The authors focus on designing smaller training data for specific tasks like localization, patch generation, and test generation to train smaller local models. The core technique involves using the smaller models to complete individual tasks and finally solve the end-to-end task.", "questions": "1. Did the authors try using two separate models for patch generation and localization and evaluating the performance?\n2. What is and what are the benefits of using with and without assertion tests?\n3. How are multiple tests generated in order to evaluate the patches for one issue and how does the patch selection process work?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "## strengths\n- Deals with a very important problem of automatically solving software development problems through the use of specialized software agents\n- The idea of using smaller distilled models instead of only using foundational sota models is a good direction to expore\n\n## weaknesses\n\nOverall, while I like the direction of the approach proposed in this work, I believe the technique and evaluation can be carried out more rigorously and would benefit from some more revision \n\nLack of separation between patch generation and localization model:\n- One major surprising part of the work is the lack of separately trained localization and patch generation models (i.e., the authors train one joint model for both localization and patch generation)\n- Localization and patch generation seems like to me two separately tasks (at least separate enough to warrant the construction different training datasets)\n- The authors did not seem to evaluate this potential in the paper and only allude to the results in the introduction (I also don't see any discussion or results in the appendix)\n- Furthermore, the authors also did not evaluate the evaluate the simple baseline of training a single model to accomplish all the tasks in order to demonstrate the importance of using separate models to solve this complex task\n\nVal-assert/no-assert models:\n- Its very unclear what with and without assertions means and how it may help to solve the particular issues, the author provides very limited description and justification for this seemly important component of the approach (as it involved 2 separate models)\n- Are the tests with assertion meant to simply verify that the issue is present, and the non-assertion tests used to indicate that the issue has been resolved?\n- Even after looking at the used prompt in appendix D this was not clear\n- Furthermore, this statement \"First, existing patching agents have two ways of generating PoC tests: with or without assertions.\" is provided with no citations or proofs. From my understanding I don't think any of the current agent-based approaches specifically distinguished between the described PoC tests. \n- Additionally \"... and 4 PoCs, using them to get one final patch\" how are multiple tests used to get one final patch, and why would there be 4 tests in total for each issue?\n\nOne small suggestion is regarding Table-1:\n- Since the authors compare against some of the prior approaches, it is important to acknowledge the different training methods\n- So for example, the training dataset used in this is constructed by distill some larger models whereas other works like Agentless-mini from my understanding does not require the use of any larger llm and mainly rely on RL-based models", "was_revised": true, "final_justification": "After reading the author replies and also additional reviews by the other reviewers, while I still hold some concerns (i.e., why repair and localization do not really benefit from separately training two models) But I believe that overall the paper fits the standard for neurips. As such I recommend acceptance and have increased my score", "rebuttal": "Thank the reviewer for the constructive comments. Please see below for our responses. \n\n## 1. Separation model between localization and patch generation\nThere are two reasons why we combine the localization and generation in one model:\n\n- **Share Core Abilities**: Both subtasks require fundamental capabilities in understanding the issue description to identify what needs to be fixed, and reasoning over the current codebase to determine the appropriate intervention points and methods.\n- **Task Interdependence**: A better understanding of how to fix a bug helps clarify what context needs to be retrieved for localization, while precise localization provides essential context for accurate patch generation. Localization applies these abilities to locate faulty lines; generation applies the same context to edit them. Because the tasks are sequential, training them together allows the model to reuse intermediate reasoning and to learn a smoother “find-then-fix” workflow without growing the parameter count.\n\nWe further experimented with two separate models—one fine-tuned for localization, the other for generation. Testing the results of pass@20(whether a correct patch is in the 20 generated candidates) on the SWE-bench Verified split as we did in ablation studies, to isolate the framework factor. The scores were 55.40% for the split setup and 56.20% for the joint model. The joint model performed at least as well, sometimes better.\n\n## 2. Single model for all the tasks\nWe also tried to explore the solution of training a single model for all the tasks in the patching process. A single 14 B model was fine-tuned jointly on localization, patch generation, and validation, and we re-evaluated it under the same best@20 on SWE-bench-Verified with the same hyperparameters in ablation experiments(5 root causes from localization, 20 patches from generation, and 4 PoCs from validation). We can see that the \n\n| Model                  | File-loc  | Line-loc  | Pass@20  | best@20  |\n|------------------------|-----------|-----------|---------------|---------------|\n| All-in-One 14 B    | 85.20%    | 51.60%    | 55.20%       |    41.40%    |\n| Co-PatcheR (3×14 B)    | 85.00%    | 51.80%    | 56.20%    | 43.20%    |\n\nAlthough the All-in-One model matches Co-PatcheR on the two upstream metrics— localization(file-level and line-level), Pass@20 resolved rate for patch generation and its best@20 resolved rate are worse than Co-PatcheR. \n\nThis gap appears in the validation stage. For the Localization & Generation task, the model shares common capabilities as demonstrated in part ##1, and benefits from consistent, stable behavior. However, the validation module, especially the test generation part, requires the ability to produce diverse and comprehensive tests to thoroughly explore edge cases and corner scenarios. When both objectives are packed into a single set of model weights, it creates a fundamental tension: maintaining consistency for localization/generation pushes the model toward conservative behavior, which conflicts with the need for creative and diverse test generation in validation. Consequently, the model produces overly narrow or insufficient test suites that fail to adequately validate patches. This means that potentially correct patches—which would pass validation if tested more comprehensively—are incorrectly rejected due to incomplete testing coverage, ultimately reducing the overall bug resolution rate.\n\n\n## 3.  Assertions and non-assertions testing\nSorry for the confusion. The main purpose of our PoC testing is to synthesize test cases for validating whether a candidate patch truly fixes the bug described in the issue. Both assertion tests and non-assertion tests serve this purpose, but in different styles. \n\nThe assertion tests **independently determine whether the bug is triggered**, whereas non-assertion tests **only record the output of the PoC and require an LLM to judge the output after execution**. \n- **Assertion tests** follow the common unit test style, which would provide an explicit oracle to the target issue—for example, assert sort([2,1]) == [1,2]. When such a test fails, we know that the bug is highly likely to still exist. However, the downside is that writing a correct oracle requires the model to infer an exact post-condition for the issue, which can be difficult.. \n- **Non-assertion tests** relax this requirement by performing a general PoC script. These are similar to differential testing, where we compare execution outputs before and after applying the patch to determine fix effectiveness. They are generated to a PoC according to the issue description, being expected to reproduce the same observable symptoms. In the downstream workflow, we would ask LLM to make a judgment according to the original issue description and the output of non-assertion tests. And another benefit from non-assertion tests rather than assertion tests is that non-assertion tests can capture the new bug introduced by the candidate patch by observing and analyzing actual output behavior.\n\nSo we keep both kinds of tests for two reasons:\n- **Coverage** –  Some bugs have well-defined expected values or exceptions that can be explicitly tested with assertions, while others manifest as subtle behavioral changes or output variations that are better captured through execution observation rather than rigid assertion checks. Using both styles ensures we can validate a wider range of issues.\n- **Regression** – A patch might solve the original failure but introduce a new one. Because non-assertion tests observe the output of the program, they can detect these new bugs even though the old assertion still passes. The whole patching system can utilize this information for further patch refinement.\n\n## 4. Multiple test usage in the validation module\nSorry for the confusion. We rank patches based on dynamic testing results (number of passed PoC and functionality tests) to identify the highest-scoring candidates. When multiple patches achieve the same highest score, we apply majority voting to select the final patch. Will clarify this in the paper.\n\nAs for the reason why we need 4 PoC tests, without the golden test case about the issue, we need to synthesize related test cases to validate whether a candidate patch truly fixes the bug described in the issue. Given that we can not make sure one-shot test generation can exactly describe the issue, we would like to increase the testing cases to have better coverage and robustness as mentioned above. And we also have an ablation study in §Appendix B.3 to show why we use 4 PoC tests in our system.\n\n## 5. Citations of PoC test generation classification\nWe acknowledge that existing methods do not explicitly categorize and emphasize the distinction between assertion and non-assertion tests. However, through our analysis of actual project implementations, we observed that Agentless[1] generates assertion-based reproduction tests for validation, while PatchPilot[2] produces non-assertion tests and employs LLMs for test result evaluation. We will make sure to clarify this in the paper. \n\n## 6. Suggestions for Table 1\n\nThanks for the valuable suggestion. We will update Table 1 with more information regarding the differences between our method and the baselines (e.g., distillation vs. RL).\n\n\n[1] Agentless: Demystifying LLM-based Software Engineering Agents \n[2] PatchPilot: A Cost-Efficient Software Engineering Agent with Early Attempts on Formal Verification"}, {"review_id": "YgScvETGAW", "reviewer": "Reviewer_awx7", "summary": "The paper investigates bug fixes.\nWhile it is common to use a single model for the task, the presented method uses dedicated smaller models for sub tasks, and reaches a higher resolved rate with fewer overall parameters.", "questions": "“The key challenge is to design the patch-correctness validation.” \nThis is indeed a key challenge and I did not understand how you coped with it.\nNote that in many fixes new tests are added for the fixed bug. These new tests fail on the original version and pass on the fixed one. Maybe you can identify such tests in your dataset and use them for validation.\nOf course that a correct solution should pass the old tests too.\n\n\nPart of the value of your model is being more efficient, represented by the number of parameters.\nPresenting other resources (e.g., time) used for training and inference and their savings will make the benefit clearer.\n\nIf you will use the actual localization in the analysis of generation\nYou will factor the localization step out\nYou’ll be sure that there is a valid path in this localization\nYou might want to also use None LLM-based localization methods in the ablation.\n\nAlso note that localization has possible problems\nIn many cases there are some correct localizations (e.g., adding the same function in  one of possible locations).\nTangling commits, taking care of few tasks like a bug fix and refactoring, might include irrelevant modified code", "limitations": "Training details are given in references and the not provided appendices and code. Please complete them and describe them better in the paper itself.\n\nResolved definition should be given.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 4, "strengths_and_weaknesses": "The paper suggests an Interesting direction.\nThe positioning is not clear, yet if this direction is new, it can be generalized to many areas.\n\nThe paper does ablation analysis, which contributes to the understanding of each component.\nUsing the actual location can help for a cleaner evaluation of generation.\n\nIt is not clear how “resolved” is decided? Comparison to actual fix? Passing the dataset tests? The generated tests? Please specify.\n\n\nIn the same point, the roles of the Val-no-assert model and Val-assert model are not clarified.\nAre they just critiques, helping to improve the generation or deciding the  “resolved” classification?\nIf so, their failure might raise the resolved rate .\nSimilarly, it is not clear enough how the expected result and loss function are used in training, if such training is done.\n\nAll specialized models perform dramatically worse than the general purpose models, making the direction interesting but not very useful.", "was_revised": true, "final_justification": "I recommended to accpet in the first round and paper improved since.", "rebuttal": "Thank the reviewer for the constructive and positive comments. Please see below for our responses. \n\n## 1. Definition of “Resolved”\n\nSorry for the confusion. We follow the standard SWE-bench program setup for issue resolving: Given only the issue description of a target issue and the entire codebase as the input, the patching tools should output a patch diff as an answer automatically. \n\nGenerally, the design of patching tools will contain three main modules to handle such tasks: (1) localize the problematic code regions, (2) generate a patch to fix the issue, and (3) validate the patch through our own validation module before submitting the final repair (given that test cases for the issues are not available).\n\n\"resolved\" means that, after applying our generated patches to the original codebase using Unix's patch program, SWE-bench will execute the existing unit and system tests (including the new golden test for the current issue) associated with that repository. A patch is considered \"resolved\" if it applies successfully and all existing tests pass. These test cases are collected by the SWE-bench and are not available to any patching tools during the issue resolving process. We acknowledge that even though the patched version passes all the tests, it is still possible that the patch introduces new bugs. We believe it is an interesting future direction to conduct more comprehensive evaluations of patch correctness and security beyond standard unit tests (e.g., using fuzzing tools).   \n\n## 2. Roles of the Validation Model\nSorry for the confusion. The main purpose of our PoC testing is to synthesize test cases for validating whether a candidate patch truly fixes the bug described in the issue. Both assertion tests and non-assertion tests serve this purpose, but in different styles. \n\nThe assertion tests **independently determine whether the bug is triggered**, whereas non-assertion tests **only record the output of the PoC and require an LLM to judge the output after execution**. \n- **Assertion tests** follow the common unit test style, which would provide an explicit oracle to the target issue—for example, assert sort([2,1]) == [1,2]. When such a test fails, we know that the bug is highly likely to still exist. However, the downside is that writing a correct oracle requires the model to infer an exact post-condition for the issue, which can be difficult.. \n- **Non-assertion tests** relax this requirement by performing a general PoC script. These are similar to differential testing, where we compare execution outputs before and after applying the patch to determine fix effectiveness. They are generated to a PoC according to the issue description, being expected to reproduce the same observable symptoms. In the downstream workflow, we would ask LLM to make a judgment according to the original issue description and the output of non-assertion tests. And another benefit from non-assertion tests rather than assertion tests is that non-assertion tests can capture the new bug introduced by the candidate patch by observing and analyzing actual output behavior.\n\nSo we keep both kinds of tests for two reasons:\n- **Coverage** –  Some bugs have well-defined expected values or exceptions that can be explicitly tested with assertions, while others manifest as subtle behavioral changes or output variations that are better captured through execution observation rather than rigid assertion checks. Using both styles ensures we can validate a wider range of issues.\n- **Regression** – A patch might solve the original failure but introduce a new one. Because non-assertion tests observe the output of the program, they can detect these new bugs even though the old assertion still passes. The whole patching system can utilize this information for further patch refinement.\n\n\n## 3.  Localization module explanation\n- **Factoring out the localization step**: We agree with the reviewer that using actual localization from patch commits could help isolate generation performance. However, it's important to note that even commit-based localization cannot guarantee completeness, as there is no true \"golden localization\" in this task setting. For effective code repair, models need sufficient contextual information beyond just the exact modified lines - including surrounding code, dependencies, and related functions. Additionally, the same bug can often be fixed through multiple valid approaches at different locations, making even commit-based localization an approximation rather than a definitive ground truth.\n- **Controlled evaluation**: In our experiments, we do use the same localization results across different generation models for comparison(in section §4.2.2, we use the same localization results from GPT-4o as the input of generation evaluation ), which effectively factors out localization variance and allows us to isolate the generation capabilities of different approaches.\n- **Tangling commits concern**: We agree with the reviewer that commits containing irrelevant modifications are due to mixed bug-fixing and refactoring activities. However, our work addresses this issue during training data collection. We address this by selecting training datasets that have already been filtered for issues primarily focused on bug fixes with corresponding unit tests, thereby minimizing the impact of irrelevant code modifications from tangling commits. This curation process ensures that our training data better reflects targeted bug-fixing scenarios rather than complex, multi-purpose commits.\n- **Non-LLM localization methods**: We acknowledge the suggestion to include non-LLM localization methods in our ablation study. However, our localization input only contains issue descriptions and repository code, where the issue descriptions often lack perfect proof-of-concept examples or precise error locations. This requires semantic understanding to map natural language problem descriptions to relevant code sections, making it difficult to effectively compare with traditional non-LLM localization methods that typically rely on explicit error traces, stack traces, or other structured debugging information.\n\n\n## 4. Inference efficiency metrics \nIn order to show the inference metric of our model and system, we have added an extra experiment to record the inference efficiency metrics, as shown in the following table. We log four kinds of metrics for an average of every issue: end-to-end wall-clock time for our testing setup in §4.1, tokens generated across all model calls (completion), latency for average one-shot model inference, and peak GPU memory. Experiments run on a pair of NVIDIA L40S (48 GB) cards with vllm deployment framework. The throughput can be further improved through parallel processing of multiple instances.\n\n|     Metric (per issue)     | Value (mean) |\n|:--------------------------:|:------------:|\n| End-to-end wall-clock time |    334.17 s    |\n|     Model-only latency     |    44.01 s    |\n|   Tokens generated   |    5.83 k    |\n| Peak GPU memory (per GPU)  |     44.64 GB    |\n\n\n## 5. Explanation of the efficiency of a small training dataset\nSorry for the issue with the figure reference, will correct it. Regarding the relatively small dataset size, there are several key reasons why our approach requires significantly fewer training samples compared to methods like SWE-RL:\n- **Knowledge Distillation**: Our model training employs supervised fine-tuning (SFT) through knowledge distillation from a teacher model, which enables efficient learning of the teacher's response structure and patterns. In contrast, RL-based methods like SWE-RL require extensive exploration and trial-and-error processes across many samples to optimize their policies, making them inherently data-hungry.\n- **Reasoning-Based Training Data**: Our training approach focuses on reasoning data that teaches the model to analyze problems systematically before generating solutions. This reasoning-first methodology allows the model to quickly internalize the teacher's logical reasoning patterns, leading to more efficient knowledge transfer compared to methods that learn from raw input-output pairs without explicit reasoning steps.\n- **Targeted Learning Objective**: Unlike other methods that need to explore vast action spaces, our approach has a more focused learning objective - replicating the teacher model's structured reasoning process for patch generation. This specificity reduces the data requirements significantly.\n\n\n## 6. Training method\nAll three models in our paper were trained using knowledge distillation from teacher models through supervised fine-tuning (SFT). The SFT training optimizes the standard cross-entropy loss between the model's predicted token probabilities and the ground truth tokens from the teacher-generated responses.\n\n\n## 7. Typos and missing appendix and code\nSorry for the typos, will correct them (e.g., “figure 4” should be “section 4”). We submitted the appendix in the supplementary material. We will also make our code, data, and models public."}]

Author rebuttal:

Meta-review: The paper received mixed ratings initially. There are a few minor concerns from the reviewers. The rebuttal has addressed most of them, and the reviewers raised their scores. The final decision is acceptance.
Decision: Accept (poster)