Title: Co-PatcheR: Collaborative Software Patching with Component-specific Small Reasoning Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: automatic program repair

autonomous software improvement

software patching agents

customized reasoning models
Abstract: Motivated by the success of general‑purpose large language models (LLMs) in software patching, recent works started to train specialized patching models. Most works trained one model to handle the end‑to‑end patching pipeline (including issue localization, patch generation, and patch validation). However, it is hard for a small model to handle all tasks, as different sub-tasks have different workflows and require different expertise. As such, by using a 70 billion model, SOTA methods can only reach up to 41% resolved rate on SWE-bench-Verified. Motivated by the collaborative nature, we propose Co-PatcheR, the first collaborative patching system with small and specialized reasoning models for individual components. Our key technique novelties are the specific task designs and training recipes. First, we train a model for localization and patch generation. Our localization pinpoints the suspicious lines through a two-step procedure, and our generation combines patch generation and critique. We then propose a hybrid patch validation that includes two models for crafting issue-reproducing test cases with and without assertions and judging patch correctness, followed by a majority vote-based patch selection. Through extensive evaluation, we show that Co-PatcheR achieves 46% resolved rate on SWE-bench-Verified with only 3 x 14B models. This makes Co-PatcheR the best patcher with specialized models, requiring the least training resources and the smallest models. We conduct a comprehensive ablation study to validate our recipes, as well as our choice of training data number, model size, and testing-phase scaling strategy.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper investigates bug fixes.\nWhile it is common to use a single model for the task, the presented method uses dedicated smaller models for sub tasks, and reaches a higher resolved rate with fewer overall parameters.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper suggests an Interesting direction.\nThe positioning is not clear, yet if this direction is new, it can be generalized to many areas.\n\nThe paper does ablation analysis, which contributes to the understanding of each component.\nUsing the actual location can help for a cleaner evaluation of generation.\n\nIt is not clear how “resolved” is decided? Comparison to actual fix? Passing the dataset tests? The generated tests? Please specify.\n\n\nIn the same point, the roles of the Val-no-assert model and Val-assert model are not clarified.\nAre they just critiques, helping to improve the generation or deciding the  “resolved” classification?\nIf so, their failure might raise the resolved rate .\nSimilarly, it is not clear enough how the expected result and loss function are used in training, if such training is done.\n\nAll specialized models perform dramatically worse than the general purpose models, making the direction interesting but not very useful.\n\nQUESTIONS:\n“The key challenge is to design the patch-correctness validation.” \nThis is indeed a key challenge and I did not understand how you coped with it.\nNote that in many fixes new tests are added for the fixed bug. These new tests fail on the original version and pass on the fixed one. Maybe you can identify such tests in your dataset and use them for validation.\nOf course that a correct solution should pass the old tests too.\n\n\nPart of the value of your model is being more efficient, represented by the number of parameters.\nPresenting other resources (e.g., time) used for training and inference and their savings will make the benefit clearer.\n\nIf you will use the actual localization in the analysis of generation\nYou will factor the localization step out\nYou’ll be sure that there is a valid path in this localization\nYou might want to also use None LLM-based localization methods in the ablation.\n\nAlso note that localization has possible problems\nIn many cases there are some correct localizations (e.g., adding the same function in  one of possible locations).\nTangling commits, taking care of few tasks like a bug fix and refactoring, might include irrelevant modified code\n\nLIMITATIONS:\nTraining details are given in references and the not provided appendices and code. Please complete them and describe them better in the paper itself.\n\nResolved definition should be given.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-16"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents Co-PatcheR -- an approach to train smaller task-specific LLMs to target software engineering tasks. The authors focus on designing smaller training data for specific tasks like localization, patch generation, and test generation to train smaller local models. The core technique involves using the smaller models to complete individual tasks and finally solve the end-to-end task.\n\nSTRENGTHS AND WEAKNESSES:\n## strengths\n- Deals with a very important problem of automatically solving software development problems through the use of specialized software agents\n- The idea of using smaller distilled models instead of only using foundational sota models is a good direction to expore\n\n## weaknesses\n\nOverall, while I like the direction of the approach proposed in this work, I believe the technique and evaluation can be carried out more rigorously and would benefit from some more revision \n\nLack of separation between patch generation and localization model:\n- One major surprising part of the work is the lack of separately trained localization and patch generation models (i.e., the authors train one joint model for both localization and patch generation)\n- Localization and patch generation seems like to me two separately tasks (at least separate enough to warrant the construction different training datasets)\n- The authors did not seem to evaluate this potential in the paper and only allude to the results in the introduction (I also don't see any discussion or results in the appendix)\n- Furthermore, the authors also did not evaluate the evaluate the simple baseline of training a single model to accomplish all the tasks in order to demonstrate the importance of using separate models to solve this complex task\n\nVal-assert/no-assert models:\n- Its very unclear what with and without assertions means and how it may help to solve the particular issues, the author provides very limited description and justification for this seemly important component of the approach (as it involved 2 separate models)\n- Are the tests with assertion meant to simply verify that the issue is present, and the non-assertion tests used to indicate that the issue has been resolved?\n- Even after looking at the used prompt in appendix D this was not clear\n- Furthermore, this statement \"First, existing patching agents have two ways of generating PoC tests: with or without assertions.\" is provided with no citations or proofs. From my understanding I don't think any of the current agent-based approaches specifically distinguished between the described PoC tests. \n- Additionally \"... and 4 PoCs, using them to get one final patch\" how are multiple tests used to get one final patch, and why would there be 4 tests in total for each issue?\n\nOne small suggestion is regarding Table-1:\n- Since the authors compare against some of the prior approaches, it is important to acknowledge the different training methods\n- So for example, the training dataset used in this is constructed by distill some larger models whereas other works like Agentless-mini from my understanding does not require the use of any larger llm and mainly rely on RL-based models\n\nQUESTIONS:\n1. Did the authors try using two separate models for patch generation and localization and evaluating the performance?\n2. What is and what are the benefits of using with and without assertion tests?\n3. How are multiple tests generated in order to evaluate the patches for one issue and how does the patch selection process work?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose decomposing the task of software repair into explicit steps: localisation, patch/repair, and validation. Furthermore, they argue that the expected capabilities and inputs at each stage can vary, and thus propose using different models tailored to each step. They identify two sets of requirements: for localisation and generation, the model requires access to the issue description and entire code base (especially for localisation); for validation, the model requires a deeper understanding of the issue description; however, it can forgo access to the full code base, instead focusing on the method under test. Further, they argue that in the interest of a fairer validation, the validation model should not have access to patch candidates. To train their models, the authors select problems across repositories that do not overlap with their test set and that over a range of problem difficulty levels. They then design and break down the problems at each phase. For localisation, they break down the task into file localisation and line localisation. For generation, they integrate patch refinement by combining the generation step with a critique step that provides feedback. For validation, they propose to generate unit tests with and without assert statements, arguing that this facilitates testing a wider range of program behaviours. The proposed final tool, Co-PatcheR, manages to achieve competitive results (compared to some commercial models) and improves over specialised open-source models at a greatly reduced sample cost, achieving a 46% resolved rate on SWE-bench-verified using 6K training examples.\n\nSTRENGTHS AND WEAKNESSES:\n+ S1 Strong results on SWE-bench-verified at a considerably reduced sample cost.\n+ S2 Modulo the remark for W2 below, justified decomposition into sub-problems and sub-models.\n+ S3 Strong results on sub-problems (localisation), not just the end-to-end task\n- W1[writing] The paper makes claims about testing in general (page 4, lines 125-127), arguing that testing does not require access to the full code base. This claim is true for unit testing, which is the scenario in the paper (though never stated explicitly), but it is not true in all testing settings: acceptance, integration, interaction testing as three examples require access to multiple units and in an end-to-end integration test suit, knowledge of the full code-base would be required. The problem setting should be clarified to be that of unit testing.\n- W2 It is unclear what the authors mean by assert and assert-less tests. Is checking the expected outcome considered an assert? If yes, what _exactly_ are the assert-less tests verifying, program crashes? The lack of clarity around writing these details confused me regarding the justification of the two validation models, as it is unclear what kind of output they produce. Further, it is unclear to this reviewer how writing unit tests that have asserts fundamentally covers different program paths than those that do not; is the latter trying to approximate fuzzing?\n\nQUESTIONS:\nQ1 (repeat from W2)  It is unclear what the authors mean by assert and assert-less tests. Is checking the expected outcome considered an assert? If yes, what _exactly_ are the assert-less tests verifying, program crashes? The lack of clarity around writing these details confused me regarding the justification of the two validation models, as it is unclear what kind of output they produce. Further, it is unclear to this reviewer how writing unit tests that have asserts fundamentally covers different program paths than those that do not; is the latter trying to approximate fuzzing?\n\nQ2 I remain unconvinced by the merging of Localisation and Generation. The Generation models would require different types of contexts compared to localisation: related/co-changed files, called files, a table of available symbols (APIs), etc. These feel materially different from the full code-base view of Localisation unless Localisation is reused to find the relevant files/symbols. Was a set-up where the two stages use different models considered? If not, is it due to a higher priority during model design being given to reducing model size/parameter count?\n\n- Nit: Page 4, Line 138, you refer to test cases as vulnerable? Is this from an older iteration of the paper where the focus was on security testing? Also, same location, you introduce vulnerable test cases as PoC (proof of concept?), this seems highly unusual and does not match the words abbreviated.\n- Writing Nit: §3.2.1 \"following recent work by S1 [31]\" -> \"following recent work in S1 [31]\" or \"following recent work by Muennighoff et al. [31]\" as \"S1\" is the tool/method name.\n\nLIMITATIONS:\nLimitations are reasonably addressed.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces Co-PatcheR, a collaborative patching framework composed of small, component-specific reasoning models for software patching. The authors propose decomposing the patching pipeline into three components: localization, patch generation, and validation. Co-PatcheR uses a single Loc-Gen model for both localization and generation, and two separate models for diverse issue-reproducing test case (PoC) generation and validation. The authors design tailored task formulations and training recipes for each component, including a lightweight reasoning data construction method and a model distillation strategy. Experimental results on the SWE-bench-Verified benchmark show that Co-PatcheR achieves a 46% resolved rate using only 3×14B models and 6K training samples, outperforming larger baselines while being substantially more parameter- and data-efficient.\n\nSTRENGTHS AND WEAKNESSES:\nStrength\n1. The idea of decomposing patching tasks into collaborative reasoning components, each trained with customized objectives and data, is practically valuable, especially in resource-constrained scenarios.\n2. The method achieves a 46% resolved rate on SWE-bench-Verified, surpassing large end-to-end baselines while using fewer parameters and less data.\n3.The authors provide extensive ablations that validate their architectural and training choices, such as the joint Loc-Gen model, multi-model PoC generation, critique-based generation, and data/model scaling considerations.\n\nWeakness\n1. Given that, when using the same model (Claude-3.5), the agent framework PatchPilot achieves better performance than the agentless setting, it would be valuable for the authors to conduct additional cross-framework evaluations. For example, comparing Co-PatcheR + Agentless with SWE-RL-70B + Agentless, or Co-PatcheR + PatchPilot with SWE-RL-70B + PatchPilot. Such comparisons would help disentangle the respective contributions of the model architecture and the agent framework, providing fairer and more convincing evidence to support the authors’ insight that multiple expert small models can outperform a large end-to-end model on SWE-bench.\n2. The authors present an encouraging result that three small models, requiring fewer parameters and less training data, can outperform a large end-to-end model on SWE-bench. However, beyond training cost, inference cost is equally critical for practical applications. The current study primarily focuses on generation quality metrics (e.g., pass@1, pass@K) without reporting inference time or the number of generated tokens. In deployment scenarios, inference latency and token generation efficiency are important considerations — particularly for approaches like Co-PatcheR, which rely on generating large numbers of candidate patches to match the performance of larger models. It would strengthen the paper to include an evaluation or at least a discussion of inference efficiency alongside generation accuracy.\n\nQUESTIONS:\nQ1. Could the authors report cross-framework results (e.g., Co-PatcheR + Agentless vs. SWE-RL-70B + Agentless, and Co-PatcheR + PatchPilot vs. SWE-RL-70B + PatchPilot) to better disentangle the effects of model architecture and agent framework?\n\nQ2.Could the authors provide inference efficiency metrics (e.g., latency, total generated tokens, or wall-clock time) to assess practical deployment cost?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-04"}