SWE-RM: Execution-free Feedback for Software Engineering Agents
Authors: KaShun SHUM
Binyuan Hui
Jiawei Chen
Lei Zhang
X. W.
Jiaxi Yang
Yuzhen Huang
Junyang Lin
Junxian He
Venue: ICLR 2026 Poster (2026)
Keywords: Coding Agent
Reward Model
Test-time Scaling
Reinforcement Learning
Primary area: foundation or frontier models, including LLMs

Abstract: Execution-based feedback like unit testing is widely used in the development of coding agents through test-time scaling (TTS) and reinforcement learning (RL). This paradigm requires scalable and reliable collection of unit test cases to provide accurate feedback, and the resulting feedback is often sparse and cannot effectively distinguish between trajectories that are both successful or both unsuccessful. In contrast, execution-free feedback from reward models can provide more fine-grained signals without depending on unit test cases. Despite this potential, execution-free feedback for realistic software engineering (SWE) agents remains underexplored. Aiming to develop versatile reward models that are effective across TTS and RL, however, we observe that two verifiers with nearly identical TTS performance can nevertheless yield very different results in RL. Intuitively, TTS primarily reflects the model’s ability to select the best trajectory, but this ability does not necessarily generalize to RL. To address this limitation, we identify two additional aspects that are crucial for RL training: classification accuracy and calibration. We then conduct comprehensive controlled experiments to investigate how to train a robust reward model that performs well across these metrics. In particular, we analyze the impact of various factors such as training data scale, policy mixtures, and data source composition.
Guided by these investigations, we introduce SWE-RM, an accurate and robust reward model adopting a mixture-of-experts architecture with 30B total parameters and 3B activated during inference. SWE-RM substantially improves SWE agents on both TTS and RL performance. For example, it increases the accuracy of Qwen3-Coder-Flash from 51.6% to 62.0%, and Qwen3-Coder-Max from 67.0% to 74.6% on SWE-Bench Verified using TTS, achieving new state-of-the-art performance among open-source models. On RL training, SWE-RM lifts the resolve rate of execution-based counterparts by 3 absolute points on SWE-Bench Verified.

Published as a conference paper at ICLR 2026
SWE-RM: EXEcUTION-FREE FEEDBACK FOR SOFT- WARE ENGINEERING AGENTS
Kashun Shum 1.2 Binyuan Hui Jiawei Chen", Lei Zhang X. , Jiaxi Yang² Yuzhen Huang', Junyang Lin", Junxian He! The Hong Kong University of Science and Technology. Qwen Team, Alibaba Group
"Equal Contribution (ksshumab, junxianh| uese.uss.hk, binyuan .byea www.iaiiaaaaa.a..
ABSTRACT
Execution-based feedback like unit testing widely used the development of coding agents through test-time scaling (TTS) and reinforcement learning (RL). This paradigm requires scalable and reliable collection of unit test cases to pro- vide accurate feedback, and the resulting feedback is often sparse and cannot effectively distinguish between trajectories that are both successful or both un- successful. In contrast, execution-free feedback from reward models can provide more fine-grained signals without depending on unit test cases. Despite this poten- tial, freet feedback for realistic software engineering (SWE) agents re- mains underexplored. Aiming to develop versatile reward models that are effective across TTS and RL, however, we observe that two verifiers with nearly identical TTS performance can nevertheless yield very different results in RL. Intuitively, TTS primarily reflects the model's ability to select the best trajectory, but this abil- ity does not necessarily generalize to RL. To address this limitation, we identify two additional aspects that are crucial for RL training: classification accuracy and calibration. We then conduct comprehensive controlled experiments to investi- gate how to train a robust reward model that performs well across these metrics. In particular, we analyze the impact of various factors such training data scale, policy mixtures, and data source composition. Guided by these investigations, we introduce SWE-RM, M, accurate and robust reward model adopting mixture-of- experts architecture with 30B total parameters and 3B activated during inference. SWE-RM substantially improves SWE agents on both TTS and RL performance. For example, it increases the accuracy of Qwen Coder Fl Flash from 51.6% 62.0%, and O ified using TTS, achieving new state-of of-the-art performance among open-source models. On RL training. SWE-RM lifts the resolve rate of execution-basee coun- terparts by 3 absolute points on SWE-Bench Verified.
from 67.0% to 74.6% on SWE-Bench Ver-
INTRODUCTION
The automation of complex software development tasks through coding agents represents a signifi- cant frontier in large language models (LLMs). A critical component in developing these agents the feedback mechanism used during training and evaluation, particularly through reinforce- ment learning (RL. (Wei et al., 2025 Qwen Team 2025 Zhang et al.) (026) and test-time scal- ing (TTS) (Xia et al. 2024 Jain et al. 2025). Broadly, these mechanisms fall into two categories: execution- based verifiers (Xia et al.) 22024] Jain et al. 2025), which rely on concrete outcomes like unit test results, and execution-free verifier (an et al. 2025 Luo et al. 2025). which are typically without sandbox environments.
model-based reward models that provide continuous
While widely used, execution-basee feedback has inherent limitations. It provides only a sparse, binary signal (pass/fail), which makes it difficult to distinguish between different successful or un- successful trajectories. Beyond this lack of granularity, unit tests require comprehensive coverage to yield accurate assessments, which is often unavailable. To address this challenge, exiting works rely
Thhouuhhott this paper, will use "reward model" and "exceution-free verifier" interchangeably.

---

Published as a conference paper at ICLR 2026
75
Owen3-Coder-Max
SWE-RM TTS (74.6%)
V3. (73.1%)
Swe-bench Verified Resolve Rate(%)
70-
MiniMax-M2 (69.4%)
+7.6
GLM-4.6 (68.0%)
Kimi K2-0-K2-005 (69.2%)
65-
Wwode-Coder-Max (67.0%)
DeepSeek-V3.1 (66.0%)
+ SWE-RM TTS (62.0%)
60-
DeepSWE (59.0%)
Kimi-Dev (60.4%)
DeepSeek-R1-0522 (57.8%)
+10.4
55-
MiniMax.M1 (56.0%)
Foen (51 (1.65%)
50-
R2E-GGy-a22 (51.0% CGM-SWE-PY (50.4%)
Skywork-328 (47%)
Kimi-Dev (48.0%)
45
Dewstral-mail-2555 (46.8%)
DeepSeek-V3 (42.0%)
DeepSWE (42.2%)
WWE-agent-LM..c. 32B (40.2%)
OpenHandss-LM-32B-Wa (37.2%)
Qwen3- 231-3-3333-A288 (34.4%)
35-
RZE-Gym-328 (34.4%)
Pass@1
SWE (32.0%)
With Test-time Scaling
30
20
80
320
1000
Model Size (B)
Figure 1: The pass@1 I and TTS resolve rate of various open-source and proprictary models on SWE- Bench Verified. Part of the baseline results are referenced from He et al. (2025).
on extracting unit test from Github repos (Jimenez al. 2024) or model-generated unit tests (Yang al. 2025 Jain et al., 2025) that are not rigorously validated. For example, in issue-fixing tasks, the unit tests used from real GitHub repositories often overly specific, and in some cases, entirely unrelated to the target issue (OpenAl) 2025). As a result, execution- based feedback limits the code data that can be used for effective reinforcement learning test-time scaling due requirements of high-quality unit tests. When such tests are unreliable, the resulting feedback becomes a signif- icant challenge for RL, where nuanced and consistent reward signals are essential. Execution-Fie -ree feedback offers a compelling alternative by providing continuous, fine-grained scores across entire trajectories, allowing for better discrimination among candidate solutions and reducing bias toward specific patches. Despite its promise, execution- feruu feedback remains largely underexplored, and its properties in the context of SWE agents are not yet well understood.
In this work, we aim to develop versatile and effective reward model usable across different sce- narios such as TTS and RI. for software engineering. While it straightforward adopt TTS (e.g., best of k)) performance directly as the metric to guide the reward model training (Pan et al.) 20255) our initial findings reveal that two verifiers with nearly identical TTS performance can show dras- tically different behavior in RL. This leads us to fundamental research question: What properties determine a reward model's 's effectiveness in RL training, and how can we develop an all-round SWE reward model that performs well in both TTS and RL?
Intuitively. TTS primarily measures a verifier's ability to rank the correct solution highest among multiple candidates, but it overlooks aspects that essential for RL: the ability to effectively dis- tinguish correct from incorrect trajectories and to produce scores that reliably correspond to the de- gree of correctness. Based on this observation, we further evaluate reward models using additional metries: AUC, which reflects the correctness of relative ordering across trajectories, and ECE (Guo) al. 2017), which measures calibration representing whether the verifier's scores align with em- pirical correctness. We demonstrate that AUC and calibration provide complementary information TTS and are both critical for ensuring the reward model delivers reliable signals RL.
To train reward model that performs well across these metrics, we conduct large-scale ablation studies examining the effects of training data scale, the ratio of positive to negative samples, mix- tures of data sources, and context length. These investigations lead to a practical recipe for building robust, execution-free reward models tailored to SWE tasks. Guided by these investigations, we obtain SWE-RM, an accurate and robust reward model with 30B total and 3B activated parameters for advancing SWE agents. On SWE-bench Verified (OpenAl 2025). SWE- RM lifts the accuracy of Qwen3-Coders FF F lash from 51.6% to 62.0% and Owen3-Codes Max from 67.0% to 74.6%, achieving best-in-class among 30B-level and all open open-source models respectively. as shown in Fig- ure T Moreover, SWE-RM highly effective when used as reward signal in agentic RL training.
2

---

Published as a conference paper at ICLR 2026
1.0
Verifier
SWE- SWE- Be-nch Verified
Resolve Rate (%)
True Positive Rate
0.8
Verifier
45
0.6
40-
0.4
35
0.2
Verifier A (AUC 0.764)
30
Verifier A (Good AUC & Cali.)
Verifier B (AUC 0.669)
Random Classifler
Verifier B (Bad AUC Cali.)
25
0..0
0.8
1.0
Training Steps
False Positive Rate
Figure 2: RL training curves two verifiers with similar TTS performance. Despite compa- rable TTS, the downstream RL outcomes differ drastically.
Figure 3: Two models with similar TTS per- formance (Model A with +4.7% and Model B with +4.5%) show significant differences in their AUC scores.
For example, it improves the RL performance of execution-based counterparts by 3 absolute points on SWE-bench Verified.
2 RELATED WORK
In Software Engineering (SWE) tasks, verifiers fall into categories: execution-based which rely on unit tests (e.g., Agentless (Xia et al. 2024), R2E-Gym (Jain et al. (0025), DeepSWE (Luo et al.
2025 ), and execution-free, which use model-based scoring (e.g.. SWE-Gym (Pan al.) 2025). OpenHands Critic (Team, 2025 ). Existing work on execution free verifiers has primarily empha- sized test-time scaling (Pan al. 20255 Jain al. [025 Luo al.) (225). with limited attention to other quality dimensions. We show that execution-free verifier's s quality also depends on classi- fication accuracy and calibration, and provide a systematic study training such verifiers. In rein- forcement learning for SWE agents (e.g., OpenHands (Wang et al., 2025). SWE-Agent (Yang al.) ), execution-based feedback akin to rule-based metrics in math (DeepSeek-Al 2025) has enabled recent model training (e.g., Qwen3-Coder (Qwen Team 2025). GLM-4.5 (Team et al.) 2025a), MiniMax-M1 MMiniMax, 20255), but is constrained by noisy test suites and sparse signals. We are the first to integrate execution free feedback into SWE agentic RL, demonstrating its poten- tial to deliver finer-grained rewards and improve efficiency. An extended related work are discussed
in Appendix
3 WHAT DEFINES A VERSATILE REWARD MODEL FOR SWE?
Our goal is to develop a versatile reward model that performs well across both TTS and RL. Follow- ing common practice, we begin by examining whether TTS performance can serve as reliable guide for selecting reward model for RL. We first present our initial findings that highlight the limitations of relying solely on TTS performance to guide reward model training. This result demonstrates that TTS alone cannot explain downstream success in RL, raising important questions about what prop- erties a verifier matter. To resolve this gap, we next revisit the role of TTS as an evaluation metric, analyze its limitations, and introduce complementary criteria-AUc and calibration that provide a more holistic view of verifier quality.
3.1 INITIAL FINDINGS: LIMITATIONS RELYING SOLELY ON TTS
As we aim to develop versatile reward model that can be applied across different scenarios such as TTS and RL, but is unknown what defines such a versatile reward model and whether TTS and RL impose different requirements. A natural question to ask is whether reward model that per- forms well on TTS will also perform well on RL. To avoid ambiguity, the reward model trained purely through supervised next-token prediction instead of using TTS the training signal. TTS
3

---

Published as a conference paper at ICLR 2026
is used exclusively introduced in $ [.11 Our initial exploration reveals an intriguing finding: two execution-free veri- fiers that achieve nearly identical TTS improvements give rise strikingly different behaviors when used as reward models in reinforcement learning. As shown in Figure both verifier A and verifier B achieve similar TTS improvements, indicating, first glance, that they equally effective at choosing the correct solution highest among candidate trajectories. Yet, when deployed in RL train- ing, verifier A supports smooth improvement, while verifier B exhibits significant instability, failing to provide reliable learning signals and eventually causing RL training to collapse.
evaluation metric for model selection etc. More training details will be
This result challenges the widely adopted view of TTS as a sufficient proxy for verifier qual- ity (Lightman et al.) [2033] Pan et al. 2025). If two models are judged equivalent by TTS but behave so differently in RI., then TTS alone cannot capture the aspects of verifier that truly matter for reinforcement learning. In other words, TTS provides only partial picture: it summarizes top-1 ranking ability but hides other properties that directly affect how reward signals shape policy up- dates. These findings prompt us to ask fundamental research question: What properties determine a reward model's effectiveness in RL training, and how can we develop an all-round SWE reward model that performs well in both TTS and RL?
To answer this, we must carefully reconsider what TTS actually measures, why it fails to explain the RL discrepancy observed, and what alternative metries reveal the missing dimensions of verifier quality. This motivates our shift beyond TTS to a broader and versatile evaluation.
3.2 MOVING BEYOND TTS: THE NEED FOR MORE VERSATILE EVALUATION ON VERIFIER QUALITY
At first glance. TTS appears to be natural metric: it checks whether the correct solution trajectory is ranked highest among a pool of candidates. Intuitively, this reflects a verificr's ability to make the right top-1 decision, however, TTS only measures narrow slice of verifier capability. By fo- cusing exclusively on whether the single best trajectory is ranked first, TTS ignores two properties that become critical once the verifier is used reward model in reinforcement learning. The first overlooked dimension is discriminative ability. In RL, the agent generates a wide range of trajecto- ries, some of which are unresolved. The verifier must provide accurate feedback not, just for the best trajectory, but across many near-miss or partially correct candidates. A verifier with weak discrim- inative ability will assign similar scores both correct and incorrect trajectories, producing noisy reward signals that compromise policy updates. The second is confidence reliability, or calibration. In RI, verification scores are often interpreted as proxies for the likelihood of correctness, serving as the reward magnitudes used to guide policy learning. If these scores are mis-calibrated, for instance, a normalized score of 0.9 reflects only a 60% probability of being actual correct-then the policy receives misleading signals about the expected value of its actions. Poor calibration can therefore poison the reward shaping process, leading to unstable or collapsed training dynamics even if top-1 accuracy (TTS) appears satisfactory.
These overlooked dimensions provide a natural explanation for the discrepancies we observed in Figure 2 and to capture these dimensions, we supplement TTS with two complementary metrics. AUC (Bradley 1997) evaluates discriminative ability by measuring how well the verifier separates resolved from unresolved trajectories across the entire distribution, rather than focusing only on the best. Calibration (Wang 2025) quantifies the alignment between predicted confidence scores and empirical correctness, for example by using Expected Calibration Error (ECE) (Guo et al. 2017]. A higher AUC means models can better descriminate resolved and unresolved trajectories while lower ECE means there is lower mismatch between confidence and accuracy, indicating higher reliability. Together, these three metrics-TTS AUC, and calibration a more versatile evaluation toolkit: they jointly capture top-1 ranking accuracy, overall discriminative power, and reliability of confidence estimates.
Empirical analysis demonstrates the importance of considering all three. (1) Discriminative gap despite equal TTS: As shown in Figure 3 verifier A and verifier B obtain nearly identical TTS im- provements (+4.7% vs. +4.5%), yet their AUC scores differ by 0.095. Thus, although both appear equivalent by TTS, only verifier A reliably distinguishes resolved from unresolved trajectories property crucial for producing consistent reward signals. (2) Calibration and score distribution disparity: Figure E reveals that verifier B suffers from widespread over- and under-confidencee,
4

---

Published as a conference paper at ICLR 2026
1.0
1.0
1.0
1.0
Fraction of Resolved
ECE - 0.077
ECE
0.210
0.8
0.8
Verifier's Score
0.6-
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0.2
0.2
0.2
0.2
0.0-
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.2
0.4
0.6
0.8
1.0
watrop 12007
saropy Latad
ciangeo Mangi--2een 12005
entrayy tattop-1145 1430
ieare Ltd
ryroy 201
atrrpy 12801
aitroyy Sarpy--actee 132id
clangeo ZeRg
altroay L.139
solkh feat--onn 281
syray Ltd
Model Confidence
Model Confidence
paioo
Accuracy
Confiderice Perfect Callbration
2/4 Under Confidence
Resolved Traj.
Unresolved Traj.
Reliability diagrams for verifier A (left) and verifier B (right) with similar TTS per- formance.
Figure 4:
Figure 5: Score distribution cases for verifier A (left) and verifier (right) with similar TTS per- formance.
while verifier is three times better calibrated according to expected calibration error. Again, TTS fails to reflect this difference, though it directly affects the trustworthiness of reward magni- tudes. Figure [5] further highlights why this matters: across 32 runs of random selected 8 instances, Model A consistently assigns high scores resolved trajectories and low scores to unresolved ones, making it possible to set reliable threshold for acceptance. In contrast, Model B frequently as- sign unexpectedly low scores for resolved trajectories, while unresolved trajectories receive inflated produu-producing overlapping distributions that might confuse policy models. The trajectories used for calibration and score distribution analysis are sampled from Qwen3-Coder-Mee the other settings are same as the one we will show in Appendix D.I] This combination of miscali- bration and poor separation explains why Model provides misleading signals in RL even when its TTS remains competitive. We also include theoretical analysis between the three metrics and RL dynamics in Appendix
further
while
These observations underscore that TTS, accuracy, and calibration are complementary metrics, each capturing distinct aspect of verifier capability. The discussion of metrics this section is lim- ited to using early, small-scale RM variants illustrative examples. And in subsequent investiga- tion- beginingn after we motivate the need for AUC and calibration-marke the start of the actual supervised RM training and large-scale, comprehensive ablations. We will show how obtain versatile and robust reward model in and discuss the implications for reinforcement learning in
$
4 HOW TO TRAIN A VERSATILE REWARD MODEL FOR SWE?
To build a versatile and robust reward model as discussed in $ we conclude and analyze several critical factors that significantly influence final performance. Specifically, we systematically inves- tigate training data scale, the ratio of positive to negative samples, policy, data source, and context length, and discuss their impact on the verifier's three core abilities. These observations collectively guide the development of SWE-RM, which achieves superior performance.
4.1 TRAINING METHODS
Following SWE-Gym (Pan et al. 2025) we formulate reward modeling generative classification task, where the reward model takes a trajectory as input and outputs special token (e.g.. YES/NO). Given the full multi-turn multi-turn trajectory, the model is prompted output single special token, either YES (resolved) or NO (unresolved). And the supervised fine-tuning utilizes standard next-token prediction loss on this special token. At inference time, by obtaining the log probability of the special token YES(1,) and NO(..), the final score r is calculated by exp (1,)) (1g) + exp (ln)), which maps to a continuous reward model score r E [0, 1]. To construct training data, we collect agent trajectories by deploying different policy models (Qwen3 CCoder and Claude 4) to interact with the agent scaffold OpenHands (Wang et al. 2025) across multiple training data sources, including SWE-Gym (Pan et al.] 2025), SWE-rebench (Badertdinov et al. 2025), SWE-smith (Yang et al. 2025). and R2E-Gym (Jain et al.) 2025). These trajectories are then labeled as positive or negative based their execution results with the provided fail2pass test.
5

---

Published as a conference paper at ICLR 2026
0.58
Resolved
Unressived
Resolve Rate RM@k
0.57
1.0
0.56
Verifier's Score
0.8
0.55
0.54
0.4
0.53
Train data 500
0.2
0.52
Train data 5k
0.51
0.0
Train data
100k
[622
0.403
0C2
0.150
[C2
0.232
622 0.337
pCE 3.007
0.50
4
10
12
16
20
24
28
32
500
2k
5k
25k
#100k
Number of Rollouts (k)
Model with # of Training Data
Test-time Scaling curve of models trained with different training data. Right: Distribution of verifier scores across all evaluated trajectories. Clearer separation between resolved and unresolved trajectories, along with a lower ECE, indicates better performance.
Figure 6: Left:
We use Qwen3-30m 30B 1----A-0 A3B (Qwen Team 2025) as the base model for reward model training, it provides balance between efficiency and strong coding capabilities. Evalua- tion primarily conducted on SWE-bench Verified (Jimenez et al.) 2024), a curated subset of 500 real-world software engi-
RM Training Setup
human-verified tasks designed reliably assess model performance neering problems. For the evaluation metrics, test-time scaling is measured using 32 independent runs for each instance on SWE-bench Verified. Accuracy calculated using the AUC score across all 32 x 500 trajectories, while calibration is assessed by the expected calibration error (Guo et al.) 2017). RM@
defined as the resolve rate of the final selected trajectories from samples. For each h < 32, we report the mean and variance over 5 random runs to ensure fair evaluation. Further details on the reward model training setup can be found in Appendix D
4.2 DATA SCALING AND RATIO EFFECT
Poorly trained reward models often exhibit unexpected behavior when evaluated on out-of-domain (OOD) data. This OOD generalization challenge is particularly severe in SWE tasks, where multi- turn interactions create a substantially larger output space compared to traditional reasoning tasks, which might requires more training data. As shown in Figure [ we uniformly sampled varying amounts of training data from different policy models and data sources. The left subfigure demon- strates that models trained on more than 20k samples generally achieve improved test-time scaling performance as k increases, whereas models trained on fewer samples (e.g., fewer than 5k) may even experience declining performance. We attribute this to the limited generalization capacity of under-trained models: A grows, the probability of encountering OOD trajectories increases, and test-time scaling becomes highly sensitive to such cases. Even a single erroneously high score as- signed to incorrect trajectory significantly distort the final resolve rate.
While TTS performance improves as the training data size increases up to 25k, further expansion to 100k yields diminishing returns. the distributions in the right subfigure of Figure 6 reveal that larger training datasets enhance discriminative ability, as evidenced by clearer separation between resolved and unresolved trajectories. Moreover, models trained on more data demonstrate improved calibration: for instance, a model trained with only 500 examples has an ECE of 0.411 1e4enn times higher than that of model trained with 100k examples. This indicates that scaling up training data produces more reliable scores, highlighting the effectiveness of data scaling.
To further investigate the role of data composition, we fix the total amount of training data and vary the ratio of positive to negative trajectories, as shown in Table We observe that across both model scales, the 2:1 ratio generally achieves the best overall performance in terms of AUC, calibration, and test-time scaling. Due to the limited availability of positive data, we experiment with ratios up to 1. To further investigate the role of data composition, we fix the total amount of training data and vary the ratio of positive to negative trajectories, shown in Table [ We observe that across both model scales, the 2:1 ratio generally achieves the best overall performance in terms of AUC, calibration, and test-time scaling. By contrast, more balanced ratios such as 1:1 avoid extreme skew but still fall short of the 2:1 configuration. Importantly, the 2:1 ratio also offers higher efficiency, as it requires smaller pool of negative data while still utilizing all available positive data in practice.

---

Published as a conference paper at ICLR 2026
Table Effect of training verifiers with different positive-to-negative data ratios on AUC, ECE, and test-time scaling performance (best results in bold).
-rode--Coder- Flash
Mwen3-Coder--ee
RATIO
AUC
ECE
RM@32
AUC
ECE +
RM@3
2:
0.805
0.080
62.0%
0.755
0.121
71.0%
0.782
0.132
60.8%
0.734
0.157
70.2%
1 1:2 2
0.789
0.235
61.0%
0.736
0.371
69.4%
1 1:44
4
0.789
0.185
61.6%
0.742
0.299
71.8%
1 1:8 0.778
0.349
60.2%
0.738
0.541
70.6%
Considering this balance between effectiveness and efficiency, adopt the 2:1 ratio the default configuration subsequent experiments.
4.3 CONTEXT LENGTH CONSTRAINT
While previous execution-free verifiers in SWE mainly support a context length of 32k (Pan al., 2025 Jain et al. 2025), free verifiers are the first to scale up to 256k context length, enabling the scoring of complex and long trajectories. This is especially impor- tant for challenging questions, which typically involve extremely long contexts. As shown in Table 2 only when the context length ex- tended 128k can more than 99% of trajec- tories be successfully scored without exceeding the limit. Furthermore, as models are able to score more trajectories, execution- achieve better test-time scaling performance, reflected in the increasing RM@ 32. A more detailed discussion on context length are shown in Appendix D.4
Table 2: Effect Context Length Scaling on veri- fier's s score rate and test-time scaling performance. Score rate represent how many percent of trajecto- ries can be successfully scored without exceeding the context window.
execution-
CONTEXT LEN.
SCORE RATE
RM
16
0.5%
66.8%
12.5%
67.4%
64
88.3%
70.6%
128 k
99.5%
73.0%
256 k
100 % 74.4%
verifiers
4.4 POLICY AND SOURCE ABLATION
We also examine the impact of training data collected from different policy models verifier per- formance. For on-policy data, we sample training examples using the corresponding Flash/Max model on SWE-rebench, while for off-policy data we sample using Claude-sonnet-4 (Anthropic). As shown in Table 3 policy ablation, while on-policy data sometimes yields stronger results on specific metrics (e.g., TTS on Swen3-Coder-Mal ter balance across AUC, ECE, and ranking. This indicates that combining on- and off-policy data enhances the generalization ability of the verifier. Such findings also reflect the advantage of our comprehensive evaluation in revealing robust trends that TTS-only analyses might overlook.
overall the Mix-Policy setting provides bet-
We further investigate the impact of training data sources on verifier performance. Table under single-source settings, SWE-rebench achieves the best results in both AUC and RM@3 32, indicating that rebench may provide the highest-quality data. However, incorporating SWE-smith and SWE-Gym leads improved calibration (lower ECE), and adding more sources enhances data scaling effects as we shown in $ 4.2 Based on these observations, our final setup uses mixture primarily derived from SWE-rebench, supplemented with data from SWE-smith and SWE-Gym, achieving a balance between quality, calibration, and scalability.
shown in
SWE-RM: VERSATILE REWARD MODEL FOR TTS AND RL
In this section, we present SWE-RM RM, an accurate and robust execution-free free verifier that not only achieves ofte-of-the-ar art test-time scaling performance but also significantly improves downstream reinforcement learning. We first demonstrate the superior performance SWE-RR in TTS ($ 5.1 and then RI. 5.2

---

Published as a conference paper at ICLR 2026
Table 3: Effect of training verifiers with different policy mixture and data source on three core abilities(bestt results in bold).
Swenn-Cooees
METHODS MODELS
AUC
ECF4
RM@32
AUC ECE
RM@32
Policy Ablation
On-Policy Off-Policy
0.785
0.148
58.6
0.727
0.067
71.0
0.778
0.113
0.728
0.145
70.6
Mix-Policy
0.804
0.033
59.6
0.751
0.082
70.2
Source Ablation
SWE-rchench Badertdinov
2025)
0.814
0.076
0.612
0.774
0.048
0.718
SWE-smith Yang al. 2025
0.781
0.033
0.584
0.736
0.039
0.70
SWE-Gyym Pan et al. 2025
0.776
0.087
0.588
0.742
0.044
0.714
SWE-Gym SWE-
0.813
0.034
0.602
0.772
0.035
0.72
SWE-Gym SWE-rebench
0.802
0.087
0.61
0.762
0.039
0.712
SWE-rebench SWE-smith
0.807
0.138
0.596
0.765
0.107
0.714
SWE-rebench SWE-smith SWE-Gym
0.807
0.067
0.612
0.766
0.033
0.718
Table 4: Comparison of different verifiers on three core abilities. Evaluation trajectories are sampled from Toren-Coode and OpenHands LM- 32B on SWE-bench Verified. EB means execution- based verifier while EF stands for execution-free verifier. Best results are in bold.
Openkands-Lige LM--22
(wen33Coder--Feaee
Owen3 Sreen3-Code-Kaa
VERIHER
TYPE
AUC
ECE.
RM@ 32
AUC
ECE,
RM@32
AUC
ECE.
RM 32
AGENTLESS Xia al 2024
EB
42.4%
52.6%
65.0%
SWE-GYM Pan
EF
0.718
0.164
41.6%
0.776
0.223
51.2%
0.752
0.283
65.4%
SWE Luo
2025
EB
44.2%
54.6%
67.66 6%.6%
EF
0.732
0.118
44.6%
0.758
0.124
53.2%
0.74
0.139
66.2%
0.080
5.1 NEW THE- IN TTS
Based on the investigation in $14 our final trained SWE-RM achieves state-of-the the-t-rttt-ttttt performance compared with previous works. We begin by discussing the baselines and evaluation setup. followed by an analysis of the SWE-RM results on TTS, AUC, and calibration.
Baselines We compare our trained execution-free free verifier against several existing execution free and execution- based verifiers: (1) Agentless (Xia al., (024), an execution- based method that gen- erates reproduction tests for each trajectory and re-ranks them based on test results; (2) SWE-Gym Verifier (Pan et al., 2025). an execution-free verifier based on Qwen2. 5-32B and trained on the SWE-Gym dataset; (3) DeepSWE-EE Verifier (Luo et al. 2025) the execution- based component of the current state-of-the-art Dee Hybrid-TTS. This verifier extends the R2E-Gym execution- based verifier (Jain al. 2025) and follows a similar mechanism to Agentless; (4) DeepSWE-EF Verifier (Luo et al. 2025), the execution component of DeepSWE: Hybrid-TT TTS, which improves upon the R2E-Gym execution-free verifier. The evaluation setup for SWE-RA RM are same the set-
ting illustrated in 4.1
SWE-RM performance Our results in Table show that SWE-RM consistently outperforms all baselines across AUC, ECE, and RM @ 32, achieving the best TTS, discrimination and calibra- tion ability. The gains are limited Qwen3 -wenn- series models, where RM@32 improves pass@I by 7-10 points, but also extend to -penHandss-M- 32B, where SWE-RM delivers the highest overall performance. This demonstrates the generalization ability of our verifier.
5.2 REINFORCHMENT LEARNING WITH EXECUTION FREE FEEDBACK IN SWE
Different from reinforcement learning in math problems, which easily receive scalable, correct ward by comparing with the ground truth answers, reinforcement learning from verifiable reward (RLVR) are facing two major challenges in software engineering tasks: (1) Most training data
constructed by some automated pipelines with unchecked quality unit tests, the execution-basse feedback are guaranteed to be correct. (2) The long horizon context length and sandbox
8

---

Published as a conference paper at ICLR 2026
Resolve Rate (%)
Training Reward
1.25
50
1.00
45
0.75
40
Hybrid Feedback
0.50
35
Executior-based only Feedback
0.25
orecution-free only Feedback
0
20
40
60
80
100
20
40
60
80
100
Training Steps
Training Steps
Figure Left: RL performance on SWE-bench Verified when using different feedback. Right:
Average training reward for different models.
ecution significantly limit the scale of RL, leading to slow improvements especially under sparse 0/1 reward. In this subsection, we show execution-free feedback offers a promising approach only accelerate training but also further enhance overall performance by providing more fine-grained reward signals. We first establish the RL setup in 5.2.1 and then discuss the results in $ 5.2.2
5.2.1 RL SETUP
Scaffold and Model For training scaffold, we adapt verl (Sheng et al. 2024) with Mega- tron (Shoeybi et al. 2019) which enables efficient multi-turn agentic reinforcement learning and SGLang for trajectory rollout. For agent scaffold, similar to the reward model rollout, employ OpenHands (Wang al. 2025)) for tool interactions. For base models, use with warm-up.
3wenn-330 3-BB-33BBBBBBBBB
(Qwen Team 2025
Evaluation Setup Similar to reward model evaluation, we conduct our evaluation on SWE-bench Verified (Jimenez et al.) 2024). In the RL setting, we only generate trajectory and patch for each instance using greedy decoding following OpenHands (Wang et al. 2025). without any test-time scaling, the final pass @ score.
Baselines We compare different types of feedback during reinforcement learning: (1) Hybrid feed- back, where the feedback a combination execution- frreenn--reeeeeee (SWE-RM) and execution-bass sig- nals, will be defined in Eq. D (2) Execution-free feedback only, where the feedback is provided solely by SWE-RM; (3) Execution-based feedbacc only, where the feedback derived exclusively from the execution results of fail2pass tests; (4) Poorly calibrated execution-free feedback, where the feedback comes from reward model with comparable TTS but lower AUC and weaker calibration ability.
Implementation Details We adapt GSPO (Zheng et al. 2025) which provides greater sta- bility for cfixture-of-Experrt RL training and define the Scoregr(y, T, patch) [0, 1]. and the overall reward is computed as:
execution-free feedback as
Scoregr Tu, patch,)) 0.5 + Score (q. 7a, patch,,)
if issue resolve, unfinished,
r(g. TR)
(1)
0 Score (g, R., patch,),
otherwise.
More details about the RL training such as data, hyper-parametent are shown in Appendix
5.2.2 EXECUTION-FREE FEEDBACK BENEFITS RL TRAINING
As shown in Figure 7 using the hybrid reward as described in Eq. [ yields the best RL performance and efficiency. Compared to the execution- based baseline, hybrid feedback improves pass @ by about absolute points (54.8% effective reward shaping. For execution-basee feedback only, we observe slower early gains and an early plateau due to the sparsity of the 0/1 signals and issues with test noise and coverage. In
51.8%) and shows faster, smoother improvements, indicating

---

Published as a conference paper at ICLR 2026
Table Performance after RL different SWE tasks other than SWE-Bench Verified, and Terminal Bench when using different feedback. SW.B. is short for
and Bold stands for the best.
METHOD
SW. LIVE (LITE)
SW.B. MULTILINGUAL
MULTI-SW.B B. TERMINAL BENCH
Hybrid
22.4
35.7
20.0
32.5
Execution-Free only
20.4
33.0
18.8
31.3
Exccution-based only
20.0
33.3
18.5
30.0
Poor Calibrated RM
12.0
21.0
10.0
15.0
contrast, execution-free feedback alone shows faster initial progress due to continuous signals but weaker convergence in later stages, likely caused by inaccuracies in its unverified signals. While our main evaluation focuses on SWE-Bench Verified, we additionally conducted experiments on broader suite of SWE tasks-including SWE- Bench Live (Lite), SWE-Bench Multilingual, Multi- SWE-Bench Mini, and Terminal Bench-to assess generalization beyond the original domain. As shown in Table 5 hybrid feedback consistently achieves better RL performance, while execution- free only feedback shows comparable results to execution-based only feedback. And if using feed- back from poorly calibrated RM, the model will also show significant decrease in other tasks. Overall, combining execution-fit free feedback with verifiable signals balances efficiency and reliabil- ity, achieving the strongest final results by providing both continuous and trustworthy rewards.
6 CONCLUSION
In this paper, we show that test-time scaling alone is an insufficient measure of verifier quality for SWE agents. Beyond top-1 ranking, reward models must also deliver strong discrimination (AUC) and reliable calibration (low ECE) to provide stable, useful signals, especially for RL. Guided by large-scale ablations on data scale, positive/negative ratios, policy mixtures, source composition etc., develop SWE-RM-4 30B MoE (3B activated) execution- -fee verifier with up to 256k context. SWE-RM achieves statc-of-the-arr open-source TTS gains on SWE-Bench Verified and, when used for RI. yields faster, more stable training and +3 absolute pass over rrecution-based feedback counterparts. This establishes execution-free, well-calibrated reward modeling as a practical and powerful foundation for advancing SWE agents in both TTS and RL.
ACKNOWLEDGGENTS
This project is partially supported by Hong Kong RGC ECS Grant 26218125, Hong Kong RGC CRF Grant C6003-24X 24Y, and NSFC Grant 62306177.
REFERENCES
Anthropic, Claude sonnet \ anthropic. URL http:tp::/wwwwarrrrpppp.....o./.
sornet
[Online; accessed 2025-09-24].
Ibragim Badertdinov, Alexander Golubev, Maksim Nekrashevich, Anton Shevtsov, Simon Karasik, Andrei Andriushchenko, Maria Trofimova, Daria Litvintseva, and Boris Yangel. Swe-rebench: An automated pipeline for task collection and decontaminated evaluation of software engineering agents, 2025. URL http::ttt////tt www.:iwww...graa.aoooooa abs 2005s//2552221111111
Andrew P. Bradlcy. The use of the area under the roc curve the evaluation of ma- chine learning algorithms. Pattern Recognit. 10:1145-1159, 1997. URL https://app.. Semant.cscholle wrw.oaaarrrrr/oorusiii:11110
DeepSeek-AL. Deepseck-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, org abs hatpr///ab/2201...0
2025. URL/tttps:
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In Doina Precup and Yee Whye Teh (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 1321-1330. PMLR, 06-11 Aug 2017. URL https: httpp:://proceecinng mir. wtt..eeeeciiaaa........eee//ooo. guol 7a h.tml.
10

---

Published as a conference paper at ICLR 2026
Zhenyu He, Qingping Yang, Wei Sheng. Xiaojian Zhong, Kechi Zhang, Chenxin An, Wenlei Shi, Tianle Cai, Di He, Jiaze Chen, Jingjing Xu, and Mingxuan Wang. Swe-swiss: A multi-task fine-tuning and rl recipe for high perfooerfor issue resolution. https: /wwp:///wwwwwwo nowionttin. Bo/SWE-Swe s inn
2025. Notion Blog.
Naman Jain, Jaskirat Singh, Manish Shetty, Liang Zheng, Koushik Sen, and lon Stoica. R2e-gym: Procedural environments and hybrid verifiers for scaling open-weights swe agents. arXiv preprint arXiv:2504.07164, 2025.
Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. SWE:-bench: Can language models resolve real-world github issues? In The Twelfth International Conference on Learning Representations, 2024. URL httttt::/opppree/ee/s
bet/Eorum: id d VTF 8yNQM66
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step, 2023. URL
https: //w.v//aavvvv//aa///a//aa.//aav. org/ abs/ 2305. 2005.00000))
Michael Luo. Naman Jain, Jaskirat Singh, Sijun Tan, Ameen Patel, Qingyang Wu, Alpay Ariyak, Colin Cai, Shang Zhu Tarun Venkat, Ben Athiwaratkun. Manan Roongta, Ce Zhang. Li Erran Li, Raluca Ada Popa, Koushik Sen, and Ion Stoica. Deepswe: Training a state-of- coding agent from scratch by scaling rl. httppp///ppeeeetyrooooooo-..... wtt.:/i...o....ioooooo..oooio.oo
Scaling |at.0222222000011111132220eeeaa0
2025. Notion Blog.
MiniMax. Minimax-m1: Scaling test-tetttteeee compute efficiently with lightning attention, 2025. URL
httpss//arxiv.orrr//8//////
OpenAl. Introducing swe-hench verified openai, Sept 2025. httpss//:////pe com/
[Online; accessed 2025-09-22].
Jayy Pan, Xingyao Wang, Graham Neubig. Navdeep Jaitly, Heng Ji, Alane Suhr, and Yizhe Zhang. Training software engineering agents and verifiers with swe-gym. In Proceedings of the 42nd International Conference on Machine Learning (ICMI. 2025), 2025. URL. httts::///raa:
org/abs/ 2412.412222 21139 arXiv:4412.21139, accepted ICML 2025.
Qwen Team. Qwen3 technical report, 2025.
Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. Hybridflow: A flexible and efficient rlhf framework. arXiv preprint arXiv: 2409.19256 2024.
Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-Im: Training multi-billion parameter language models using model par- allelism. arXiv preprint arXiv: (ee.7/999000000 2019.
GLM-4.5 Team, Aohan Zeng, Xin Lv, Qinkai Zheng. Zhenyu Hou, Bin Chen, Chengxing Xie, Cunxiang Wang, Da Yin. Hao Zeng, Jiajie Zhang, Kedong Wang, Lucen Zhong, Mingdao Liu, Rui Lu, Shulin Cao, Xiaohan Zhang, Xuancheng Huang, Yao Wei, Yean Cheng, Yifan An, Yilin Niu, Yuanhao Wen, Yushi Bai, Zhengxiao Du, Zihan Wang, Zilin Zhu, Bohan Zhang, Bosi Wen, Bowen Wu. Bowen Xu, Can Huang, Casey Zhao, Changpeng Cai. Chao Yu, Chen Li, Chendi Ge. Chenghua Huang, Chenhui Zhang, Chenxi Xu, Chenzheng Zhu, Chuang Li, Congfeng Yin, Daoyan Lin, Dayong Yang, Dazhi Jiang, Ding Ai, Erle Zhu, Fei Wang. Gengzheng Pan, Guo Wang, Hailong Sun, Haitao Li, Haiyang Li, Haiyi Hu, Hanyu Zhang, Hao Peng, Hao Tai, Haoke Zhang, Haoran Wang. Haoyu Yang, He Liu, He Zhao, Hongwei Liu, Hongxi Yan, Huan Liu, Huilong Chen, Ji Li, Jiajing Zhao, Jiamin Ren, Jian Jiao, Jiani Zhao. Jianyang Yan, Jiaqi Wang, Jiayi Gui, Jiayue Zhao, Jie Liu, Jijie Li, Jing Li, Jing Lu, Jingsen Wang. Jingwei Yuan, Jingxuan Li, Jingzhao Du, Jinhua Du, Jinxin Liu, Junkai Zhi, Junli Gao, Ke Wang. Lekang Yang, Liang Xu, Lin Fan, Lindong Wu, Lintao Ding, Lu Wang. Man Zhang, Minghao Li, Minghuan Xu, Mingming Zhao, Mingshu Zhai, Pengfan Du, Qian Dong, Shangde Lei, Shangqing Tu, Shangtong Yang.
11

---

swe-benc----bbeh----e [Online; ac- Published as a conference paper at ICLR 2026
Shaoyou Lu, Shijie Li, Shuang Li, Shuang-Li, Shuxun Yang, Sibo Yi, Tianshu Yu, Wei Tian, Weihan Wang, Wenbo Yu, Weng Lam Tam, Wenjie Liang, Wentao Liu, Xiao Wang, Xiaohan Jia, Xiaotao Gu, Xiaoying Ling, Xin Wang, Xing Fan, Xingru Pan, Xinyuan Zhang, Xinze Zhang, Xiuqing Fu, Xunkai Zhang, Yabo Xu, Yandong Wu, Yida Lu, Yidong Wang, Yilin Zhou, Yiming Pan, Ying Zhang. Yingli Wang, Yingru Li, Yinpei Su, Yipeng Geng, Yitong Zhu, Yongkun Yang. Yuhang Li, Yuhao Wu, Yujiang Li. Yunan Liu, Yunqing Wang. Yuntao Li, Yuxuan Zhang, Zezhen Liu, Zhen Yang, Zhengda Zhou, Zhongpei Qiao, Zhuoer Feng, Zhuorui Liu, Zichen Zhang, Zihan Wang, Zijun Yao, Zikang Wang, Ziqiang Liu, Ziwei Chai, Zixuan Li, Zuodong Zhao, Wenguang Chen, Jidong Zhai, Bin Xu, Minlie Huang. Hongning Wang, Juanzi Li, Yuxiao Dong, and Jie Tang. Glm-4.5: Agentic, reasoning, and coding (arc) foundation models, 2025a. URL https: 7/arxiv. www.voorg/aaae hat///ab///// /500b.///58..0 06471
Kimi Team, Yifan Bai, Yiping Bao, Guanduo Chen, Jiahao Chen, Ningxin Chen, Ruijue Chen, Yanru Chen, Yuankun Chen, Yutian Chen, Zhuofu Chen, Jialei Cui, Hao Ding, Mengnan Dong, Angang Du, Chenzhuang Du, Dikang Du, Yulun Du, Yu Fan, Yichen Feng, Kelin Fu, Bofei Gao, Hongcheng Gao, Peizhong Gao, Tong Gao, Xinran Gu, Longyu Guan, Haiqing Guo, Jianhang Guo, Hao Hu, Xiaoru Hao, Tianhong He, Weiran He, Wenyang He, Chao Hong, Yangyang Hu, Zhenxing Hu, Weixiao Huang. Zhiqi Huang. Zihao Huang. Tao Jiang, Zhejun Jiang, Xinyi Jin, Yongsheng Kang. Guokun Lai, Cheng Li, Fang Li, Haoyang Li, Ming Li, Wentao Li, Yanhao Li, Yiwei Li, Zhaowei Li, Zheming Li,, Hongzhan Lin, Xiaohan Lin, Zongyu Lin, Chengyin Liu, Chenyu Liu, Hongzhang Liu, Jingyuan Liu, Junqi Liu, Liang Liu, Shaowei Liu, T. Y. Liu, Tianwei Liu, Weizhou Liu, Yangyang Liu, Yibo Liu, Yiping Liu, Yue Liu. Zhengying Liu, Enzhe Lu, Lijun Lu, Shengling Ma. Xinyu Ma, Yingwei Ma, Shaoguang Mao, Jie Mei, Xin Men, Yibo Miao, Siyuan Pan, Yebo Peng, Ruoyu Qin, Bowen Qu, Zeyu Shang. Lidong Shi, Shengyuan Shi, Feifan Song, Jianlin Su, Zhengyuan Su, Xinjie Sun, Flood Sung. Heyi Tang, Jiawen Tao, Qifeng Teng, Chensi Wang, Dinglu Wang, Feng Wang, Haiming Wang, Jianzhou Wang. Jiaxing Wang, Jinhong Wang, Shengjie Wang, Shuyi Wang, Yao Wang, Yejie Wang, Yiqin Wang, Yuxin Wang. Yuzhi Wang, Zhaoji Wang, Zhengtao Wang. Zhexu Wang, Chu Wei, Qianqian Wei, Wenhao Wu, Xingzhe Wu, Yuxin Wu, Chenjun Xiao, Xiaotong Xie, Weimin Xiong, Boyu Xu, Jing Xu, Jinjing Xu, L. H. Xu, Lin Xu, Suting Xu, Weixin Xu, Xinran Xu, Yangchuan Xu, Ziyao Xu, Junjie Yan, Yuzi Yan, Xiaofei Yang. Ying Yang. Zhen Yang. Zhilin Yang. Zonghan Yang. Haotian Yao, Xingcheng Yao, Wenjie Ye, Zhuorui Ye, Bohong Yin, Longhui Yu, Enming Yuan, Hongbang Yuan, Mengjie Yuan, Haobing Zhan, Dehao Zhang, Hao Zhang, Wanlu Zhang, Xiaobin Zhang, Yangkun Zhang. Yizhi Zhang. Yongting Zhang, Yu Zhang, Yutao Zhang, Yutong Zhang, Zheng Zhang, Haotian Zhao, Yikai Zhao, Huabin Zheng, Shaojie Zheng, Jianren Zhou, Xinyu Zhou, Zaida Zhou, Zhen Zhu, Weiyu Zhuang, and Xinxing Zu. Kimi k2: Open agentic intelligence, 20534
2025b. URL
OpenHands Team. Sota on swe-bench verified with inference-time scaling and critic model,
cessed 2025-09-22).
Cheng Wang. Calibration in deep learning: A survey of the state-of-the-arr, 2025. URI. https:
//arxiv.org/abs////
01222
Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma. Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng, Heng Ji, and Graham Neubig. Openhands: An open platform for AI soft- ware developers as generalist agents. In The Thirteenth International Conference on Learning Representations, 2025. URL https::///// / / revi n www.ffrrrriiooomoommoo (=tpummmiio00dd0a0D0o00
Yuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux, Lingming Zhang, Daniel Fried, Gabriel Synnaeve, Rishabh Singh, and Sida I. Wang. Swe-rl: Advancing 1m reasoning via rein- forcement learning on open software evolution. arXiv preprint arXiv:2502.18449 2025.
Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. Agentless: Demystifying llmmbased software engineering agents. arXiv preprint, 2024.
12

---

Published as a conference paper at ICLR 2026
John Yang, Carlos E Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik R Narasimhan, and Ofir Press. SWE-agent: Agent-computer interfaces enable automated software engineering. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL
https: / tttpaaiiiorrg/aoeaa///ooo/ooooot abss/////////// 2405. 2ttps//43...15995 15793
John Yang, Kilian Lieret, Carlos E. Jimenez, Alexander Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, and Diyi Yang. Swe-smith: Scaling data for software engineering agents, 2025.
htttpps:/axxx..orcc//ooooooo 2ttp.//bb///54.1795999 21798
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guang- ming Sheng, Yuxuan Tong, Chi Zhang. Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei- Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, and Mingx- uan Wang. Dapo: An open-source IIm reinforcement learning system at scale, 2025. URL
https: /wtp/aaaavvv...e/.a.eeeeee..ae.a otg/ abo/a//3//2222 14476
Jiajun Zhang, Zeyu Cui, Jiaxi Yang, Lei Zhang. Yuheng Jing. Zeyao Ma, Tianyi Bai, Zilei Wang, Qiang Liu, Liang Wang, et al. From completion to editing: Unlocking context-aware code infilling via search-and-replace instruction tuning. arXiv preprint arXiv:2601. 13384, 2026.
Chujie Zheng, Shixuan Liu, Mingze Li, Xiong- Coong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, Jingren Zhou, and Junyang Lin. Group sequence policy op- timization, 2025. URL htlpttts::// //a //t////axiiii www.oor/aba///...o.. 18071
13

---

Published as a conference paper at ICLR 2026
Table 6: Comparison of SWE-RM with other execution-free verifiers' setting: SWE-Gym Veri- fier (Pan et al. 2025), R2E-Gr Gym Verifier (Jain et al., 2025), OpenHands Critic (Team 2025) and
DeepSWE Veriher (Luo et al. )025). means the statistic are not disclosed. Our training data
source contains SwE-Gym (Pan et al. (025). R2E-Gym (Jain al. 2025), SWE-smith (Yngg al. 2025 and SWE-rebench (Badertdinov et al. 2025).
REWARD MODEL
DATA
REPO
POLICY
SOURCE
CONTEXT LEN.
PROMPT
SWE-Gy Gym Verifier
2636
11
Mix
SWE-Gi WWE-Gymm
32k
Traj.
R2F--Gym Verifier
3321
10
Claude-3.5
R2F-Gym
32K
Traj Patch
Openhands Critic
11
SWE-Gym
32k
Traj.
DeepSWE Verifier
10
R2F-Gym
76k
Traj. + Patch
SWE-RM
100k
~ 170
Mix
Multiple*
256k
Traj. Patch
A
THE USE OF LARGE LANGUAGE MODELS
For this paper. large language (LLM))(LLMs) used solely for polishing the writing. The entire research process, including but not limited to ideation, was conducted without any assistance from L.Mss.
B EXTENDED RELATED WORK
B.1 VERIFIERS FOR SWE TASKS
In Software Engineering (SWE) tasks, there are mainly two kinds of verifiers: (1) execution-based verifiers and (2) execution-free verifiers. Execution-based verifiers typically consist of various types of unit tests, either human-written or model-generated. Agentless (Xia et al., 2024), R2E-Gym (Jain) et al. 2025), and DeepSWE (Luo et al.) 2025) have demonstrated the effectiveness of execution- based verifiers in test-time scaling by ranking patches based on the number unit tests passed. However, exccution-based verifiers struggle to distinguish between patches that achieve the same number of test passes, and they suffer from the inherent unreliability of poorly written or model generated unit tests. In contrast, irecution-free verifiers typically model-based and provide a continuous score for a given trajectory, allowing finer-grained discrimination. Early work such SWE-Gym (Pan al. 2025) and OpenHands Critic (Team 2025). has initially explored naive execution-free verifiers with limited coverage and were confined to relatively simple settings. Subse- quent work, including nym (Jain al. (025) and DeepSWE (Luo et al. (2055), has shown that combining execution- axccetion-based and execution- free verifiers leads to improved test-time scaling. Nev- ertheless, as summarized in Table 6 the exploration of execution-free verifiers remains preliminary and has largely focused only on scaling performance. Different from them, this work first demon- strates that test-time scaling (TTS) performance alone not a sufficient measure of an exccution free verifier's 's quality its accuracy and calibration equally important. We further present system- atic study on training versatile execution-fit free verifiers, considering factors such as data scaling, data ratio, policy mixture, source mixture, and context length.
B.2 AGENTIC REINFORCEMENT LEARNING FEEDBACK IN SWE TASKS
Unlike non-agent scaffolds such as Agentless Xia al. 2024). which are single-turn and pipeline- based, agent scaffolds in SWE such as OpenHands (Wang et al. 2025) and SWE-Agent (Yang et al. 2024) deploy sandbox environment that allows models to interact completion, a fail-to-pass co-pass unit test is executed to assess whether the generated patch resolves the is- sue. This type of execution-basee based feedhack plays a role similar to that of rule-based metries in math problems (DeepSeek-Al) 2023). as it aims to provide verifiable and relatively accurate reward for reinforcement learning. Such leedback has been widely adopted in recent coding agent training, in- cluding Qwen3-Coder (Qwen Team 2025), GLM-4.5 (Team et al. 2025a), and MiniMax-MI (Min- MMxx 2025). While effective in principle, execution- based feedback limited by the quality of the test suites it relies It assumes that the test oracle is perfect, yet in practice, many test cases genereee, making them an unreliable proxy for correctness. addition, raccttionbaasee feedback cannot. distinguish between trajectories that yield the same outcome, either passing or fail-
multi-turn setting. Upon
14

---

Published as a conference paper at ICLR 2026
ing test. Consequently, it often provides signals that overly sparse misleading for reinforcement learning. In this work. we are the first to integrate execution- free feedback into SWE agentic reinforcement learning. We find that versatile execution-free feedback can offer more fine-grained rewards, and lead to improved training efficiency and performance.
C THEORETICAL LINK BETWEEN TTS, AUC, AND ECE AND RL DYNAMICS
In this section, we make explicit how the three metrics TTS, AUC, and ECE correspond to three distinct failure modes of reward models (RMs) when used as optimization signals RL. Throughout this section, let and c(r) E {0, 1} the binary correctness label (the ideal true correctness). If the RM score used directly the reward, the volicy-gradiies update is
denote sampled trajectory from the policy, r(r) E [0. 1] is the RM score,
(oJ(O) R E (r(r) Vo Vogog mT(T))
The ideal update using the true correctness label
(0)* Ey (c(T) V log (T(r))
The gap between (2) and determines the stability and correctness of RL. Here use simple policy gradient as an example, which has same nature to GRPO (DeepSeck-Al 20/25//GGO (Zheng al. 2025), one can simply extend the analysis GRPO setting. We then analyze below how TTS, AUC, and ECE each correspond to different component of this gap.
C.1
TTS: EXTREME- -TOE ERRORS AND THEIR IMPACT ON RL
TTS@K k evaluates whether the highest-scored trajectory among RM-scored samples correct:
TTS@k = Pr(c(r") = 1), T' arg max r(Ti).
<E(l:k]
Thus (1-TTS@K) is exactly the probability that an unresolved trajectory receives the largest reward among the samples.
Implication for RL When the top-1 trajectory r* is incorrect (c(T*) = 0), we know that
r(r")= max r(T.). [E(1:k]
This does not imply that negatives have larger average rewards than positives, but it does imply that the batch contains a negative trajectory with the largest reward weight. Since policy-gradien updates scale linearly with reward,
VoJ(O) ~
r(T)) Vo log (T()))
the contribution of rft becomes a dominant term in the update, even if all other negatives have small scores.
Conditioning on correctness of the top-ranked sample gives the decomposition:
VoJ(0) TTS E(jpdate e(r") 1] + (1 TTS@k) E Ejjudate dominated by rī]. (5)
Under the event e(T*) )= 0, RL receives the strongest possible reward signal for unresolved tra- jectory. This causes the policy Te to increase the probability of sampling this undesirable behavior, and the effect compounds over iterations.
C.2 AUC: PAIRWISE RANKING QUALITY AND REVERSED- -RAADIENT FREQUENCY
AUC measures the probability that the RM correctly orders a positive trajectory above a negative one (for all trajectories):
AUC =
(6)
Therefore the mis-ranking probability is
AUC Pr(r((-)>r(t))
15

---

Published as a conference paper at ICLR 2026
Mis-rankings imply reversed gradient contributions Whenever r(()), the RM assigns higher reward to an incorrect trajectory. The corresponding policy-gradient contributions satisfy
r( r(r-) ) Ve log (T( >
r(T+) V log (T())),
which is opposite to what the ideal update would encourage. Thus fraction - AUC of all positive-negative trajectory pairs induce updates pointing in the wrong direction. Conditioning on the correctness of ranking yields the decomposition
(0) AUC Elcorrect pair update] + AUC) Efreversed pair update].
(9)
Consequences for RL stability Since gradient contributions aggregate linearly, the expected frac- tion of bad (reversed) updates grows exactly in proportion to (1 - AUC). Therefore,
low AUC many reversed- -radient terms unstable or divergent RL behavior.
Unlike TTS (which concerns only the extreme top), AUC measures global ranking correctness, which affects every sampled trajectory in RL.
C.3 ECE: CALIBRATION ERROR AND SYSTEMATIC BIAS IN RL UPDATES
For reward model that outputs a score r(r) (interpreted as confidence in "this trajectory is good") and binary "good/bad" ground truth E {0, (). reward model is calibrated if
Pr(c= = | ====d a) = ax Va E [0, 1].
(10)
Using the binned approximation for the reward-model scores across a dataset of trajectories, we can compute:
M
Bml
ECE-
acc conf(B..)
(11)
m==
where Bm is the divided bins and:
conf Con)(Bm)
Σ ri iE Bm
ace Bm) =
Ci
|Bml
[mm i
Calibration and unbiased RL updates If Eq. 1O holds, then Eje = r. e.g. For all trajecto- ries to which the model assigns confidence score of (r = 0.7), the actual proportion of successful trajectories should also be 70%. This exactly the statement (Eje matches the model's own predicted confidence (r). And thus
0.7] = 0.7), meaning
Vo (T) Vo log mo(r))
(12)
meaning the RM induces no systematic bias in the expected gradient.
Bias induced by miscalibration In general, the deviation between the RM-induced and ideal updates is
A bias Erowo Vo Volog (T) (r(T) Blc| [[(r)))]
(13)
Define the calibration bias function
b(a) = Efc | a) a,
so that
r(r) - Elc r(r)] -b((T)))
Here b(a) measures the calibration bias confidence level among all trajectories for which the RM predicts score r = a, E|c r = a. is the true success frequency, while a is the predicted success probability. Their difference therefore captures the systematic over- under-confidence of the reward model at that score. ECE is then a binned approximation of the expected magnitude of this bias over the score distribution, meaning high ECE = large systematic distortion in (13
16

---

Published as a conference paper at ICLR 2026
Additional effect: gradient variance inflation.
Write the RM-induced gradient estimator
g(T) r(r) V log (T(T).
Using the decomposition
r(r) = B(c(F) r(()]]] - b(r(F)),
obtain
g(T) B[c(T) (((()) Vo (T( b(((T)) Vo log To(
(")r)
80(T)
Here 9 (r) is the "calibbatedd" part of the gradient (which would be obtained if we replaced r(r) by the true success probability E(c(T) r(r)), while 5g(T) is purely micalibration-induee noise
term. The variance of g(T) decomposes
Var [g)] Varlg* (+)) + Var[ōg(T)] + Cov (g" (r),
In particular, always have
Var[g(T)] 2 Varlg" (T)],
with the excess variance controlled by
Var[óg(r)] Var (ar()(r(r)) Ve log Te(T)]
Thus calibration error b(r) couples multiplicatively with the policy gradient Ve log (T()). injecting additional variance into the gradient estimator. Since ECE is discrete approximation of [, [[b(r)]] higher ECE typically implies a larger variance contribution from 5g(T) and therefore less stable RL. training.
D REWARD MODEL TRAINING DETAILS
D.1 DETAILED TRAINING SETUP
Data Collection To train reward model, we first rollout and collect over 400k multi-turn tra- jectories up to 100 iterations using OpenHands (Wang et al. (025) and SWE-Agent (Yang al. 2024). which are two widely used open-sourced coding agent scaffold using different policy mod- els and data sources, including SWE-Gym Paan al. 2025). SWE-rebench (Badertdinov et al. 2025), SWE-smith (Yang et al. 2025), and R2E-Gym Jain et al. 2025). Specifically, we adapt Qwen Canee--ooderaaae
Qwen3 - Coder-FFoFFooo ash, -luude4--sonee for rollout. Since a large portion of the data might be wedesolved and some trajectories may be incomplete or contains bad tool calls, thus the final usable trajectories for training is around 100k.
These trajectories are then labeled as positive(resolved) or negative(unresolvee) based on their exe- cution results with the provided fail2pass test. Though some them might be noisy as we discussed that unit tests might not be able to truly reflect the correctness of the generated patch, we applied data cleaning such as filtering out instances without any successful trajectory (typically cases affected by surrstr unit tests under-specified descriptions) to maintain the highest possible label quality. With data filtering, we believe sufficiently large and diverse dataset enables the RM to learn denoised and generalized correctness signal despite noisy supervision.
Scaffold and Model For training scaffold, we adapt Megatron (Shoeybi et al. 2019 for super- vised fine-tuning which enables efficient long context training. For rollout, we use SGLang together with agent scaffold OpenHands (Team) 2025) and SWE-Agent (Yang al.) 2024). While for the base model, we use Owen3-30B-AA This MoE architecture is not a choice that claiming the calibration advantages. rather, we follow the prevailing practice in state-of-the-art coding agents (e.g., Qwen3-Codes Kimi-K2, MiniMax-M1/2), which predominantly adopt MoE backbones. Using the same backbone ensures compatibility with existing pipelines(e.g. infra) and allows reward model and trained policies to be directly inte- grated without additional adaptation overhead. Our focus is therefore on reward-mode training and calibration, while architectural comparisons (MoE vs. Dense Adapters) are left future work.
(Qwen Team (025) as the backbone for further training-
17

---

Published as a conference paper at ICLR 2026
Baselines We compare trained execution- free verifier with the following different execution- free verifiers as well as execution- verifiers: (1) Agentless (Xia et al., 2024), which proposes exccution-baae based method that generates reproduction tests for each trajectory and re- re-ranks based
on passed test numbers; (2) SWE-Gym Verifier (Pan et al. 2025). which releases Qwen2. 5 5- 32B based execution-free verifier trained SWE-Ciym; (3) DeepSWEE-EB Verifier (Luo al. 2025):
which is the execution-basee component of current State-of-the-art DeepSWE Hybrid-TTS. which
also the improved version of R2E-Gym Execution-based verifier (Jain al. 2025 with similar
mechanism to Agentless. (4) DeepSWE EF Verifier (Luo et al. (2025): which is the execution- -ree
component of current ofate-of-the-ar DeepSWE Hybrid-TTS also the improved version of R2E- Gym execution-free verifier.
Evaluation Setup We mainly conduct the evaluation on SWE-bench Verified (Jimenez et al, 2024) which is curated subset of 500 human-verifit tasks for reliably assessing model perfor- mance on real-world software engincering tasks. For test-time scaling and further accuracy and calibration evaluation, we the most widely used open-sourced coding agent scaffold Open- Hands to collect 32 independent runs for each instance, resulting 32 x 500 trajectories in total. The sampling configs use temperature of 1.0, top_p of 0.95, max_iterationn of 100. Accuracy is calculated by AUC score on all 16k trajectories while calibration is measured by expected calibra-
tion error(Guo et al. 2017), where ECE ΣM [B..
acc(Bm) - conf( conf( =
n
|m. ΣiEB,, ri. acc Bm) = 1 |P...] ΣieBm. Gi. And we follow common practice to divide confi-
dence into 10 bins (M = 10).
Pass defines the resolve rate of model with least one successful solution among trajectories which is also the upper bound while RM@K defines the resolve rate of final selected trajectories from samples. For every 32, we obtain the mean and variance of random runs for fair assessment.
D.2 TRAINING TEMPLATE
Following SWE-Gym (Pan et al. 2025 we adapt from their template which splices all turns (model action output and tool responses) and end with a YES/NO token for reward model classification. Given the full multi-turn trajectory, the model is prompted to output single special token, either <YES> (resolved) or <NO> (unresolved). And the supervised fine-tuning utilizes standard next- token prediction loss this special token. At inference time, by obtaining the log probability of the special token <YES>(1,) and <NO>(I,), the final score r is calculated by exp (l,)) (4g) + exp (l,)), which maps to continuous reward model score [0, For tool parsing, adapt Qwen3 -Coder's XMI. format, which optimized for code-related argument parsing. The example trajectory shown in Figure
D.3 TRAINING HYPER-PARAMETERS
For reward model training hyperparameters, adapt 256k context window to support scoring for complex questions which contains extremely long contexts. The global batch size is set to 128. Also widely used AdamW optimizer and cosine decay learning rate scheduler are used. The detailed training hyperparameters are listed in Table Tablee]a below. We use 4 nodes of H100 for large scale, long context reward model training, which takes around 20 hours for 100k samples.
Table The detailed training hyperparameters for reward model.
Model Size
Global Batch Size
Learning Rate
Schedular
LR Warmup
Optimizer
Epoch
30BA3
128
7e-6 + 7e-7
cosine cosine
3%
AdamW
1
235BA22
128
7e-6 7c-7
3%
Adam 1
480BA35
128
7e-6 + 7
cosine
3%
Adam
We also summarize the detailed training setting for Verifiers we used (Verifier A, Verifier and Poor calibrated RM in Figure 7 in Table below.
18

---

Published as a conference paper at ICLR 2026
<user>
Please evaluate the following interaction between an Al assistant and a user:
INTERACTION LOG ===
System Message that describes the assistant's behavior [system message here]
Turn USER
Turn ASSISTANT
Turn ASSISTANT
Let's ignore the existing error and focus on our specific issue. <tool_cal
innction=str_replace editor>
<parameter=path>
wworkspace/atrroyy_sspeeee sptprrduueppceuueeeeeeeee 1.2/ppecreduce/tests
</parameter>
<parameter=commandd
view
</parameter>
</function>
</tool_call
*** Turn 10 USER ***
stoolrrespone>
Here's the files and directories up to 2 levels deep in /workspace/ 1.2/specreducc/tests.
resooorrsspose>>>
== END INTERACTION ===
GENERATED PATCH ===
[patch here]
END GENERATED PATCH ===
Based the above interaction and the generated patch, did the assistant
successfully resolve the user's initial request? Respond with YES NO.
</user>
<assistant>
NO
Figure 8: Prompt template for reward model training. Pink refer the tool parsing XMI. format example.
D.4 ADDITIONAL ANALYSIS ON CONTEXT CONSTRAINT
In $ 4.3 we show a substantial performance increase when we raise the context window from 32k 256k tokens.
The larger context allows the model to cover far tokens of the input (for
19

---

Published as a conference paper at ICLR 2026
Table Comparison of detailed training setting of Verifier A, Verifier B in Figure 2,3,4,5 and Poorly Calibrated RM in Figure 7.
REWARD MODEL
DATA
RATIO
POLICY
SOURCE
CONTEXT LEN.
Verifier
20k
Mix-Policy
Mixed-Source
256k
Verifier B
20k
1:4
Off-Policy
SWE-Rebench
256k
Poorly Cali. RM in Figure 7
5k
1:2
Off-Policy
SWE-Rebench
256k
example, longer trajectories, multi-file code, richer history) without needing to compress or truncate information. When the window is small, many long trajectories cannot be scored all (i.e., fall out of the window) and thus receive no valid reward signal, meaning correct but long solutions are effectively dropped from TTS selection. With 256 k context, we reduce the "no-score" rate (increase the score rate), thereby enabling full-trajectory scoring and enabling our verifier to pick up solutions that would otherwise be ignored. And another reason for us to insist on 256k reward model training is that modern high-end coding agent policy models such as Claude, GPT-5, Qwen3- Coder-Max support 256k token windows (and up to M tokens in some settings). The community therefore urgently needs reward model that is compatible with this scale, so trajectories from such long-context agents can be properly scored - yet many existing reward/verifier models are constrained to much shorter context windows and thus cannot handle those long trajectories.
By aligning our verifier to the 256 k-token scale we fill crucial gap. Finally, we acknowledge that increasing the context to 256k tokens is not free it incurs higher memory usage. However, since our output generation only contains one token, all prompt computation can be run in parallel, thus the latency is more or less the same for different context length inference time. While for deployment, a 256k model takes only around 2x GPU memory usage than a 32k model, a user with 2x A100 GPUs can easily deploy the model with high efficiency.
E
RL TRAINING DETAILS
E.1 RL SETUP
Models We conduct reinforcement learning based on a Qwen3-30B-A3B (Qwen Team 2025)
with SFT warm-up. WE use in-house collected agentic trajectories including but limited to SWE tasks to fine-tune the base model. Then this fine-tuned model served as the starting point of RL experiments. We use MoE architecture, following the prevailing practice in kate-of-the-are coding agents (c.g., Qwen3-Coder (Qwen Tcam 22025), Kimi-K2 (Team et al. 2025b). MiniMax- M1/2 MiniMax 2025)), which predominantly adopt MoE backbones. Using the same backbone ensures compatibility with existing pipelines(e.2 infra) and allows our trained policies to be directly integrated without additional adaptation overhead.
Implementation Details We train the model using curated data from SWE-Gym (Pan et al. 2025) and SWE-rebench (Badertdinov et al. (225)) with batch size of 64. For each problem, sample 16 rollouts, use a maximum of 100 iterations, and set the context length to 128k. This context length was selected because it accommodates most problem cases within single context window while being more cost-efficient than 256k context window. The training data is further filtered by difficulty, as problems that are either too easy or too difficult can negatively affect RL performance.
Unlike single-turn RL training, where the model needs to generate only one response per prob- lem, agentic RL requires interaction with an agent scaffold (i.e.. tool calls) across multiple turns to construct a full trajectory. Specifically, given a problem y. the policy model generates an ac- tion a; and then receives tool response Ui, repeating this process T times to form a trajectory {ai,on 01 02, 02..... ... 0r, or). During optimization, tool responses masked. Instead of the GRPO objective, we adopt the GSPO objective (Zheng et al., 2025), which provides greater stability, particularly in ceature-of--Expeer (MoE) RL training:
JasPo (0) B.p-D.bt,, (40((19)))
1 min
[T(Ti ④) To (T, 9)
[.
cr,plip (
((ma(r) 9))
[21 ,1 - + A.
(14)
To (Ts
20

---

Published as a conference paper at ICLR 2026
with the baoup- advantage estimation:
A. = r(q, mean ((r(g, 7))2)) std (fr(g. =))_
(15)
The optimization integrate several standard tricks such as Clip High (Yu et al. 2025), NO KL loss Yu et al. 2025 etc.
21


Reviews:
[{"review_id": "eqZMSyWxSZ", "reviewer": "Reviewer_YTBh", "summary": "This paper proposes to train an execution-free reward model (RM) to address the reliance on the real-world execution feedback for training software engineering (SWE) agents, which is often sparse, unreliable, and hard to scale. The paper's core insight is athat good performance on test-time scaling (TTS), a common metric for verifiers, does not gaurantee good performance when the verifier is used as a reward model for RL. The authors identify 2 crucial properties for a robust RM: discriminative ability (measured by AUC) and confidence reliability (measured by calibration, or ECE). Based on this insight the authors conduct comprehensive experiments to determine how to train a robust RM, by investigating factors like training data sclae, positive-to-negative data ratios, data source etc. The authors train SWE-RM a 30B MoE model, starting with Qwen3-30B-A3B, which achieves solid improvement of pass@1 on SWE-Bench Verified.", "questions": "- What dataset Figure 4 reports on? I would love to know more details about how the authors compute the ECE, e.g. on which dataset, how many rollouts, sampling config, etc.\n- How were the base reward constants (+1.0 for resolve, -0.5 for unfinished, 0.0 for otherwise) chosen? Did you experiment with other values, and how sensitive is the final RL performance (Figure 7) to these specific hyperparameters?\n- The data scaling experiment (Figure 6) and the data composition experiments (Tables 1 & 3)  are presented separately. For clarity, was the 100k data point in the scaling experiment trained using the optimal 2:1 ratio and policy/source mix found later? And conversely, were the composition experiments in Tables 1 & 3 conducted at the full 100k data scale, or a smaller one? I'm trying to understand the interplay between data quantity and quality/composition.\n\nFor the rest, please see above for the weakness part.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper's primary strength is its core insight, which is clearly motivated and empirically demonstrated. The finding that test-time scaling (TTS) performance is an insufficient proxy for a reward model's utility in RL  is interesting. \n- The experiments are well-executed, a series of ablation is done on data scale, data composition, policy mixture and context length. This provides good guidance and insight for the community to identify the source of gain.\n- The writing is exceptionally well-written, logically structured, and easy to follow. The paper also presents strong empirical results when the RM is used as a verifier for TTS and as additional source of signal during RL.", "weaknesses": "- Opacity of the \"Poor Calibrated RM\" (Verifier B): The entire paper's motivation hinges on the comparison between \"Verifier A (Good AUC & Cali.)\" and \"Verifier B (Bad AUC & Cali.)\". We are shown they have similar TTS but different RL outcomes. However, the paper never explains how Verifier B was trained or why it has bad calibration. Is it off-the-shelf (together with verifier A) or trained by the authors (if so could authors explain how to train it)? Is it the same as \"poorly calibrated RM\" in Figure 7? \n- There's a slight tension in the paper's premise. The RM is trained on positive/negative labels generated by \"execution results with the provided fail2pass test\". However, the paper's introduction effectively argues that these very tests are unreliable, sparse, and \"in some cases, entirely unrelated to the target issue\" . The paper \"believe reward model can still achieve relative good performance by training on a large number of data\", which is a reasonable assumption (i.e., the model learns a denoised, generalized signal). Still, this is a key point: the RM is trained on the same (flawed) signal it is meant to improve upon. A deeper discussion of this \"denoising\" effect would be welcome. And the issue of noise is also coupled with the Figure 7 result of execution-only run, where the authors attributes its performance drops to \"issues with test noise and coverage\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "jGZKV101El", "reviewer": "Reviewer_dWfQ", "summary": "This paper proposes SWE-RM, a large-scale, execution-free reward model for SWE agents, and shows it is effective both for test-time scaling (TTS) and as a reward signal in RL. Execution-based verifiers rely on incomplete or noisy unit tests, producing sparse feedback, while execution-free models provide denser, more informative signals. The authors argue that TTS performance alone is insufficient for evaluating verifier quality and introduce AUC (discriminative ability) and ECE (calibration error) as complementary metrics. Extensive ablations explore the effects of dataset size, composition, positive/negative ratios, and policy mixtures. SWE-RM achieves strong TTS gains on SWE-Bench Verified and modest RL improvements when combined with execution-based rewards.", "questions": "- The authors show cases where similar TTS but better AUC/ECE give better RL. Is there evidence that improving calibration directly improves RL, or guidance on thresholds?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Showing that identical TTS performance can yield divergent RL outcomes is an important empirical finding with implications for verifier evaluation and selection.\n- The ablations on data scale, source composition, and context length provide concrete, actionable insights for building robust SWE reward models.\n- The model supports very long (256k) contexts so the verifier can score large numbers of trajectories.\n- A single verifier that supports both evaluation-time reranking and RL training is practically useful.", "weaknesses": "- The narrative sometimes shifts between TTS (used for evaluation/selection) and the execution-free RM (the actual trained model), which can make the pipeline slightly harder to follow. Being explicit about where TTS stops and supervised RM training begins would help.\n- Because the paper starts from TTS limitations, it would help to spell out the actual supervised reward-model objective earlier so readers don’t assume TTS is the training signal.\n- Evaluation is limited to SWE-Bench Verified, leaving generalization to other SWE tasks or domains untested.\n- The model's scale and context requirements may constrain practical deployment.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZbR6BSjTmR", "reviewer": "Reviewer_aGbU", "summary": "This paper proposes SWE-RM, an execution-free reward model for SE agents that provides continuous feedback without requiring unit test execution. The authors argue that execution-based feedback has limitations including sparsity and dependence on test quality, and propose using a learned reward model instead. They identify three key metrics for evaluating reward models: test-time scaling (TTS) performance, classification accuracy (AUC), and calibration (ECE). Through empirical ablations on training data scale, positive/negative ratios, policy mixtures, and data sources, they develop a 30B parameter mixture-of-experts model that achieves state-of-the-art results on SWE-Bench Verified for both test-time scaling and reinforcement learning.", "questions": "Please refer to questions raised in weaknesses.", "limitations": "", "rating": 2, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper makes a reasonable observation that test-time scaling performance alone is insufficient for evaluating reward models intended for RL use. The empirical finding in Figure 2 showing that two verifiers with similar TTS can have drastically different RL performance is interesting and motivates the need for better evaluation criteria.\nThe reported improvements are good - lifting Qwen3-Coder-Flash and Max on SWE-Bench Verified represents meaningful progress.\nScaling the reward model to 256k context length addresses a practical limitation of prior work and enables scoring of complex, long trajectories.", "weaknesses": "The paper does not adequately address a critical limitation: verifying program correctness can be as difficult as, or even harder than, executing the program. A reward model must essentially predict execution outcomes across potentially many different code paths and edge cases without actually running the code. This requires the model to simulate program semantics, which is fundamentally challenging and may introduce systematic errors. The paper does not discuss: How the reward model handles corner cases or edge conditions that unit tests would catch. The types of errors the reward model systematically misses (false negatives) or incorrectly flags (false positives). Whether execution-free verification can ever be truly reliable for complex software engineering tasks. Why we should expect a learned model to generalize to unseen code patterns and bugs.\nThis represents a significant gap in the theoretical justification for the entire approach. The claim that execution-free feedback is superior needs much stronger grounding.\n\nWhile the paper presents extensive ablations, the intuitions behind many design decisions are unclear:\nWhy is a 2:1 positive/negative ratio optimal? Is this simply an artifact of data availability, or is there a principled reason?\nWhy do mixed policies help? What specific distribution shift does this address?\nThe paper states \"we believe reward model can still achieve relative good performance by training on a large number of data\" (Appendix C.1) when discussing noisy labels, but provides no theoretical or empirical justification for this belief\nWhy are AUC and ECE the right metrics? While intuitively reasonable, there's no formal argument for why these metrics should predict RL performance\n\nThe core contribution appears to be training a larger reward model on more diverse data and identifying AUC/calibration as important metrics. The approach of training a classifier on successful/unsuccessful trajectories and using it for ranking is standard. The paper does not introduce novel algorithms, architectures (beyond using MoE), or training objectives. The identification of additional evaluation metrics, while useful, is incremental rather than constituting a significant methodological advance.\n\nThe reward model is trained on trajectories labeled by execution-based feedback, then claimed to be superior to execution-based feedback. This creates a circular dependency - if the execution-based labels are noisy (as the paper argues), then the reward model learns to approximate noisy signals. The paper acknowledges this (\"some of them might be noisy\") but does not rigorously analyze how noise in training labels affects model quality or how the model could possibly exceed the quality of its training signal.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XDHa4KzLqD", "reviewer": "Reviewer_nzG1", "summary": "The paper presents SWE-RM, an execution-free reward model for software engineering agents that replaces unit-test feedback with model-based scoring. By introducing AUC and calibration metrics beyond test-time scaling, it achieves SOTA results and improves RL performance on SWE-Bench Verified.", "questions": "Please address the weaknesses noted above.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1.\tThe paper addresses an important and emerging challenge in LLM based software engineering that how to design effective reward signals for coding agents when execution-based feedback is unreliable or unavailable. \n2.\tThe authors provide a convincing empirical motivation showing that similar TTS scores can lead to drastically different RL outcomes. \n3.\tSection 4 includes a broad set of ablations on training data scale, data ratio, policy mixture, and source composition. These controlled studies give a practical recipe for training more robust reward models.\n4.\tThe proposed SWE-RM achieves consistent performance gains on SWE-Bench Verified, improving both test-time scaling results and downstream RL outcomes.", "weaknesses": "1.\tThe paper proposes that three metrics TTS, AUC, and ECE jointly define a “versatile” reward model, but offers no theoretical justification or analytical link to reinforcement learning dynamics. The framework remains purely empirical, leaving the causal relationship between these metrics and RL stability unexplained.\n2.\tThe model adopts a large 30B-parameter MoE with 3B active experts, but the rationale behind this choice is missing. There is no ablation comparing MoE against dense or adapter-based architectures, nor any argument explaining why modularization benefits calibration or discrimination.\n3.\tThe paper vaguely describes the reward model as predicting a special token (“YES/NO”) and mapping it to a continuous score. However, it never specifies the precise loss function. This omission makes it impossible to reproduce or understand the optimization process.\n4.\tAlthough the method claims to produce execution-free feedback, the training labels are derived from fail2pass execution tests. This dependency contradicts the notion of being execution-free and blurs the boundary between model-based and execution-based supervision.\n5.\tSection 4.3 shows performance improvements when increasing context length from 32k to 256k tokens, but provides no explanation of why this happens. The analysis ignores computational costs, latency trade-offs, and whether the gain comes from longer reasoning chains or merely from covering more tokens.\n6.\tSWE-RM employs a Mixture-of-Experts (30B total, 3B active) architecture, but the authors do not compare it with alternative designs. Nor do they explain why this structure improves reward calibration. The absence of such analysis weakens the methodological contribution.\n7.\tThe paper simultaneously uses TTS, AUC, and ECE as evaluation metrics but never establishes their quantitative relationships or complementarity. For example, AUC and TTS both measure ranking ability; it is unclear why optimizing all three leads to better RL stability. The metric design does not appear to be theoretically motivated.\n8.\tThe authors acknowledge that some trajectory labels are derived from fail2pass test results that may not reflect true correctness, yet they do not provide any estimate of label noise, data cleaning strategy, or robustness analysis.\n9.\tDespite claiming a 30B-parameter model trained on 100k samples, the paper omits key details such as GPU count, training duration, or energy cost. The reproducibility statement is overly brief and fails to include crucial hyperparameters, limiting verifiability.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)