R2E-Gym: Procedural Environment Generation and Hybrid Verifiers for Scaling Open-Weights SWE Agents

Naman Jain^*,1, Jaskirat Singh^*,2, Manish Shetty¹, Liang Zheng², Koushik Sen¹, Ion Stoica¹

¹UC Berkeley, ²ANU
_{^*Equal contribution, ^{^}Equal supervision}

💻 Code • 📃 Paper • 🤗 Data & Models • 🌐 Project Page

We present R2E-Gym, the largest procedurally curated environment for training real-world SWE-Agents. We show that R2E-Gym enables more scalable train and test-time scaling, achieving 51% on the SWE-Bench Verified benchmark, reflecting a new state-of-the-art for open-weight SWE-Agents and for first time being competitive with proprietary models such as o1 and sonnet-3.5-v2 with tools.

R2E-Gym is powered by two main contributions: (a) SWE-GEN: a synthetic data curation recipe for curating executable training environments w/o relying on human tests and issues. (b) Hybrid Inference Time Scaling: showing that while both execution-based and execution-free verifiers elicit inference-time gains; significantly better performance can be achieved by leveraging the strengths of both. (c) Overall, the final approach reflects SOTA performance for open-weight SWE-Agents, while also being competitive with some proprietary model baselines.

Usage and Training

Please refer our Github Repo for detailed notes on Gym Environment Usage, Training, Inference and Executable SWE Environment Generation.

📚 Citation

@misc{jain2025r2e-gym,
      title={R2E-Gym: Procedural Environment Generation and Hybrid Verifiers for Scaling Open-Weights SWE Agents},
      author={Jain Naman and Singh Jaskirat and Shetty Manish and Zheng Liang and Sen Koushik and Stoica Ion},
      year={2025},
      eprint={xxx.xxxx},
      archivePrefix={arXiv},
      primaryClass={cs.SE},
      url={https://arxiv.org/abs/xxx.xxxx}, 
}