Skywork-SWE: Unveiling Data Scaling Laws for Software Engineering in
  LLMs
Authors: name: Liang Zeng; name: Yongcong Li; name: Yuzhen Xiao; name: Changshi Li; name: Chris Yuhao Liu; name: Rui Yan; name: Tianwen Wei; name: Jujie He; name: Xuchen Song; name: Yang Liu; name: Yahui Zhou
Published: 2025-06-24 03:53:36

Abstract: Software engineering (SWE) has recently emerged as a crucial testbed for
next-generation LLM agents, demanding inherent capabilities in two critical
dimensions: sustained iterative problem-solving (e.g., >50 interaction rounds)
and long-context dependency resolution (e.g., >32k tokens). However, the data
curation process in SWE remains notoriously time-consuming, as it heavily
relies on manual annotation for code file filtering and the setup of dedicated
runtime environments to execute and validate unit tests. Consequently, most
existing datasets are limited to only a few thousand GitHub-sourced instances.
To this end, we propose an incremental, automated data-curation pipeline that
systematically scales both the volume and diversity of SWE datasets. Our
dataset comprises 10,169 real-world Python task instances from 2,531 distinct
GitHub repositories, each accompanied by a task specified in natural language
and a dedicated runtime-environment image for automated unit-test validation.
We have carefully curated over 8,000 successfully runtime-validated training
trajectories from our proposed SWE dataset. When fine-tuning the Skywork-SWE
model on these trajectories, we uncover a striking data scaling phenomenon: the
trained model's performance for software engineering capabilities in LLMs
continues to improve as the data size increases, showing no signs of
saturation. Notably, our Skywork-SWE model achieves 38.0% pass@1 accuracy on
the SWE-bench Verified benchmark without using verifiers or multiple rollouts,
establishing a new state-of-the-art (SOTA) among the Qwen2.5-Coder-32B-based
LLMs built on the OpenHands agent framework. Furthermore, with the
incorporation of test-time scaling techniques, the performance further improves
to 47.0% accuracy, surpassing the previous SOTA results for sub-32B parameter
models. We release the Skywork-SWE-32B model checkpoint to accelerate future
research.