SoRFT: Issue Resolving with Subtask-oriented Reinforced Fine-Tuning
Authors: name: Zexiong Ma; name: Chao Peng; name: Pengfei Gao; name: Xiangxin Meng; name: Yanzhen Zou; name: Bing Xie
Published: 2025-02-27 14:19:45

Abstract: Mainstream issue-resolving frameworks predominantly rely on commercial
models, leading to high costs and privacy concerns. Existing training
approaches for issue resolving struggle with poor generalization and fail to
fully leverage open-source development resources. We propose Subtask-oriented
Reinforced Fine-Tuning (SoRFT), a novel training approach to enhance the issue
resolving capability of LLMs. We decomposes issue resolving into structured
subtasks: file localization, function localization, line localization, and code
edit generation. SoRFT consists of two training stages: (1) rejection-sampled
supervised fine-tuning, Chain of Thought (CoT) data is filtered using
ground-truth before fine-tuning the LLM, and (2) rule-based reinforcement
learning, which leverages PPO with ground-truth based rewards. We evaluate the
SoRFT-trained model on SWE-Bench Verified and SWE-Bench Lite, achieving
state-of-the-art (SOTA) performance among open-source models (e.g., resolve
21.4% issues on SWE-Bench Verified with SoRFT-Qwen-7B). The experimental
results demonstrate that SoRFT significantly enhances issue-resolving
performance, improves model generalization, and provides a cost-efficient
alternative to commercial models.