IterPref: Focal Preference Learning for Code Generation via Iterative
  Debugging
Authors: name: Jie Wu; name: Haoling Li; name: Xin Zhang; name: Jianwen Luo; name: Yangyu Huang; name: Ruihang Chu; name: Yujiu Yang; name: Scarlett Li
Published: 2025-03-04 16:56:34

Abstract: Preference learning enhances Code LLMs beyond supervised fine-tuning by
leveraging relative quality comparisons. Existing methods construct preference
pairs from
  candidates based on test case success, treating the higher pass rate sample
as positive and the lower as negative. However, this approach does not pinpoint
specific errors in the code, which prevents the model from learning more
informative error correction patterns, as aligning failing code as a whole
lacks the granularity needed to capture meaningful error-resolution
relationships. To address these issues, we propose IterPref, a new preference
alignment framework that mimics human iterative debugging to refine Code LLMs.
IterPref explicitly locates error regions and aligns the corresponding tokens
via a tailored DPO algorithm. To generate informative pairs, we introduce the
CodeFlow dataset, where samples are iteratively refined until passing tests,
with modifications capturing error corrections. Extensive experiments show that
a diverse suite of Code LLMs equipped with IterPref achieves significant
performance gains in code generation and improves on challenging tasks like
BigCodeBench. In-depth analysis reveals that IterPref yields fewer errors. Our
code and data will be made publicaly available.