How Far Can We Go with Practical Function-Level Program Repair?
Authors: name: Jiahong Xiang; name: Xiaoyang Xu; name: Fanchu Kong; name: Mingyuan Wu; name: Haotian Zhang; name: Yuqun Zhang
Published: 2024-04-19 12:14:09

Abstract: Recently, multiple Automated Program Repair (APR) techniques based on Large
Language Models (LLMs) have been proposed to enhance the repair performance.
While these techniques mainly focus on the single-line or hunk-level repair,
they face significant challenges in real-world application due to the limited
repair task scope and costly statement-level fault localization. However, the
more practical function-level APR, which broadens the scope of APR task to fix
entire buggy functions and requires only cost-efficient function-level fault
localization, remains underexplored. In this paper, we conduct the first
comprehensive study of LLM-based function-level APR including investigating the
effect of the few-shot learning mechanism and the auxiliary repair-relevant
information. Specifically, we adopt six widely-studied LLMs and construct a
benchmark in both the Defects4J 1.2 and 2.0 datasets. Our study demonstrates
that LLMs with zero-shot learning are already powerful function-level APR
techniques, while applying the few-shot learning mechanism leads to disparate
repair performance. Moreover, we find that directly applying the auxiliary
repair-relevant information to LLMs significantly increases function-level
repair performance. Inspired by our findings, we propose an LLM-based
function-level APR technique, namely SRepair, which adopts a dual-LLM framework
to leverage the power of the auxiliary repair-relevant information for
advancing the repair performance. The evaluation results demonstrate that
SRepair can correctly fix 300 single-function bugs in the Defects4J dataset,
largely surpassing all previous APR techniques by at least 85%, without the
need for the costly statement-level fault location information. Furthermore,
SRepair successfully fixes 32 multi-function bugs in the Defects4J dataset,
which is the first time achieved by any APR technique ever to our best
knowledge.