M ulti F ix: Learning to Repair Multiple Errors by Optimal Alignment Learning (2021)

Abstract: We consider the problem of learning to repair erroneous C programs by learning optimal alignments with correct programs. Since the previous approaches fix a single error in a line, it is inevitable to iterate the fixing process until no errors remain. In this work, we propose a novel sequence-to-sequence learning framework for fixing multiple program errors at a time. We introduce the edit-distance-based data labeling approach for program error correction. Instead of labeling a program repair example by pairing an erroneous program with a line fix, we label the example by paring an erroneous program with an optimal alignment to the corresponding correct program produced by the edit-distance computation. We evaluate our proposed approach on a publicly available dataset (DeepFix dataset) that consists of erroneous C programs submitted by novice programming students. On a set of 6,975 erroneous C programs from the DeepFix dataset, our approach achieves the state-of-the-art result in terms of full repair rate on the DeepFix dataset (without extra data such as compiler error message or additional source codes for pre-training).

## introduction recurrent neural networks (rnns) @xcite are one of the fundamental concepts in deep learning to process sequential data such as text, speech and time-series data. in particular, rnns already have become a standard and general-purpose tool for various natural language tasks and successfully replaced conventional methods. the encoder-decoder sequence-to-sequence architecture @xcite @xcite is becoming the de facto standard for translating source sequences into target sequences. while the most popular application of the sequence-to-sequence learning is machine translation, there are many approaches for adopting this method to other fields such as dialogue systems, text summarization, program synthesis, grammatical error detection and correction. automatic program repair is one of the applications where the sequence-to-sequence learning framework has been successfully employed. by automatically localizing the compilation errors and suggesting the possible fixes to the programmers, we can dramatically improve the productivity of programmers. while there have been many approaches to the problem of repairing programs using rule-based algorithms, it is very difficult to implement a useful rule-based repair program as there are too many cases to consider. therefore, applying deep neural networks and learning-based framework to automatic program repair is an inevitable consequence. in this work, we consider the problem of learning to repair erroneous c programs based on optimal alignment learning. figure 1 describes our approach. given a potentially erroneous program p, our goal is to train a sequence-to-sequence model that takes p as input sequence and produces a sequence of edits that repairs compilation errors from p. there are two major challenges in learning to repair programs. first, it is very difficult to localize multiple errors in a program and correct the errors simultaneously. previous approaches @xcite @xcite exploited sequence-to-sequence models to learn the mapping between erroneous programs and line fixes. they train a model that repairs only one error at a time. if there exist multiple errors in several lines of code, then it is inevitable to iterate the same process. in light of these observations, we propose the following ideas to improve the previous program repair models. ## background our approach is based on the sequence-to-sequence learning framework which is originally proposed for the task of machine translation. on top of the idea of sequence-to-sequence learning, we incorporate several ideas for optimizing the training performance on the domain of program repair including the optimal alignment output encoding and synchronized position embedding. here we briefly provide an overview of each idea. ## our approach we employ the following techniques to improve the previous learning-based program repair. encoding optimal alignment as target. the previous approaches to the program repair based on the sequence-to-sequence learning framework aim to learn pairs of erroneous programs and corresponding line fixes. however, it is very difficult to learn to produce multiple lines of fixes within this framework as it is almost impossible to learn the alignment between the input tokens and the output tokens. in order to resolve this problem, we encode the target sequence as an optimal alignment of the input program to the target program. ## experimental setup in this section, we summarize the setup of our experiments including the datasets used for evaluation of our model and several training details. ## conclusions our experimental results concerned with the use of beam search decoding and samplefix @xcite suggest that we can improve the performance of learning-based program repair approaches by adopting diverse sampling techniques such as cvae as in samplefix. moreover, the performance difference between two versions of multifix model trained with the original deepfix dataset and the drperturb dataset suggests that both the quality and diversity of the training dataset are crucial in learning to repair various types of program errors.