Jointly Learning to Repair Code and Generate Commit Message (2021)

Abstract: We propose a novel task of jointly repairing program codes and generating commit messages. Code repair and commit message generation are two essential and related tasks for software development. However, existing work usually performs the two tasks independently. We construct a multilingual triple dataset including buggy code, fixed code, and commit messages for this novel task. We first introduce a cascaded method with two models, one is to generate the fixed code first, and the other generates the commit message based on the fixed and original codes. We enhance the cascaded method with different training approaches, including the teacher-student method, the multi-task method, and the back-translation method. To deal with the error propagation problem of the cascaded method, we also propose a joint model that can both repair the program code and generate the commit message in a unified framework. Massive experiments on our constructed buggy-fixed-commit dataset reflect the challenge of this task and that the enhanced cascaded model and the proposed joint model significantly outperform baselines in both quality of code and commit messages.

## introduction deep learning has been demonstrated remarkably adept at numerous natural language processing (nlp) tasks, such as machine translation @xcite , relation extraction @xcite , grammar error correction @xcite , and so on. the success of deep learning in nlp also promotes the development of which in programming languages @xcite . recently, researchers have exploited deep learning to programming-language related tasks, such as code completion @xcite , automated code repair @xcite , commit messages generation @xcite , code search @xcite , and so on. among these tasks, automated code repair and commit message generation are the two most active and closely related tasks. the former is to repair software bugs automatically without the intervention of a human programmer. the latter aims to generate natural language descriptions of code changes, which act as a record of feature additions and bug repairs. because these two tasks can potentially reduce debugging costs in software development and helps programmers to understand the high-level rationale of changes, a lot of great work has been proposed to deal with automated program repair @xcite @xcite @xcite and commit message generation @xcite @xcite , respectively. however, existing work tackles the two tasks independently, ignoring the underlying relationship between these two closely related tasks, e.g., after fixing the bug, commit message can record the process of code repair. therefore it is crucial to explore how to bridge these two tasks and achieve the code repair and commit messages generation simultaneously. in this paper, we formulate a novel task to jointly repair program code and generate commit message, given the buggy code. to facilitate the study of this task, we create a dataset with multiple programming languages. the dataset is collected from commit and buggy-fixed histories of open-source software projects, where each example consists of buggy code, fixed code, and the corresponding commit message. we first introduce the cascaded methods as baseline. the cascaded model employs one model to repair code and the other to generate commit message successively. we enhance this cascaded model with three training approaches inspired by the low-resource machine translation, including the teacher-student method @xcite , the multi-task learning method @xcite , and the back-translation method @xcite . to deal with the error propagation problem of the cascaded method, we propose a joint model which can achieve both code repair and commit message generation in a single model. we train and evaluate our model using the created triple (buggy-fixed-commit) dataset. the results demonstrate the validity of our proposed methods, which achieve a significant improvement over baseline in both qualities of code and commit messages. particularly, the enhanced cascaded method obtains the best performance on code repair task, and the joint method behaves better than the cascaded method on commit message generation task. our main contributions are as follows: ## task definition we aim to generate the repaired code and the commit message based on the given buggy code. for the example in figure 1 , our goal is to replace syntax bug "int" with "float" for the return type of the "multiplication" method, then generate a piece of commit message to describe the change. formally, given a triple (buggy-fixed-commit) dataset @xmath0, f i , c i )} k @xmath1, where the i-th sample consists of a buggy code snippet b i , its fixed version f i , and a commit message c i that is used to describe the changes from b i to f i , our goal is to learn probability distribution p ( c, f | b). in practice, the commit message c is hard to estimate without the full consideration of both b and f . therefore, it is a reasonable way to firstly predict the fixed code f based on b. then learn how to generate an appropriate message c according to b and f . the probability p ( c, f | b) can be decomposed as: thus, given a new buggy code b, we can generate its fixed version f , and the commit message c following the conditional probability p ( f | b) and p ( c| b, f ), respectively. ## approach in this section, we firstly introduce the cascaded models enhanced by the teacher-student method, the multi-task learning method, and the backtranslation method (section 3.1), which generate repaired code and commit message in a two-stage manner. then, we propose a joint model (section 3.2), which is capable of jointly optimizing the generation of repaired code and commit message in an end-to-end manner. the models described in this section are all build on the transformer model @xcite , where we devise the model to take in some representation of input and then yield a distribution over output vocabulary. ## data in this section, we describe the creation of the dataset in detail. we first describe how we collect the data in the wild. then, we introduce the preparation process of the data to make it suitable for our tasks. ## conclusion in this paper, we propose a novel task to jointly repair code and generate commit message. we provide several competitive architectures, including cascaded model and joint model. to train and evaluate our models, we collect a multilingual buggyfixed-commit dataset from github. the empirical study is conducted to demonstrate the effectiveness of our proposed methods. for future work, we plan to incorporate the tree structure of code into the task and employ more indicative metrics @xcite to evaluate the model performance.