# An Automatic Software Defect Repair Method Based on Multi-Objective Genetic Programming
## 4 Experimental Evaluation
### Experimental Results and Analysis
\begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Project**} & \multirow{2}{*}{
\begin{tabular}{c} **Number of** \\ **Defects** \\ \end{tabular} } & \multicolumn{3}{c}{**Fixable Defect ID**} \\ \cline{3-5}  & & **MGPRepair** & **jGenprog** & **ARJA** \\ \hline Chart & 26 & C1, C3, C5, C13, C15, C19, C25 & C3, C7, C13, C15, C25 & C1, C5, C13, C15, C19 \\ \hline Lang & 65 & L7, L43, L45, L46, L51, L61, L63 & - & L7, L43, L51, L55, L59, L61 \\ \hline \multirow{4}{*}{Math} & \multirow{4}{*}{106} & M2, M8, M20, M28, M32, & \multirow{4}{*}{} & M2, M8, M28, M40, M49, & M2, M20, M28, M32, \\  & & M40, M49, M50, M53, M60, & & M50, M70, M73, M82, M84, & M39, M40, M49, M50, \\  & & M64, M73, M74, M78, M80, & & M50, M70, M73, M82, M84, & M53, M60, M73, M74, \\  & & M81, M82, M84, M95 & & & M82, M84, M85 \\ \hline Time & 27 & T4 & - & - \\ \hline Closure & 133 & Clo5 & - & - \\ \hline Mockito & 27 & - & - & - \\ \hline Total & 395 & 35 & 17 & 25 \\ \hline Success Rate & & 8.86\% & 4.3\% & 6.33\% \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison of Repair Outcomes between MGPRepair, jGenprog, and ARJA. Figure 6: Venn Diagram Showing the Repaired Patches by Three Different Repair Methods. (2) Research Question 2: Compare the time performance of MGPRepair in defect repair with other outstanding automatic defect repair methods. Time is an important metric for evaluating the performance of an automatic defect repair method, and the time required to generate a correct patch for a defect should be within a reasonable range acceptable to people. Excessively long or uncertain repair times clearly do not meet people's needs. Therefore, this research question analyzes the time required for four repair methods: MGPRepair, ARJA, RSRepair, and jGenprog, to repair defects. The reason for selecting these methods is that jGenprog is the most classic genetic programming-based repair method, ARJA is a relatively advanced method, and RSRepair uses the genetic programming framework but omits the process of selecting offspring using a fitness function. The reason for not choosing other types of repair methods for comparison with MGPRepair is that different types of repair methods can vary greatly in the time required to repair defects. For example, machine learning-based repair methods require a significant amount of time to train the data model in the early stages, while the defect repair process itself may take a short time. Therefore, comparing with other types of methods is not meaningful. This paper conducts a statistical analysis of the time required for each repair method to successfully repair defects. Each reparable defect is repaired 10 times, and the average value is taken as the comparison standard. Table 6 shows the repair time for the four methods, with the left column representing the four repair methods and the right column showing the corresponding repair time. The statistics include the minimum time, median time, maximum time, and average time required for each method to repair defects. Among them, the median and average time best reflect the general performance of the repair method. The median time for MGPRepair to repair defects is 3.49 min, and the average time is 9.91 min, which is the shortest among the four methods. Additionally, the minimum time required for MGPRepair to generate a patch is 0.71 min, and the maximum time is 58.35 min. It is evident from the statistical data of the experimental results that the time required for MGPRepair to repair defects is within an acceptable range for people. To provide a more detailed view of the time distribution for each repair method, the data has been summarized and presented in the form of a box plot. As shown in Figure 7, the vertical axis represents the time required for defect repair, and the horizontal axis represents MGPRepair and the other three methods. The small white square represents the average value, and the line within the rectangle represents the median.