# An Automatic Software Defect Repair Method Based on Multi-Objective Genetic Programming
## 4 Experimental Evaluation
### Experimental Results and Analysis
It is necessary to compare the generated credible patch with the manually created patch to determine if a correct patch has been generated. Only when the generated patch and the manually created patch are fully interchangeable in syntax and semantics is the patch considered correct. Next, we further conduct a manual comparative analysis of the patches generated by MGPRepair and similar methods jGenprog and ARJA to determine the number of truly effective patches that our method can produce for specific project defects. Table 5 is a statistical table showing the specific defect IDs repaired in the Chart, Closure, Lang, Math, Time, and Mockito projects by MGPRepair, jGenprog, and ARJA. The repair patches corresponding to each defect in the table have been manually verified. According to the data in the table, MGPRepair successfully generated correct patches for 35 defects, jGenprog successfully generated correct patches for 17 defects, and ARJA successfully generated correct patches for 25 defects. Among the six projects, MGPRepair repaired the highest number of defects in any given project. The percentage of confirmed repaired defects by MGPRepair is 8.86%, approximately 2.06 times higher than jGenprog and 1.4 times higher than ARJA. It also achieved successful repairs for 2 defects in the Time and Closure projects, which the other two methods failed to do. Moreover, the defects repaired by MGPRepair almost cover those repaired by the other two methods. Unfortunately, none of the three methods were able to successfully repair defects in the Mockito project. This may be due to the special nature of the code in this project. The search space did not contain the repair materials corresponding to the defects, making it difficult to generate correct patches for the defects. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline \multirow{2}{*}{**Methodology Tools**} & **Chart** & **Closure** & **Lang** & **Math** & **Time** & **Mockito** & **Total** & \multirow{2}{*}{_p (\%)_} \\ \cline{2-3} \cline{5-8}  & **26** & **133** & **65** & **106** & **38** & **27** & **395** & \\ \hline jGenProg & 7 & 0 & 0 & 18 & 0 & 2 & 27 & 6.84 \\ Nopol & 6 & 0 & 7 & 21 & 0 & 1 & 35 & 8.86 \\ Cappen & 4 & 0 & 5 & 16 & 0 & 0 & 25 & 6.33 \\ SequenceR & 3 & 5 & 2 & 10 & 1 & 0 & 21 & 5.32 \\ ARJA & 8 & 2 & 8 & 23 & 1 & 2 & 44 & 11.4 \\ RSRepair & 7 & 1 & 6 & 17 & 0 & 1 & 32 & 8.1 \\ \hline MGPRepair & 8 & 3 & 10 & 27 & 1 & 2 & 51 & 12.91 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of Repair Outcomes between MGPRepair and Other Six Automated Defect Repair Methods. \(p\) (%) refers to the defect repair rate on Defects4]. Figure 6 presents a Venn diagram comparing the repair capabilities of MGPRepair, jGenprog, and ARJA. From the diagram, it is clear that MGPRepair is able to successfully repair most of the defects that jGenprog and ARJA can repair, while jGenprog and ARJA can only repair a small portion of the defects repaired by the other two methods. In the Closure and Mockotic projects, only MGPRepair was able to successfully repair 2 defects. In addition, there are 9 defects repaired by MGPRepair that neither of the other two methods was able to repair, indicating that the repair capabilities of these two methods are significantly weaker than MGPRepair. In summary, it can be concluded that MGPRepair has achieved significant improvements in automated defect repair based on search techniques, especially in the genetic programming approach by optimizing the initial population and selection of offspring patches. MGPRepair, to some extent, alleviates the problem of slow search speed caused by poor initial population quality. Changing single-objective search to multi-objective search also accelerates patch generation and makes it easier to find optimal solutions, thus gaining more advantages in repairing defects on Defects4J.