# An Automatic Software Defect Repair Method Based on Multi-Objective Genetic Programming
## 4 Experimental Evaluation
### Experimental Results and Analysis
As can be seen from the figure, in addition to the average and median mentioned earlier, the upper quartile and lower quartile of the time consumed by each repair method can also be observed. MGPRepair has both the upper and lower quartiles lower than the other three methods. The narrow yellow rectangle represents that the time required for MGPRepair to repair defects has relatively small fluctuations and is more stable. In summary, MGPRepair outperforms both the classic and the most advanced methods in the same category in terms of the time required for defect repair. The main reason for this is that the method in this chapter optimizes the initial population and makes changes in selecting offspring patches, which accelerates and optimizes the genetic search process. The \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Repair Method**} & \multicolumn{4}{c}{**Consumption Time**} \\ \cline{2-5}  & **Min Time** & **Median** & **Max Time** & **Average Time** \\ \hline MGPRepair & 0.71 & 3.49 & 58.35 & 9.91 \\ jGenProg & 0.63 & 8.13 & 78.65 & 15.82 \\ RSRepair & 0.88 & 7.92 & 79.34 & 16.59 \\ ARJA & 0.73 & 4.91 & 63.73 & 11.62 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Time Consumption Comparison of MGPRepair and Other Genetic Programming-Based Repair Methods. addition of test case filtering techniques significantly speeds up the offspring validation process, saving a considerable amount of time. 3. Research Question 3: The substantial improvements brought by the methodology and techniques used in MGPRepair for automated defect repair. The MGPRepair method utilizes a lightweight context filtering approach and further classifies and filters the repair materials to generate a higher-quality population. The NSGA-II algorithm is employed in the genetic search process to find patches that can pass more test cases with minimal changes. Test case filtering is also introduced in the process of selecting offspring patches, which speeds up the selection process. To investigate whether these methods play a role in MGPRepair and to what extent, this experiment designs and implements three variants of MGPRepair: MGPRepair-A, MGPRepair-B, and MGPRepair-C. These three variants share the same basic framework as MGPRepair, but differ in that they only utilize some of the techniques in MGPRepair. As detailed in Table 7, the table lists the three most important techniques of MGPRepair and introduces the techniques used by each variant. It can be concluded from the table that MGPRepair-A does not utilize test case filtering, MGPRepair-B does not use the technique to optimize the initial population, and MGPRepair-C does not employ the multi-objective genetic algorithm. Next, an ablation study will be conducted to verify the improvements brought by each technique. This experiment was validated on the Defects4] defect dataset using the \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Repair Method** & \begin{tabular}{c} **Multi-Target** \\ **Genetics** \\ \end{tabular} & \
\begin{tabular}{c} **Repair Material** \\ **Boosting** \\ \end{tabular} &
\begin{tabular}{c} **Overfitting Detection** \\ \end{tabular} \\ \hline MGPRepair & \(\surd\) & \(\surd\) & \(\surd\) \\ MGPRepair-A & \(\surd\) & \(\surd\) & \(\times\) \\ MGPRepair-B & \(\surd\) & \(\times\) & \(\surd\) \\ MGPRepair-C & \(\times\) & \(\surd\) & \(\surd\) \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Techniques Used in MGPRepair and Its Three Variants. Figure 7: Box Plot of Time Consumption Comparison between MGPRepair and Other Genetic Programming-Based Repair Methods. three variants. To reduce labor costs, this experiment only counts the credible patches that pass the test case set, without comparing the credible patches with manual patches. As shown in Figure 8, MGPRepair-A, MGPRepair-B, and MGPRepair-C successfully generated credible patches for 48, 39, and 45 defects, respectively, representing a reduction of 3, 12, and 6 patches compared to MGPRepair.