## 5 Evaluation
### Results
#### 5.4.3 Results for RQ3  
Two components are involved in our approach, FM and the DFNN. To evaluate the contribution of each component in our proposed method, we conduct ablation experiments by comparing their performance. Our method still adheres to the previously defined parameter settings, with the FM epoch value set to 100. The primary objective of this experimental setup is to ensure that the selected parameter values or the grid search range remain consistent. As a result, the differences observed in the model's predictions are most likely attributable to architectural differences. Table 6 presents the prediction results for MovePerf and its two components. On the whole, MovePerf demonstrates superior performance compared to FM and the DFNN across the four datasets, achieving a lower average mean value of 7.69%--representing reductions of 4.98% and 2.45% over FM and the DFNN, respectively. Additionally, its average margin value of 2.15 is significantly better, being 6.90 and 0.78 lower than that of FM and the DFNN. These results show that MovePerf significantly improves the prediction accuracy and stability by combining the strengths of FM and the DFNN. For JUnit, FM exhibits a relatively high mean of 7.82% and a large margin of 15.32, indicating that the model's predictions are prone to significant fluctuations, with relatively low accuracy and high instability. In contrast, the DFNN reduces the mean to 6.03%, and the \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Project**} & \multirow{2}{*}{**Mean**} & \multirow{2}{*}{**FM**} & \multicolumn{2}{c}{**DFNN**} & \multicolumn{2}{c}{**MovePerf**} \\  & & & **Margin** & **Mean** & **Margin** & **Mean** & **Margin** \\ \hline JUnit & 7.82 & 15.32 & 6.03 & 2.26 & **5.41** & **0.03** \\ LC-problems & 0.87 & 5.24 & 0.32 & 0.10 & **0.28** & **0.01** \\ Concurrency & 13.24 & 8.76 & 8.99 & 2.07 & **4.92** & **0.02** \\ Kevin & 28.74 & 6.89 & 25.21 & 7.29 & **20.16** & **8.53** \\ \hline Average & 12.67 & 9.05 & 10.14 & 2.93 & 7.69 & 2.15 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Ablation experiment. margin decreases significantly to 2.26, demonstrating better stability and consistency. While prediction MREs still exist, the DFNN exhibits smaller prediction fluctuations and more reliable results. MovePerf outperforms both FM and the DFNN with a mean of 5.41% and an extremely small margin of 0.03, indicating high precision and remarkable stability and consistency. Clearly, MovePerf provides superior performance to both FM and the DFNN, showing that combining FM and the DFNN improves the model's overall performance. For LC-problems, FM shows a mean of 0.87%, indicating relatively small prediction MREs, but the margin is 5.24, suggesting significant prediction fluctuations. This implies that, while the model provides accurate predictions in certain instances, its stability is poor, with the results varying considerably across different data points. Compared to FM, the DFNN performs even better, with the mean dropping to 0.28% and the margin significantly reduced to 0.08. This demonstrates not only a significant improvement in accuracy but also stronger stability and consistency, with the predictions showing almost no fluctuation. MovePerf, which integrates both FM and the DFNN, further enhances the prediction performance, achieving a mean of 0.28% and an almost zero margin of 0.01, indicating extremely high accuracy with virtually no fluctuation. This confirms that, by combining FM and the DFNN, MovePerf significantly improves the prediction accuracy and consistency, overcoming the limitations of individual models and demonstrating exceptional performance for the LC-problems project. For Concurrency, FM performs poorly, with a mean of 13.24% and a margin of 8.76, indicating large prediction MREs and considerable fluctuations in the results. The DFNN improves upon FM, reducing the mean to 8.99% and the margin to 2.07. While the prediction MRE remains, the DFNN exhibits more stability and smaller fluctuations.