## 5 Empirical Study
### Performance of the ML Effort Estimation Models  
The performance of the ML effort estimation models is presented in Tables 10 and 11. Table 10 presents the training RMSE, test RMSE, Training Accuracy, Testing Accuracy, and MAE values, whereas Table 11 presents MBRE, MIBRE, LSD, SA, MdAt, MDBRE, and MDIRE. For instance, Linear Regression, Extra Trees, and Random Forest performed well in terms of low RMSE, high Testing Accuracy, and strong SA scores.  
The authors in [88] reported that Standard Accuracy (SA) can be used as the main metric to evaluate software effort estimation models. SA provides a more reliable evaluation of estimation accuracy than conventional error metrics, such as Mean Relative Error (MRE) and Mean Magnitude of Relative Error (MMRE), which have been criticized for being biased toward underestimates.  
In addition, Table 10 includes Testing Accuracy, which is a machine learning metric used to evaluate the capability of the models for generalization by quantifying the percentage of accurate predictions.  
The following key comparisons from Tables 10 and 11 highlight the strong performance of the Linear Regression, extra stress, and Voting Regressor ML techniques while using our dataset to predict the web analytics effort based on COSMIC.  
1. Comparison based on the prediction accuracy and generalization: \(\bullet\) Extra Trees had the best test RMSE (0.46) and highest accuracy (98%), making it the most accurate predictor. \(\bullet\) Voting Regressor also achieved a strong performance (RMSE = 0.46, accuracy = 98%), demonstrating robust generalization. \(\bullet\) Linear Regression had the lowest MAE (0.35), indicating that it provided more stable predictions, on average.
2. Comparison based on the error analysis (MBRE, MIBRE, MDBRE) \(\bullet\) Linear Regression consistently had the lowest error values, making it a highly reliable model. \(\bullet\) The Voting Regressor has a moderate MBRE (1.74) and MIBRE (1.13), showing a good balance between precision and generalization. \(\bullet\) Extra Trees had slightly higher MBRE (2.10) and MDBRE (1.70) but compensated for lower RMSE and high Testing Accuracy.
3. Comparison based on the Standard Accuracy (SA): \(\bullet\) Linear Regression (0.86) had the highest SA, indicating that it provided correct predictions more frequently. \(\bullet\) Voting Regressor (0.86) matched Linear Regression in SA, reinforcing its reliability. \(\bullet\) Extra Trees (0.83) is slightly behind but remains effective.  
Figures 6-8 present examples of the performance of the Extra Trees, Linear Regression, and Random Forest models because these models performed well in terms of SA and Testing Accuracy. These figures illustrate the actual effort (in green) and predicted effort values (in blue) and visualize the variations between them.  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Model** & **MBRE** & **MIBRE** & **LSD** & **SA** & **MdAt** & **MDBRE** & **MDIRE** \\ \hline Boosting & 2.15 & 1.92 & 0.02 & 0.83 & 0.37 & 1.92 & 1.89 \\ \hline Voting & 1.74 & 1.13 & 0.02 & 0.86 & 0.25 & 1.13 & 1.12 \\ \hline Stacking & 1.70 & 0.91 & 0.02 & 0.86 & 0.16 & 0.91 & 0.91 \\ \hline \hline \end{tabular}
\end{table}
Table 11: _Cont._Figure 8: Predicted vs. Actual values—Voting Regressor for 20% testing data.  
Figure 6: Predicted vs. Actual Values—Extra Trees Regressor for 20% testing data.  
Figure 7: Predicted vs. Actual values—Linear Regression for 20% testing data.