## 4 Experimental Analysis
### Presentation of the Result of Training Models
\begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Model** & **Hyperparameter** & **Range Tried** & **Best Value Found** & **Default Value** \\ \hline \multirow{3}{*}{Decision Tree} & max\_depth & [5; 10; 15; 20, None] & 15 & None \\ \cline{2-5}  & min\_samples\_split & [2; 5; 10] & 5 & 2 \\ \cline{2-5}  & criterion & [’gini’, ’entropy’] & ’entropy’ & ’gini’ \\ \hline \multirow{3}{*}{Bagging (BA)} & n\_estimators & [10; 50; 100; 200] & 100 & 10 \\ \cline{2-5}  & max\_samples & [0.5, 0.7, 1.0] & 0.7 & 1 \\ \cline{2-5}  & bootstrap & [True, False] & TRUE & TRUE \\ \hline \multirow{3}{*}{XGBoost} & learning\_rate & [0.01, 0.05, 0.1, 0.2] & 0.1 & 0.3 \\ \cline{2-5}  & n\_estimators & [50, 100, 200, 300] & 200 & 100 \\ \cline{2-5}  & max\_depth & [3; 5, 7, 10] & 7 & 6 \\ \cline{2-5}  & subsample & [0.5, 0.7, 1.0] & 0.8 & 1 \\ \hline \multirow{3}{*}{Random Forest} & n\_estimators & [50, 100, 200, 300] & 200 & 100 \\ \cline{2-5}  & max\_depth & [10; 20, 30, None] & 20 & None \\ \cline{2-5}  & max\_features & [’sqrt’, ’log2’, None] & ’sqrt’ & ’sqrt’ \\ \hline \multirow{3}{*}{Extra Trees} & n\_estimators & [50, 100, 200, 300] & 200 & 100 \\ \cline{2-5}  & max\_depth & [10; 20, 30, None] & 20 & None \\ \cline{1-1} \cline{2-5}  & max\_features & [’sqrt’, ’log2’, None] & ’log2’ & ’auto/sqrt’ \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Hyperparameter tuning summary on open data source dataset. \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **F1-Score** & **ROC Area** \\ \hline Decision Tree & 0.9147 & 0.9194 & 0.6598 & 0.7683 & 0.7941 \\ \hline Bootstrap Aggregating & 0.8781 & 0.9266 & 0.7094 & 0.8036 & 0.7608 \\ \hline XGBoost & 0.9021 & 0.9402 & 0.8132 & 0.8721 & 0.7831 \\ \hline Random Forest & 0.9089 & 0.9318 & 0.7323 & 0.8201 & 0.8005 \\ \hline Extra Trees & 0.9431 & 0.9297 & 0.7197 & 0.8113 & 0.8100 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: The model training performance of open data source after fine-tuning. The accuracy trends among the optimized models were compared (see Figure 1). The increase highlights how each model's accuracy has benefited from hyperparameter tuning and cross-validation. The tuned accuracy reflects the improvements achieved through hyperparameter tuning and cross-validation, with all models showing increased accuracy compared to their initial accuracy. Adjusting the hyperparameters and evaluating model configurations using cross-validation led to improved performance metrics for each model. The role of cross-validation was not as part of the final model, but rather as an evaluation mechanism during training to prevent overfitting and ensure the robustness of parameter selection. This comparison highlights the positive impact of fine-tuning and optimization across different models. Further analysis to affirm the performance of the fine-tuned trained model with the ROC curve presents the complete spectrum of performance at different thresholds. In terms of the Decision Tree (see Figure 2), it indicates that the model performs well, with a high ability to distinguish between positive and negative classes, suggesting that this model reliably identifies true cases of interest. The shape of the curve suggests that the model has a good balance. The Bootstrap Aggregating (Bagging) model ROC curve (see Figure 3) represents excellent performance, suggesting the model is very good at distinguishing between the positive and negative classes. The smoothness of the ROC curve here, without sharp jumps, implies that Bagging provides a balanced approach that reduces the instability often seen in single Decision Trees. The high AUC score and shape of the curve suggest that this Bagging model is very reliable for detecting true positive cases without a significant increase in false positives (see Figure 3). Figure 1: Accuracy trends among the optimized models. The ROC curve for the XGBoost model indicates that the model is highly effective at distinguishing between the positive and negative classes (see Figure 4).