## 4 Experimental Analysis
### Presentation of the Result of Training Models
XGBoost's high recall and overall balance across metrics make it the most effective model for this high-stakes application, where missing positive cases is significantly more detrimental than classifying neutral terms as risky. After fine-tuning, the ROC curves for each model indicate a substantial improvement in their ability to distinguish between positive and negative classes. The decision tree model now demonstrates strong discriminatory power with a significantly improved AUC of 0.93. This indicates a more reliable classification, although not as high as the ensemble models. It shows the model's ability to reduce misclassification after fine-tuning (see Figure 8). \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **F1-Score** & **ROC Area** \\ \hline Decision Tree & 0.8676 & 0.8723 & 0.6127 & 0.7198 & 0.7470 \\ \hline Bootstrap Aggregating & 0.8131 & 0.8795 & 0.6623 & 0.7556 & 0.7137 \\ \hline XGBoost & 0.8455 & 0.8931 & 0.7661 & 0.8247 & 0.7360 \\ \hline Random Forest & 0.8818 & 0.8849 & 0.6847 & 0.7720 & 0.7534 \\ \hline Extra Trees & 0.8716 & 0.8877 & 0.6923 & 0.7779 & 0.7629 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: The model training performance of GTD. \begin{table}
\begin{tabular}{l c c c c c} \hline \hline
**Model** & **Accuracy** & **Precision** & **Recall** & **F1-Score** & **ROC Area** \\ \hline Decision Tree & 0.9028 & 0.9182 & 0.6754 & 0.7783 & 0.7871 \\ \hline Bootstrap Aggregating & 0.8811 & 0.9201 & 0.7236 & 0.8101 & 0.7538 \\ \hline XGBoost & 0.9001 & 0.9389 & 0.8252 & 0.8784 & 0.7761 \\ \hline Random Forest & 0.9042 & 0.9332 & 0.7432 & 0.8274 & 0.7935 \\ \hline Extra Trees & 0.9422 & 0.9354 & 0.7411 & 0.8270 & 0.8008 \\ \hline \hline \end{tabular}
\end{table} 
Table 10: The model training performance of GTD after fine-tuning. Figure 8: The decision tree model ROC curve on GTD dataset. \begin{table}
\begin{tabular}{l l l l} \hline \hline
**Model** & **Hyperparameter** & **Range Tried** & **Best Value Found** & **Default Value** \\ \hline Decision Tree & max\_depth & [5, 10, 15, 20, None] & 15 & None \\ \hline  & min\_samples\_split & [2, 5, 10] & 5 & 2 \\ \hline  & Criterion & [’gini’, 'entropy’] & ’gini’ & ’gini’ \\ \hline Bagging (BA) & n\_estimators & [50, 100, 200] & 100 & 10 \\ \hline  & max\_samples & [0.5, 0.7, 1.0] & 0.8 & 1 \\ \hline  & Bootstrap & [True, False] & TRUE & TRUE \\ \hline XGBoost & learning\_rate & [0.01, 0.05, 0.1] & 0.1 & 0.3 \\ \hline  & n\_estimators & [100, 200, 300] & 200 & 100 \\ \hline  & max\_depth & [3, 5, 7, 10] & 7 & 6 \\ \hline  & Subsample & [0.5, 0.7, 1.0] & 0.8 & 1 \\ \hline Random Forest & n\_estimators & [50, 100, 200] & 200 & 100 \\ \hline  & max\_depth & [10, 20, 30] & 20 & None \\ \hline  & max\_features & [’sqrt’, ‘log2’, None] & ’sqrt’ & ‘sqrt’ \\ \hline Extra Trees & n\_estimators & [100, 200, 300] & 200 & 100 \\ \hline  & max\_depth & [10, 20, 30] & 20 & None \\ \hline  & max\_features & [’sqrt’, ‘log2’, None] & ’sqrt’ & ‘auto/sqrt’ \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Hyperparameter tuning summary for Global Terrorism Database (GTD). The Bagging approach shows a high AUC of 0.96, reflecting an effective ensemble strategy to stabilize and improve the model's performance (see Figure 9). This enhancement indicates that the model can now distinguish positive and negative instances with greater accuracy, making it suitable for applications requiring stable prediction. XGBoost achieves an AUC of 0.95, indicating robust performance with high discriminatory power. This improvement suggests that XGBoost is effectively leveraging boosted trees to enhance its ability to identify true positive instances, making it a reliable choice for complex predictive tasks (see Figure 10). Figure 10: The XGBoost Model ROC curve on GTD dataset. Figure 9: The Bootstrap Aggregating Model ROC curve on GTD dataset. With an AUC of 0.98, the random forest model shows excellent performance, achieving near-optimal classification accuracy.