# Hyperparameter Optimization of Ensemble Models for Spam Email Detection
## 4 Results and Discussion
### Results and Discussion
#### 4.2.1 Performance Evaluation Metrics
However, it was outperformed by the other models in terms of accuracy, precision, specificity, and F1 score. Precision measures the amount of accurately predicted spam emails against the total number of positive predictions. In other words, it measures the performance of the model with respect to the false positive. It is determined as thus:  
\[Precision=\frac{TP}{(TP+FP)} \tag{7}\]  
Figure 6 revealed that the tuned XGBoost model has the highest precision of 0.9748. This implies that for every email that was predicted spam, 97% of those emails are spam. The random forest base model, XGBoost-baseline model, and the random forest tuned model achieved a precision of 0.9723, 0.9623, and 0.9726, respectively. F1-Score  
The F1-score is a weighted mean of precision and sensitivity. It is determined as presented in Equation (8). The F1 score helps to measure the impact of false positives  
Figure 6: Performance evaluation metric comparison of the baseline and tuned random forest and XGBoost models. and false negatives in the models. In other words, it balances the impact of low or high precision and recall (sensitivity) on the model. \[F1-Score=2*\frac{Precision*Sensitivity}{Precision+Sensitivity} \tag{8}\]  
As seen in Figure 6, the tuned XGBoost model achieved the highest F1 score of 0.9816. The random forest baseline model, XGBoost baseline model, and the random forest tuned model achieved an F1-score of 0.9759, 0.9758, and 0.9785, respectively. Specificity  
Specificity measures the ability of the model to classify hams correctly. It is the percentage of negatives that are correctly detected as a negative class. It is determined as presented in Equation (9). The tuned random forest and XGBoost models were better at detecting legitimate emails, with the XGBoost tuned model achieving the highest specificity of 0.9730. \[Specificity=\frac{TN}{(TN+FP)} \tag{9}\]  
Receiver Operating Curve (ROC)  
The ROC is a plot of the number of accurate predictions in the prediction of the positive class against the number of negative samples wrongly predicted as being in a positive class. It ranks the ability of the model to distinguish between the classes of emails. It measures the quality of predictions by the model. The Area Under the ROC Curve (AUC) measures the space under the ROC curve. The AUC ranged from 0 to 1. As the value of the AUC tends to 1, the better its distinguishing ability. The ROC curves for the random forest baseline model, XGBoost baseline model, random forest tuned model, and XGBoost tuned model are shown in Figure 7. The models' Areas Under the Curve (AUC) are 0.9894, 0.9892, 0.9966, and 0.9978, as shown in Figure 7. The AUC for the random forest baseline model, with the highest AUC value of 0.9978, indicates that the tuned XGBoost model distinguished between the spam and ham emails better than the other models. The hyperparameter tuning with 10-fold cross-validation significantly improved the performance of the ensemble models. Hyperparameter tuning does not improve model performance in all cases [37]. The tuned RF and XGBoost model performed better than  
Figure 7: Area Under the Receiver Operating Curves of random forest and XGBoost models. The yellow plot line is the TPR against the FPR plot of the random forest-based model while the red, blue, and green are for the tuned random forest model, XGBoost baseline model, and XGBoost tuned model, respectively. their un-tuned baseline models for all metrics except for the XGBoost baseline model with the highest sensitivity. The tuned XGBoost model outperformed the tuned random forest model for all measures evaluated as shown in Figures 6 and 7. It is also important to note that the XGBoost model is a faster algorithm (the XGBoost algorithm supports parallel computation) when compared with random forest. This is also established in the study by [38]. The random forest baseline model took 3762.38 s to train while it took 55.45 s to train the baseline XGBoost model.