# Achieving Excellence in Cyber Fraud Detection: A Hybrid ML+DL Ensemble Approach for Credit Cards
## 5 Results and Discussion  
Initially, we assess the performance of ML algorithms individually and in the absence of employing ensemble techniques. In datasets with class imbalances, such as fraud detection, the F1 score provides a balanced evaluation by harmonising precision (minimising false positives) and recall (minimising false negatives). Unlike accuracy, which can be misleading in such cases, the F1 score focuses on the minority class while addressing both critical error types. This makes it a nuanced metric for assessing model effectiveness, especially when both detecting fraud and avoiding legitimate transaction misclassification are crucial. The outcomes derived from this evaluation are comprehensively depicted in Table 3, illustrating a comparative analysis of the algorithms. Figure 6 shows the performance of ML algorithms without ensemble techniques. The DT model achieves high accuracy (99.93%) and precision (89.89%) but has a lower recall (81.63%), resulting in an F1 score of 85.56%. RF delivers exceptional accuracy (99.96%) and precision (97.40%) but falls short on recall (76.53%), with an F1 score of 85.71%. Similarly, SVM shows strong accuracy (99.94%) and precision (97.02%) but lower recall (66.33%), yielding an F1 score of 78.79%. XGBoost balances performance with 99.95% accuracy, 95.00% precision, and an F1 score of 85.39%. CatBoost excels with 99.96% accuracy, 97.46% precision, and the highest F1 score of 87.01%, indicating an excellent balance between precision and recall. In contrast, LR achieves 99.92% accuracy but lower precision (88.06%) and recall (60.20%), resulting in a lower F1 score of 71.52%. Overall, CatBoost emerges as the top performer, followed by RF and XGBoost, while LR and SVM lag in capturing positive instances effectively. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**ML** & **Accuracy** & **Precision** & **Recall** & **F1 Score** & **AUC** \\ \hline
**DT** & 99.93\% & 89.89\% & 81.63\% & 85.56\% & 90.80\% \\ \hline
**RF** & 99.96\% & 97.40\% & 76.53\% & 85.71\% & 97.25\% \\ \hline
**SVM** & 99.94\% & 97.02\% & 66.33\% & 78.79\% & 95.13\% \\ \hline
**XGBoost** & 99.95\% & 95.00\% & 77.55\% & 85.39\% & 97.83\% \\ \hline
**CatBoost** & 99.96\% & 97.44\% & 77.55\% & **86.36\%** & 98.37\% \\ \hline
**LR** & 99.92\% & 88.06\% & 60.20\% & 71.52\% & 97.01\% \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Algorithms’ performance. Figure 6: Performance of ML algorithms. Subsequently, we evaluate the performance of DL algorithms independently, without incorporating ensemble techniques. This evaluation focuses on understanding the effectiveness of standalone DL methods in detecting credit card cyber fraud, particularly in handling sequential data and uncovering intricate patterns. The results of this assessment are summarised in Table 4 for CNN and Table 5 for BiLSTM. Figures 7 and 8 visually represent the confusion matrix for CNN and BiLSTM. The comparison between CNN and BiLSTM models highlights key differences in performance, with the F1 score as the primary focus. For the CNN model, an F1 score of 82.80% was achieved with an epoch size of 20 and a batch size of 64, improving slightly to 83.98% with larger epoch and batch sizes (50 and 128). The improvement was driven by increased precision (91.57%), though recall slightly decreased (77.55%), indicating strong identification of fraud but limited coverage of all cases. In contrast, the BiLSTM model excelled, particularly at larger configurations. Its F1 score increased from 82.16% (epoch size 20, batch size 64) to 85.86% (epoch size 50, batch size 128), with higher recall (83.67%) and solid precision (88.17%). This demonstrates BiLSTM's superior balance and effectiveness in detecting fraudulent transactions compared to CNN, particularly for imbalanced datasets.