## 4 Results  
In this investigation, the proficiency of the ANN model in classifying firewall packets based on their attributes is scrutinized. A range of widely accepted performance metrics, encompassing the confusion matrix, accuracy, precision, recall, and F1-Score, are employed to comprehensively evaluate the model's performance. As presented in Table 4, the initial experiments involved using ANNs without SMOTE to classify firewall packets. The results revealed that for the \"allow\", \"deny\", and \"drop\" classes, the model achieved perfect precision, recall, and F1-Score values (1.00), indicating flawless performance. However, for the \"reset-both\" class, all these metrics were at 0.00, indicating that the model could not accurately predict any instance of this class. Despite the issue with the \"reset-both\" class, the model's overall accuracy was still extremely high (1.00), as measured across a total of 13.107 samples. The confusion matrix delineated in Figure 5 illuminates the proficiency of the ANNs in categorizing network actions into 'allow', 'deny', and 'drop'. The ANN exhibits high accuracy with 7521 correct predictions for 'allow' and a complete accuracy for 'drop' actions at 2562 predictions. While the 'deny' class has a notable accuracy with 2939 correct classifications, there is a small fraction of misclassifications, with 55 instances erroneously predicted as 'drop'. The model notably underperforms in the'reset-both' category, misclassifying all instances into other categories, indicating a need for model refinement or reconsideration of the feature set for this particular class to improve its predictive capability. The results highlight the strengths and areas for improvement in the ANN's capacity to discern between classes, particularly when handling imbalanced datasets or underrepresented classes. Figure 6 presents the graphs depicting the model's training and validation accuracy and its training and validation loss. Figure 6 presents the accuracy and loss graphs for the training and validation phases of the ANN model. The accuracy graph shows a rapid increase during the initial epochs, with both training and validation accuracy quickly reaching and stabilizing around 99%. This high level of accuracy, maintained consistently across epochs, indicates that the model is well-trained and highly capable of generalizing to unseen data, with minimal discrepancies between the training and validation sets. The loss graph complements this observation, showing a steep decline in both training and validation losses early in the training process. The losses stabilize at low values, reflecting the model's strong learning capability and its ability to minimize errors. Although there are minor fluctuations, particularly a noticeable spike in the validation loss around epoch 30, the overall trend suggests that the model effectively converges, with both losses remaining low and closely aligned. These results underscore the model's robustness and efficiency, highlighting its ability to achieve near-perfect accuracy with minimal overfitting. The minor variations in the validation loss suggest that the model handles the complexity of the data well, maintaining a high level of performance throughout the training process. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Class** & **Precision** & **Recall** & **F1-Score** & **Support** \\ \hline allow & 1.00 & 1.00 & 1.00 & 7545 \\ deny & 0.99 & 1.00 & 1.00 & 2994 \\ drop & 1.00 & 1.00 & 1.00 & 2562 \\ reset-both & 0.00 & 0.00 & 0.00 & 6 \\ accuracy & & & 1.00 & 13,107 \\ weighted avg & 0.75 & 0.75 & 0.75 & 13,107 \\ macro avg & 1.00 & 1.00 & 1.00 & 13,107 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Numerical results of the experimental study. The SMOTE algorithm was applied to balance the dataset and address the shortcomings of the initial model.