## 4 Results
The results of the subsequent experiments using this balanced dataset are presented in Table 5. Here, we see that the model performance was improved for all classes, including the previously problematic \"reset-both\" class, which now showed a precision of 0.77, a recall of 0.87, and an F1-Score of 0.82. The \"allow\" and \"drop\" classes maintained their perfect scores, while the \"deny\" classes showed a slight drop but still had high scores. When tested on the set of 13,107 samples in Table 4, the model's overall accuracy was 1.00, indicating a high level of predictive capability across all classes. The confusion matrix presented in Figure 7 underscores the elevated level of predictive precision achieved by the ANN after applying SMOTE to a balanced training dataset. Figure 5: Confusion matrix output of the study. Figure 6: Accuracy and loss graphs of training and validation metrics of ANN’s analysis. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Class** & **Precision** & **Recall** & **F1-Score** & **Support** \\ \hline allow & 1.00 & 1.00 & 1.00 & 6005 \\ deny & 0.85 & 0.75 & 0.80 & 6174 \\ drop & 1.00 & 1.00 & 1.00 & 5956 \\ reset-both & 0.77 & 0.87 & 0.82 & 5941 \\ accuracy & & & 0.90 & 24,076 \\ weighted avg & 0.91 & 0.90 & 0.90 & 24,076 \\ macro avg & 0.90 & 0.90 & 0.90 & 24,076 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Numerical results of the experimental study with ANNs + SMOTE function. The ANN demonstrated exceptional accuracy in the classification of 'allow', 'deny', and 'drop' directives, with the classification of 'drop' actions being particularly noteworthy due to its flawless accuracy. Notwithstanding a slight misclassification in the'reset-both' category, such instances were negligible in the grand scope of the data, emphasizing the model's resilience. The findings corroborate the effectiveness of employing SMOTE with an ANN to amplify the training dataset's balance, thereby significantly boosting the model's discriminative capacity across the various classes within the dataset. Figure 8 presents the graphs depicting the model's training and validation accuracy and its training and validation loss. The training and validation graphs presented provide critical insights into the performance of the ANN model when combined with SMOTE, as reflected in the numerical results of Table 5. The accuracy graph (left) shows that while training accuracy steadily increases, validation accuracy demonstrates fluctuations across epochs. These fluctuations indicate potential overfitting, where the model might be learning patterns specific to the training data but not generalizing well to unseen data. This behavior is consistent with the precision, recall, and F1-Scores observed, particularly for the minority classes like \"deny\" and \"reset-both\", where the recall values are lower, leading to a weighted average accuracy  
Figure 8: Accuracy and loss graphs of training and validation metrics of ANNs + SMOTE analysis. Figure 7: Confusion matrix output of study with ANNs + SMOTE function. of 90%. The loss graph (right) further corroborates these observations, with the validation loss not following the smooth decrease seen in the training loss. This divergence suggests that while the model is minimizing error on the training data, it struggles to achieve the same level of performance on the validation set, reinforcing the need for further tuning or alternative approaches to improve generalization. The results presented in Table 6 demonstrate the performance of the ANN model when applying the ADASYN function for data balancing. The overall accuracy is 85%, indicating a decrease compared to the SMOTE-enhanced model. This reduction is primarily attributable to the \"deny\" class, which shows a recall of 60% and an F1-Score of 68%, highlighting the model's struggle to accurately identify instances in this class. While the \"allow\" and \"drop\" classes exhibit high precision and recall, the \"reset-both\" class also encounters moderate challenges, as evidenced by its F1-Score of 79%.