# Securing Network Traffic Classification Models against Adversarial Examples Using Derived Variables
## 6 Results and Analysis
### Comparative Analysis
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Work** & **Attack** & **Acc (\%)** & **Rec (\%)** & **F1 (\%)** \\ \hline [PERSON] et al. [6] & RAF & - & 0.90 & 0.82 \\ \hline [PERSON] et al. [59] & GAN & 0.82 & 0.83 & 0.84 \\ \hline [PERSON] et al. [60] & A2 & 0.64 & - & - \\ A3 & 0.51 & - & - \\ A4 & 0.63 & - & - \\ A5 & 0.78 & - & - \\ \hline Ours & DVars & 0.84 & 0.84 & 0.83 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Comparison of defense effectiveness of the leading models using the CSE-CIC-IDS2018 dataset. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Work** & **Model** & **Acc (\%)** & **Prec (\%)** & **Rec (\%)** & **F1 (\%)** \\ \hline [PERSON] et al. [6] & KNN & - & 0.99 & 0.99 & 0.99 \\ [PERSON] et al. [58] & RF & 0.92 & - & 0.91 & 0.94 \\ [PERSON] et al. [59] & RF & 0.91 & - & 0.91 & 0.94 \\ [PERSON] et al. [60] & N\(/\)A & 0.94 & - & - & - \\ Ours & KNN & 0.99 & 0.99 & 0.99 & 0.99 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Comparison of baseline performance of the leading models using the CSE-CIC-IDS2018 dataset. Figure 5: Performance evaluation of AdverNet trained on the baseline dataset. Figure 6: The baseline results showing accuracy, precision, recall, and F1-score with respect to each class of the dataset. Furthermore, Figure 7 presents the results of when the models were subjected to adversarial attacks with different perturbation intensity. The figure highlights the decrease in accuracy for each model to below 50%. As observed in the confusion matrix, the performance of these models declined even with small, subtle changes in the dataset, leading to a high class misclassification rate. A large proportion of DoS attacks were classified as benign (Figure 8). However, our proposed defense method still provided reasonable protection, maintaining relatively high accuracy, 84%, and achieving satisfactory precision, 87%, using our best-performing model, KNN. Based on the results presented in Table 8 and Figures 9 and 10, it is evident that the proposed defense method enhances the robustness of network traffic classification models against adversarial attacks. In the absence of attacks, the models achieved high accuracy, precision, recall, and F1-score, indicating their effectiveness in classifying the network traffic flows. However, when subjected to adversarial attacks of increasing intensity, the performance of these models gradually declined. Nevertheless, our defense method still provided significant protection, consistently maintaining accuracy of above 70% and 65% for low- and high-intensity attacks, respectively. Notably, KNN achieved the highest accuracy rates, 84%, and 72%, for low- and medium-intensity adversarial attacks, respectively, outperforming the DT and RF models. These results highlight the efficacy of the proposed approach in mitigating the impact of adversarial attacks on network traffic classification models. Through the integration of feature selection and variable derivation techniques, our defense method introduces additional randomness into input data. This increased randomness poses a challenge for attackers attempting to craft AEs that can completely deceive classification models. As a result, our defense method maintains reasonable levels of accuracy, precision, recall, and F1-score, even in adversarial scenarios. Figure 7: The results with the AE effect showing accuracy with respect to the perturbation intensity. Figure 8: The confusion matrix showing results with AEs present with respect to low perturbation intensity (\(\epsilon=0.01\)). Figure 9: The results with defense effect, showing accuracy, precision, recall, and F1-score with respect to perturbation intensity.