# Adversarial Robust and Explainable Network Intrusion Detection Systems Based on Deep Learning
## 3 Experiments and Results
### Baseline Model Adversarial Resistant Evaluation
#### 3.9.1 Adversarial Robustness Evaluation  
Table 8 presents the major results of our research on the adversarial robustness of NIDS. We used our adversarial robust evaluation (10) to measure the robustness of adversarial-trained models. From Table 8, the figures under the block of columns below the ROC-AUC label, Normal, indicate ROC-AUC results for benign test samples, and DeepFool represents ROC-AUC results for DeepFool adversarial test samples. Likewise, FGSM represents the AUC results for FGSM adversarial test samples, and PGD represents ROC-AUC results for PGD adversarial test samples. Columns that are below the Method label indicate the models under evaluation. For example, Normal indicates ROC-AUC adversarial-trained models, DeepFool_M indicates a DeepFool adversarial-trained model, FGSM_M indicates FGSM adversarial trained, and PGD_M indicates PGD adversarial-trained model. The block of columns under the Adversarial Robust Evaluation label indicates the adversarial robust measure. This evaluation follows Equation (12) in percentage terms. It measures the rate of change of a model's ROC-AUC after the model has been attacked by an adversarial attack. The Normal column indicates no attack, meaning that the classification or prediction process is being performed using benign samples. The DeepFool_A column indicates the rate of change of the respective model's ROC-AUC after the DeepFool attack. The FGSM_A column indicates the rate of change of the respective model's ROC-AUC after the FGSM, and PGS_A indicates the rate of change of the respective model's ROC-AUC after the PGD attack. The overall results indicate that the adversarial-trained models are more robust to new adversarial samples and outperform the baseline MPL. This confirms the work of [2; 3; 19; 25; 39]. From Table 8, it can be noted that all adversarial attacks greatly reduced baseline MPL's \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{4}{c}{**ROC-AUC**} \\ \cline{2-5}  & **Normal** & **DeepFool** & **PGD** & **FGSM** \\ \hline Decision Tree & 0.79 & 0.65 & 0.27 & 0.30 \\ Random Forest & 0.78 & 0.53 & 0.23 & 0.26 \\ Linear SVM & 0.78 & 0.51 & 0.23 & 0.26 \\ Base MLP & 0.87 & 0.62 & 0.61 & 0.60 \\ \hline \hline \end{tabular}
\end{table} 
Table 7: Adversarial attacks evaluation on base models. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Method**} & \multicolumn{4}{c}{**ROC-AUC**} & \multicolumn{4}{c}{**Adversarial Robust Evaluation**} \\ \cline{2-7}  & **Normal** & **DeepFool** & **FGSM** & **PGD** & **Normal** & **DeepFool** & **A** & **FGSM_A** & **PGD_A** \\ \hline DeepFool\_M & 0.86 & 0.86 & 0.84 & 0.73 & 0 & \(-0.20\%\) & \(-2.33\%\) & \(-\)**15.12\%** \\ \hline FGSM\_M & 0.86 & 0.77 & 0.86 & 0.75 & 0 & \(-10.47\%\) & 0.01\% & \(-\)**12.79\%** \\ \hline PGD\_M & 0.87 & 0.79 & 0.87 & 0.76 & 0 & \(-\)**9.20\%** & **0** & \(-\)**13.65\%** \\ \hline Baseline MPL & 0.87 & 0.62 & 0.60 & 0.61 & 0 & \(-28.74\%\) & \(-31.03\%\) & \(-29.89\%\) \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Adversarial attacks evaluation on adversarial-trained models. ROC-AUC; under PGD attack, it was reduced by 29.89%, while under FGSM, it was reduced by 31.03%, and under DeepFool attack, it was reduced by 28.74%. PGD adversarial-trained model was a more robust mode than DeepFool- and FGSM-trained models. FGSM attack did not impact the PGD-trained model; there was no change to the PGD-trained model's ROC-AUC after the FGSM attack. The DeepFool attack caused minimal damage to a PGD adversarial-trained model with a \(-9.20\%\) reduction to the PGD adversarial-trained model's ROC-AUC compared to what the PGD adversarial attack caused to the DeepFool adversarial-trained model. PGD attack caused a 15.12% reduction in the DeepFool adversarial-trained model and a 12.79% reduction in the FGSM-trained model.