# Adversarial Training for Mitigating Insider-Driven XAI-Based Backdoor Attacks
## 4 Experiments
### Adversarial Training
#### 4.2.2 Performance Metrics
\begin{table}
\begin{tabular}{c c c c c c c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Data**} & \multirow{2}{*}{**Model**} & \multicolumn{8}{c}{**ROS**} & \multicolumn{8}{c}{**SMOTE**} & \multicolumn{8}{c}{**VAE**} \\ \cline{3-14}  & & **P** & **R** & **F** & **K** & **M** & **P** & **R** & **F** & **K** & **M** & **P** & **R** & **F** & **K** & **M** \\ \hline \multirow{8}{*}{V4} & RF & 0.549 & 0.729 & 0.544 & 0.343 & 0.426 & 0.556 & 0.734 & 0.615 & 0.779 & 0.786 & 0.581 & 0.712 & 0.610 & 0.811 & 0.810 \\ \cline{2-14}  & XGB & 0.782 & 0.761 & 0.759 & 0.891 & 0.886 & 0.749 & 0.518 & 0.575 & 0.427 & 0.279 & 1.000 & 0.709 & 0.792 & 0.789 & 0.869 \\ \cline{2-14}  & LGB & 0.368 & 0.866 & 0.419 & 0.156 & 0.290 & 0.391 & 0.793 & 0.422 & 0.071 & 0.193 & **0.420** & 0.80 & 0.481 & 0.201 & 0.32 \\ \cline{2-14}  & SNN\_MLP & 0.531 & 0.778 & 0.551 & 0.358 & 0.461 & 0.556 & 0.764 & 0.568 & 0.491 & 0.563 & 0.510 & 0.550 & 0.61 & 0.331 & 0.452 \\ \cline{2-14}  & SNN\_1 DCNN & 0.277 & 0.921 & 0.289 & 0.061 & 0.174 & 0.44 & 0.793 & 0.517 & 0.487 & 0.557 & 0.45 & 0.801 & 0.53 & 0.5 & 0.462 \\ \cline{2-14}  & TabNet & 0.373 & 0.491 & 0.298 & 0.086 & 0.086 & 0.742 & 0.487 & 0.508 & 0.367 & 0.462 & 0.341 & 0.440 & 0.54 & 0.381 & 0.420 \\ \hline \multirow{8}{*}{V5} & RF & 0.317 & 0.456 & 0.341 & 0.225 & 0.261 & 0.253 & 0.464 & 0.287 & 0.175 & 0.234 & 0.280 & 0.481 & 0.312 & 0.190 & 0.24 \\ \cline{2-14}  & XGB & 0.208 & 0.719 & 0.181 & 0.008 & 0.059 & 0.207 & 0.714 & 0.189 & 0.012 & 0.073 & 0.220 & 0.731 & 0.21 & 0.021 & 0.08 \\ \cline{1-1} \cline{2-14}  & LGB & 0.372 & 0.665 & 0.337 & 0.069 & 0.171 & 0.236 & 0.659 & 0.262 & 0.079 & 0.179 & 0.251 & 0.67 & 0.272 & 0.08 & 0.181 \\ \cline{1-1} \cline{2-14}  & SNN\_MLP & 0.231 & 0.606 & 0.252 & 0.089 & 0.177 & 0.221 & 0.614 & 0.234 & 0.048 & 0.131 & 0.240 & 0.622 & 0.261 & 0.054 & 0.146 \\ \cline{1-1} \cline{2-14}  & SNN\_1 DCNN & 0.204 & 0.779 & 0.16 & 0.007 & 0.056 & 0.745 & 0.329 & 0.724 & 0.039 & 0.14 & **0.321** & **0.750** & **0.342** & **0.052** & **0.159** \\ \cline{1-1} \cline{2-14}  & TabNet & 0.383 & 0.364 & 0.375 & 0.324 & 0.324 & 0.625 & 0.556 & 0.111 & 0.134 & 0.541 & **0.630** & **0.563** & **0.121** & **0.145** & **0.083** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Performance comparison of various models using oversampling and VAE methods. We trained the models on original data and used adversarial training that combined the original dataset with artificial samples produced by generative adversarial models. We used Random Oversampling (ROS), SMOTE oversampling, and a variational autoencoder (VAE). Tables 3 and 4 give the performance analysis of the multi-class classification with and without data augmentation. The Kappa and MCC measures the number of false positives and false negatives in the model that result in higher misclassification. In an ideal case, all the metrics should have higher values. Since the original data were insufficient, we applied the oversampling methods ROS and SMOTE. Even though ROS and SMOTE generated more samples, the diversity of data being created was not appreciable and is reflected in the model performance. Overall, ROS did not help in boosting the model, whereas SMOTE led to improved metrics in specific models. This trend is seen in both versions of the data, which have varying classes and class distribution. As shown Table 3, the original extremely imbalanced data led to worse performance in all models except for XGBoost. As usual, the tree-based algorithms work well for insider threat data, even with extreme imbalance. Specifically, the XGBoost algorithm can classify well when the data are highly imbalanced, whereas Random Forest does not perform well. Random Forest slightly increased the metric when trained using the SMOTE oversampling. As seen in the table, TabNet and LightGBM could yield better precision. LightGBM shows improvement in all metrics, whereas TabNet obtained high precision but did not show improvement in other metrics.