# AI-Assisted Security Alert Data Analysis with Imbalanced Learning Methods
## 4 Experiments
### Performance Evaluation
#### 4.4.1 Imbalanced Data
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**Algorithms** & **Accuracy (\%)** & **Recall (\%)** & \
\begin{tabular}{c} **Precision** \\ **(\%)** \\ \end{tabular} & **FPR (\%)** & **TNR (\%)** & **F-Score (\%)** & **BCR (\%)** \\ \hline \hline \multicolumn{8}{c}{Imbalanced security alert data} \\ \hline LR & 99.504 & 77.404 & 89.694 & 0.142 & 99.858 & 83.097 & 88.631 \\ \(k\)-NN & 99.341 & 91.346 & 73.359 & 0.531 & 99.469 & 81.370 & 95.407 \\ DT & 99.822 & 96.154 & 92.807 & 0.119 & 99.881 & 94.451 & 98.017 \\ LightGBM & 99.848 & **98.317** & 92.534 & 0.127 & 99.873 & 95.338 & **99.095** \\ XGBoost & **99.913** & 97.837 & **96.675** & **0.054** & **99.946** & **97.252** & 98.891 \\ \hline \multicolumn{8}{c}{Oversampled security alert data} \\ \hline \hline  & & \multicolumn{4}{c}{SMOTE} \\ \hline LR & 98.950 & **99.760** & 60.058 & 1.063 & 98.937 & 74.97 & 99.348 \\ \(k\)-NN & 98.753 & 99.279 & 55.886 & 1.255 & 98.745 & 71.515 & 99.012 \\ DT & 99.538 & 99.519 & 77.528 & 0.462 & 99.538 & 87.158 & 99.529 \\ LightGBM & **99.883** & 98.798 & **94.050** & **0.100** & **99.900** & **96.366** & 99.349 \\ XGBoost & 99.814 & 99.519 & 89.805 & 0.181 & 99.819 & 94.413 & **99.669** \\ \hline \multicolumn{8}{c}{ADASYN} \\ \hline LR & 99.000 & 92.067 & 62.378 & 0.889 & 99.111 & 74.369 & 95.589 \\ \(k\)-NN & 98.837 & **99.279** & 57.601 & 1.171 & 98.829 & 72.904 & 99.054 \\ DT & **99.860** & **99.279** & **92.394** & **0.131** & **99.869** & **95.713** & **99.574** \\ LightGBM & 99.841 & 98.798 & 91.741 & 0.142 & 99.858 & 95.139 & 99.328 \\ XGBoost & 99.818 & **99.279** & 90.175 & 0.173 & 99.827 & 94.508 & 99.553 \\ \hline \multicolumn{8}{c}{SVMSMOTE} \\ \hline LR & 98.924 & **99.760** & 59.456 & 1.090 & 98.910 & 74.506 & 99.335 \\ \(k\)-NN & 98.833 & 98.077 & 57.627 & 1.155 & 98.845 & 72.598 & 98.461 \\ DT & 99.602 & 99.519 & 80.077 & 0.397 & 99.603 & 88.746 & 99.561 \\ LightGBM & **99.886** & 99.038 & **94.064** & **0.100** & **99.900** & **96.487** & 99.469 \\ XGBoost & 99.822 & 99.519 & 90.196 & 0.173 & 99.827 & 94.629 & **99.673** \\ \hline \multicolumn{8}{c}{Ensemble resampled security alert data} \\ \hline  & & \multicolumn{4}{c}{MAS-NCR} \\ \hline LR & 99.189 & **99.519** & 66.134 & 0.816 & 99.184 & 79.463 & 99.351 \\ \(k\)-NN & 99.208 & 98.558 & 66.884 & 0.782 & 99.218 & 79.689 & 98.888 \\ DT & 99.792 & 99.038 & 88.985 & 0.196 & 99.804 & 93.743 & 99.421 \\ LightGBM & 99.833 & **99.519** & 90.789 & 0.162 & 99.838 & 94.954 & 99.679 \\ XGBoost & **99.848** & **99.519** & **91.593** & **0.146** & **99.854** & **95.392** & **99.686** \\ \hline \multicolumn{8}{c}{MAS-OSS} \\ \hline LR & 99.090 & 99.279 & 63.538 & 0.913 & 99.087 & 77.486 & 99.183 \\ \(k\)-NN & 99.420 & 98.317 & 73.694 & 0.562 & 99.438 & 84.243 & 98.878 \\ DT & 99.826 & 99.038 & 90.749 & 0.162 & 99.838 & 94.713 & 99.438 \\ LightGBM & 99.867 & 99.279 & 92.809 & 0.123 & 99.877 & 95.935 & 99.578 \\ XGBoost & **99.875** & **99.519** & **93.034** & **0.119** & **99.881** & **96.167** & **99.700** \\ \hline \hline \end{tabular}
\begin{tabular}{c c c c c c c} \hline \hline  & & \multicolumn{4}{c}{LR, \(k\)-NN, DT, LightGBM, and XGBoost stand for logistic regression, \(k\)-nearest neighbors, decision trees, light gradient boosting machine, and extreme gradient boosting algorithms, respectively. The highest evaluation performance scores are shown in bold font.  
\end{table} 
Table 4: Performance evaluation with different sampling settings on enterprise dataset.