## 5 Results
### Performance Analysis of Adversarial Training with LGBM and Realistic Windows PE Attack Methods
From this point of view, any reduction in \(\mathsf{fnr}\) may be considered a desirable behaviour, as long as it does not come at the expense of an excessive increase in \(\mathsf{fpr}\). We continued the analysis by examining the performance achieved in the configuration A + T to evaluate the accuracy of the predictions produced for the collection of the original Windows PE files augmented with the realistic Windows PE malware produced with one of the attack methods considered in the evaluation stage. Table 5 reports the values collected for both the detailed accuracy metrics and the summary accuracy metrics measured using the collection of predictions produced from \(LGBM^{\text{O}}\) and \(LGBM^{AT}\) in the evaluation configuration O + A. In this evaluation setting, the LGBM model learned with the adversarial training strategy gained accuracy with respect to all the summary accuracy metrics of this study when the realistic adversarial Windows PE malware considered in both the training and evaluation stages was produced with GAMMA. In fact, in examining the detailed accuracy metrics, we noted that the adversarial training strategy performed with GAMMA decreased the amount of true goodware (\(\mathsf{tg}\)) by increasing the amount of false malware (\(\mathsf{fm}\)) detected. However, it also increased the amount of true malware (\(\mathsf{tm}\)) by decreasing the amount of false goodware (\(\mathsf{fg}\)) detected. As the gain in the ability to correctly detect malware was greater than the reduction in the ability to correctly detect goodware, the use of the adversarial training strategy may be considered beneficial with GAMMA. In addition, the use of the adversarial training strategy gained accuracy with respect to \(\mathsf{fg}\), \(\mathsf{tm}\), \(\mathsf{oa}\), \(\mathsf{F}\) and \(\mathsf{fnr}\) when the realistic adversarial PE malware was produced with Extend. We recall that GAMMA was identified in this evaluation study as the most effective attack method, with Extend as the runner-up. So, this analysis highlights that adversarial training with GAMMA-produced samples can be a strategy to strengthen the LGBM model against this attack type. Notice that this analysis was performed on LGBM, which was, in the end, the most effective decision model for the Windows PE malware detection task in this study. \begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multicolumn{8}{c}{**Test Set (O)**} \\ \cline{2-9}  & **tg** & **fm** & **fg** & **tm** & **oa** & **F** & **fmr** & **fpr** \\ \hline \(LGBM^{O}\) & 13,432 & 62 & 60 & 13,481 & 0.9955 & 0.9955 & 0.0044 & 0.0046 \\ \hline \(LGBM^{AT}\) (Extend) & 13,423 & 71 & 54 & 13,487 & 0.9954 & 0.9954 & 0.0040 & 0.0053 \\ \hline \(LGBM^{AT}\) (Full DOS) & 13,422 & 72 & 55 & 13,486 & 0.9953 & 0.9953 & 0.0041 & 0.0053 \\ \hline \(LGBM^{AT}\) (Shift) & 13,423 & 71 & 60 & 13,481 & 0.9952 & 0.9952 & 0.0044 & 0.0053 \\ \hline \(LGBM^{AT}\) (FGSM) & 13,430 & 64 & 62 & 13,479 & 0.9953 & 0.9953 & 0.0046 & 0.0047 \\ \hline \(LGBM^{AT}\) (GAMMA) & 13,425 & 69 & 61 & 13,480 & 0.9952 & 0.9952 & 0.0045 & 0.0051 \\ \hline \hline \end{tabular} \({}^{1}\) O denotes the collection of decisions produced for the study’s collection of original Windows PE files, while A denotes the collection of decisions produced for the collection of realistic adversarial malware produced from the study’s original PE malware recorded in O.  
\end{table} 
Table 4: Detailed accuracy metrics (tg—amount of true goodware, fm—amount of false malware, fg—amount of false goodware and tm—amount of true malware), and summary accuracy metrics (oa—overall accuracy, F—Fscore, fmr—the false negative rate and fpr—the false positive rate) of LGBM\({}^{O}\) and \(LGBM^{AT}\) measured in the evaluation configuration O 1. The evaluation was conducted through the 3-fold CV. Adversarial Windows PE malware files used to perform the adversarial training strategy in the training stages of \(LGBM^{AT}\) were produced with Extend, Full DOS, Shift, FGSM padding + slack and GAMMA.