## 5 Results
### Performance Analysis of Adversarial Training with LGBM and Realistic Windows PE Attack Methods  
In this section, we analyse the accuracy performance of the adversarial training strategy by exploring the effect of this strategy on the accuracy performance of the machine learning model trained with LGBM when this strategy is evaluated using a collection of the original Windows PE files (denoted as O), as well as when it is evaluated using the collection of the original Windows PE files extended with the adversarial Windows PE malware (denoted  
Figure 5: Shapley values measured for the top 20 input features (axis \(Y\)) of the LGBM input space and plotted with respect to the values measured for the features in the explained samples (axis \(X\)) for the original 6857 PE malware files (**a**) and the adversarial malware counterparts produced from GAMMA (**b**). Changes in the Shapley value-based feature ranking are marked in (**b**). as O + A). The study was conducted considering the adversarial Windows PE malware files generated with Extend, Full DOS, Shift, FGSM padding+slack and GAMMA. In this study, we used a 3-fold cross-validation (CV) of the study's dataset. For each fold, we learned both \(LGBM^{\text{O}}\) from the Windows PE files falling in the two hold-out folds and \(LGBM^{AT}\) from the Windows PE files falling in the two hold-out folds augmented with the realistic adversarial malware produced with a selected attack method. Both models were used to classify the Windows PE files of the selected fold. The accuracy metrics were, finally, computed using the decisions produced with the 3-fold CV methodology for all the Windows PE files (and eventually their adversarial counterparts) of the study's dataset. We started the analysis of the results by examining the performance achieved in configuration O to evaluate the accuracy of predictions produced for the collection of the original Windows PE files. Table 4 reports the values collected for both the detailed accuracy metrics (tg--the amount of true goodware, \(\mathsf{fm}\)--the amount of false malware, \(\mathsf{fg}\)--the amount of false goodware and \(\mathsf{tm}\)--the amount of true malware) and the summary accuracy metrics (oa--overall accuracy, \(\mathsf{F}\)--Fscore, \(\mathsf{fm}\)-- the false negative rate and \(\mathsf{fpr}\)--false positive rate). These metrics were measured using the collection of predictions produced from \(LGBM^{\text{O}}\) and \(LGBM^{AT}\) in the evaluation configuration O. These results show that the use of the adversarial training strategy worsened the performance of LGBM in terms of \(\mathsf{tg}\), \(\mathsf{fm}\), \(\mathsf{oa}\), \(\mathsf{F}\) and \(\mathsf{fpr}\) regardless of the attack method used to perform the adversarial training strategy. On the other hand, the use of the adversarial training strategy allowed \(LGBM^{AT}\) to perform better than (or equally to) \(LGBM^{\text{O}}\) in terms of \(\mathsf{fg}\), \(\mathsf{tm}\) and \(\mathsf{fnr}\) when adversarial samples considered in the adversarial training stage of \(LGBM^{AT}\) were produced with Extend, Full DOS and GAMMA. Specifically, when considering these attack methods, the injection of realistic adversarial samples in the training stage allowed us to learn about an LGBM model that increases the number of malware decisions while also considering the original PE files only for evaluation. Notably, this had the consequence of increasing the amount of true malware detected. However, this also had the consequence of reducing the amount of true goodware detected by increasing the amount of false malware. Although we are aware that handling false malware is an inefficient use of time and resources, which may prevent a cybersecurity team from handling actual malware, we also note that false goodware may raise a serious cyber risk affecting the capacity to promptly mitigate cyber threats' consequences.