# Improving the Robustness of AI-Based Malware Detection Using Adversarial Machine Learning
## 8 Discussion  
The main goal of this research is to improve the robustness of the classification models by using adversarial training methodology. In the proposed architecture, the experimental results are produced by applying the methodology which was presented by the authors on a malware dataset known as MaleVis. By using this dataset, the authors were able to observe the behavior of the adversaries of different epsilon values on malware images belonging to 26 different classes. The authors applied this methodology on three different classification models which enabled them to study how vulnerable each model was and whether they could defend against such attack with the help of a defense mechanism.  
Table 5 shows comparison of the accuracies obtained by training all three models before the attack, after attack and after defense. After evaluating the accuracies of the three models which were trained by conventional techniques, it was observed that EfficientNet B0 performed better than the random forest algorithm and the CNN algorithm in terms of  
\begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Class**} & \multirow{2}{*}{**Normal Image**} & **Images with** & \multirow{2}{*}{**Adversarial Image**} & **The Anomaly on** \\  & & **Anomaly Detection** & & **Adversarial Image** \\ \hline \multirow{4}{*}{Expiro} & \multirow{4}{*}{Expiro} & \multirow{4}{*}{Expiro} & \multirow{4}{*}{Expiro} \\  & & & 25 \\  & & & 25 \\  & & & 25 \\  & & & 25 \\  & &  
producing the highest accuracy, even though CNN was able to classify more classes with an accuracy of 100 percent. This happened because the number of misclassified classes was also higher in the CNN model than in the EfficientNet B0 model. During the analysis, the authors also saw that the Vilsel class, which belonged to the trojan family of malware, was classified correctly the most often by all the three models, whereas the Neshta class, which belongs to the virus family, was misclassified the most by all three models.  
After the adversarial attack was performed, EfficientNet B0 was affected the least when compared to the other two models. In all three models, when attacked with the images perturbed with an epsilon value of 0.01, the authors found that only the confidence level in predicting the respective malware classes dropped for the classes with significantly higher accuracies before the attack was performed (example: class Vilsel), but for the classes such as Neshta, where the prediction accuracies were extremely low, images perturbed with 0.01 epsilon completely misclassified them. Additionally, on average for each class, an accuracy drop of 35% was observed for all of the classes for all the models when the models were attacked with images that were perturbed with an epsilon value of 0.15. It needs to be noted that when attacked with images perturbed with a 0.01 epsilon value, the perturbations are not significantly visible to the naked eye, whereas, in contrast, the epsilon value of 0.15 perturbs the images to such an extent that it becomes a little obvious that the image has been tampered with.  
After evaluating the defense techniques applied by the authors in order to make the models much more robust against the adversarial attacks, it was observed that models were now able to regain their approximate actual accuracies which were compromised during the attacks, thus making the models far more robust and accurate against the FGSM adversarial attack. Not only the accuracy also but the F1 score and the precision showed a significant rise, and thus the goal of having a highly trained and robust model was achieved.