# Leveraging Adversarial Samples for Enhanced Classification of Malicious and Evasive PDF Files
## 1 Introduction
However, there are two issues with such an approach, (1) there is a need to identify the range of predicted scores for the uncertain classification, and (2) there is no guarantee that evasive samples are the ones falling in the uncertainty region. To address these issues, we propose the idea of leveraging samples that are known to be evasive and to perform adversarial learning instead of active learning, where we train the malware classifier on data containing evasive and non-evasive samples. Moreover, instead of relying on the certainty of the malware classifier's predictions to detect evasion, we propose the idea of using this mix of evasive and non-evasive data to build standalone models that detect evasion. These two missing pieces are the focus of this study, and accordingly, we propose three approaches. 1. Building a robust malware classifier by performing the training on a mix of evasive and non-evasive data. 2. Building a hierarchical system that first classifies if a PDF is evasive or not, and then, checks for malware by forwarding the PDF to a model that deals exclusively with evasive data or another model that deals only with non-evasive data. 3. Building a multi-label classifier that detects evasion and maliciouss simultaneously and independently instead of relying on two dependent models as in the second approach. This classifier is also trained on a combination of evasive and non-evasive data. The building blocks of the three approaches are illustrated in Figure 1. To implement these approaches, we collected training data from various sources, and they fall under two categories: evasive and non-evasive. * Evasive data: these were used for testing or performing attacks on PDF malware classifiers in multiple previous studies [9; 16; 17; 18; 19]. * Non-evasive data: we used the Contagio dataset [20] along with the Surrogate dataset [18], which include benign files from Google, and malicious files from VirusTotal [21]. To assess the capabilities of the proposed approaches to detect malware under normal circumstances, as well as when dealing with evasive PDF files, we evaluated two datasets. The first one contains more than 100,000 samples collected from the network of a large university campus for 6 days [22]. The second one contains 500,000 evasive samples that we generated using a Tabular Variational AutoEncoder (IVAE) model [23] by relying on the Synthetic Data Vault (SDV) library [24]. Both test cases confirmed the robustness and reliable performance of the proposed approaches on all metrics. We compared the results with a baseline model that was trained on non-evasive samples only. The three approaches performed similarly to the baseline when tested against the university dataset: f1-score of 95% and AUC 0.99. On the other hand, the approaches provided an increase of 80% in the f1-score and an increase of 0.8 in the AUC compared to the baseline when tested against the generated evasive dataset. The f1-score of the baseline model is 15.32% and its AUC is 0.13, while the f1-score of our approaches is 95%, and the AUC is 0.95-0.97. Note that approaches 2 and 3 provided an accuracy of 88.29% and 94.23%, respectively, in detecting evasion when tested against the generated evasive dataset. The contributions of this work are listed below:  
* To the best of our knowledge, this work is the first (1) to apply adversarial retraining for PDF malware classifiers to enhance their robustness against evasion attacks and (2) to develop a standalone model capable of detecting evasion attacks in PDF files with a high accuracy, which lays the foundation for future extensions. * We proposed two systems that can simultaneously detect evasion and maliciousness in PDF files. * We generated a large dataset of 500,000 PDF samples with evasive-like signatures and made it publicly available, providing researchers with a valuable resource for building and testing evasion detection systems, as well as enhancing the robustness of malware detection systems.