During the past two years, Flash malware has become one of the most insidious threats to detect, with almost 600 critical vulnerabilities targeting Adobe Flash Player disclosed in the wild. Research has shown that machine learning can be successfully used to tackle this increasing variability and sophistication of Flash malware, by simply leveraging static analysis to extract information from the structure of the file or from its bytecode. However, the robustness of such systems against well-crafted evasion attempts - also known as adversarial examples - has never been investigated. In this paper, we first discuss how to craft adversarial Flash malware examples, and show that it suffices to only slightly manipulate them to evade detection. We then empirically demonstrate that popular defense techniques proposed to mitigate such threat, including re-training on adversarial examples, may not always be effective. We argue that this occurs when the feature vectors extracted from adversarial examples become indistinguishable from those of benign data, meaning that the given feature representation is intrinsically vulnerable. In this respect, we are the first to formally define and quantitatively characterize this vulnerability, highlighting when an attack can be countered by solely improving the security of the learning algorithm, or when it requires also considering additional features. We conclude the paper by suggesting alternative research directions to improve the security of learning-based Flash malware detectors.
Hidost @cite_10 is a static system that only focuses on the structure of the file. More specifically, it considers sequences of objects belonging to the structure of the file as features. The system evaluates the most occurring paths in the training dataset, and extracts features that are based on the training data. This might be dangerous from the perspective of targeted attacks, as a malicious test file with completely different paths might be able to evade detection. Moreover, the system does not analyze the embedded ActionScript code. In this way, an attacker might simply evade the system by perfectly mimicking the structure of the file, regardless of the type of code it contains.
@cite_10
2525598640
Malicious software, i.e., malware, has been a persistent threat in the information security landscape since the early days of personal computing. The recent targeted attacks extensively use non-executable malware as a stealthy attack vector. There exists a substantial body of previous work on the detection of non-executable malware, including static, dynamic, and combined methods. While static methods perform orders of magnitude faster, their applicability has been hitherto limited to specific file formats. This paper introduces Hidost, the first static machine-learning-based malware detection system designed to operate on multiple file formats. Extending a previously published, highly effective method, it combines the logical structure of files with their content for even better detection accuracy. Our system has been implemented and evaluated on two formats, PDF and SWF (Flash). Thanks to its modular design and general feature set, it is extensible to other formats whose logical structure is organized as a hierarchy. Evaluated in realistic experiments on timestamped datasets comprising 440,000 PDF and 40,000 SWF files collected during several months, Hidost outperformed all antivirus engines deployed by the website VirusTotal to detect the highest number of malicious PDF files and ranked among the best on SWF malware. Adobe Flash is present on nearly every PC, and it is increasingly being targeted by malware authors. Despite this, research into methods for detecting malicious Flash files has been limited. Similarly, there is very little documentation available about the techniques commonly used by Flash malware. Instead, most research has focused on JavaScript malware. This paper discusses common techniques such as heap spraying, JIT spraying, and type confusion exploitation in the context of Flash malware. Where applicable, these techniques are compared to those used in malicious JavaScript. Subsequently, FlashDetect is presented, an offline Flash file analyzer that uses both dynamic and static analysis, and that can detect malicious Flash files using ActionScript 3. FlashDetect classifies submitted files using a naive Bayesian classifier based on a set of predefined features. Our experiments show that FlashDetect has high classification accuracy, and that its efficacy is comparable with that of commercial anti-virus products. This paper describes EMBER: a labeled benchmark dataset for training machine learning models to statically detect malicious Windows portable executable files. The dataset includes features extracted from 1.1M binary files: 900K training samples (300K malicious, 300K benign, 300K unlabeled) and 200K test samples (100K malicious, 100K benign). To accompany the dataset, we also release open source code for extracting features from additional binaries so that additional sample features can be appended to the dataset. This dataset fills a void in the information security machine learning community: a benign malicious dataset that is large, open and general enough to cover several interesting use cases. We enumerate several use cases that we considered when structuring the dataset. Additionally, we demonstrate one use case wherein we compare a baseline gradient boosted decision tree model trained using LightGBM with default settings to MalConv, a recently published end-to-end (featureless) deep learning model for malware detection. Results show that even without hyper-parameter optimization, the baseline EMBER model outperforms MalConv. The authors hope that the dataset, code and baseline model provided by EMBER will help invigorate machine learning research for malware detection, in much the same way that benchmark datasets have advanced computer vision research. The ability to automatically classify files based on their low-level, short-range structures is of particular importance in computer forensics. We report a study on the automatic learning of file classification using byte sub-stream kernels that capture these low-level structures. We automatically discover byte-level patterns in a file by extracting a byte sequence feature map and use a suffix trie data structure to efficiently store and manipulate the feature map. Using the feature map we compute the spectrum kernel and, together with a support vector machine classifier algorithm, we are able to efficiently categorize a variety of different system and application file types. Experiments have provided good file classification performance results.