MoAT: Meta-Evaluation of Anti-Malware Trustworthiness
Abstract
Many studies have proposed methods for the automated detection of malware.
The benchmarks used for evaluating these methods often vary, hindering a trustworthy comparative analysis of models.
We analyzed the evaluation criteria of overmalware detection methods from 2018-2022 in order to understand the current state of malware detection.
From our study, we devised several criteria for evaluating future malware detection methods.
Our findings indicate that a finer-grained class balance in datasets is necessary to ensure the robustness of models.
In addition, a metric robust to distribution shifts, e.g. PR-AUC, should be used in future studies to prevent the inflation of results in unrealistic distribution regimes.
The composition of datasets should also be disclosed in order to ensure a fair comparison of models.
To our knowledge, this study is the first to assess the trustworthiness of evaluations from multi-domain malware detection methods.
Introduction
In the last decade, the volume of malware distributed online has significantly increased.
There are more than one billion malicious programs being distributed online, and more than half a million new programs detected every day [15].
Common types of malware, such as adware, spyware and worms, increased by 200% in the past year.
Newer attacks have also emerged, including malware that mines cryptocurrency on victims’ devices or targets IoT devices [25].
Malware detection methods generally rely on file signatures and heuristics to identify malicious software.
Repositories such as VirusTotal, Mapedia and MalShare store file signatures from verified malware samples and match files based on existing signatures, while heuristic methods extract features from file contents and metadata and match based on rules that indicate whether a file is suspicious or not [3].
Though signatures can be a highly accurate signal of a malicious file it is trivial for attackers to evade detection by inserting perturbations into their code.
For heuristic analysis, the most common methods are static and dynamic analysis.
Static analysis, which uses static data such as PE headers, compression ratio, or opcodes, is similarly easy to evade through binary obfuscation [26].
While dynamic analysis offers more direct insights into the execution of a program, it is also more time intensive and requires analysts to run files in an emulator or virtual environment to observe dynamic effects.
Furthermore, dynamic analysis techniques can also be evaded through sandbox detection [2].
As the volume of distributed malware grows, comparing millions of files for similarities becomes an ever-increasing scalability challenge.
More granular matching techniques have emerged from security research [10] in an effort to reduce the time required for analysts to classify malware.
Thus, an opportunity has emerged for machine learning systems to assist in the defense against malicious cyberattacks.
The dynamics of machine learning models are ideal for large-scale classification and recent work in the use of models for malware detection have demonstrated great promise.
Many papers on malware detection methods boast >95% accuracy in detection [1].
However, the benchmarks used for deriving this evaluation differ widely.
While some authors used standardized platform-specific datasets to evaluate their malware detection methods, e.g. Drebin, others evaluated their methods on custom platform-agnostic datasets gathered from the wild, e.g. VirusTotal.
These datasets are often closed-source, which further limits the reproducibility of their work.
The metrics used to compare accuracy also vary across studies, from overall accuracy to Area Under the ROC Curve (AUC or AUC-ROC).
In other fields of applied machine learning, such as image recognition, benchmarks exist for comparing the results of studies, such as CartPole for reinforcement learning or ImageNet for image recognition.
Malware detection lacks such a benchmark, due in part to the moving window effect of attackers routinely altering and introducing new forms of malware into the environment, posing a challenge for researchers to maintain a trustworthy standardized dataset.
Nevertheless, as the volume of malware distribution continues to increase, large-scale models entrusted with protecting critical systems from cyberattacks will rely more on trustworthy evaluations, and the impact of maintaining a standard of evaluation across the field will become increasingly profound.
Related works
Several papers in the last few years have conducted comparative studies across various machine learning methods to detect different forms of malware.
Challenges cited often include dataset class imbalance [3], a lack of open and public benchmarks [22], concept drift [14], and adversarial examples [7, 1].
A study by Ge et al.
[11] analyzes the effects that imbalanced datasets can have on machine learning performance in malware detection.
Their study specifically focused on Android malware and found that the performance of the models in other studies varied heavily depending on the datasets used for training and evaluation.
In particular, by varying dataset composition by real-world and experimental settings, including quality, timeliness, and class imbalance between malware and benign files, they found that models tend to perform better on more balanced and recent datasets.
Liu et al.
[23] also considered the lack of a common data source for Android malware detection.
While datasets such as Drebin are among the largest and most cited [31], there are still numerous problems with data quality and timeliness, which can easily make malware detection models trained on these datasets obselete to newer attacks [4].
For instance, Drebin only contains samples from
August 2010 to October 2012, MalGenome only contains samples from August 2010 to October 2011, and data from VirusShare and Contagio lack metadata such as descriptions, user ratings, and download numbers [18].
There is the further problem of sample duplication in the case where multiple datasets are used in order to obtain a larger corpus for training.
Zhao et al.
[33] investigated the effects of overlapping data between Genome, Drebin, AMD, and RmvDroid.
Their study found that duplication rates between datasets ranged from <1% to 97.5%, and that performance achieved via training datasets with duplicate samples was always higher than that of datasets without duplicate samples.
Several more recent projects have also launched to address the dataset problem in malware detection.
The Canadian Institute for Cybersecurity (CIC) released several datasets in the last few years, e.g. CICMalDroid 2020, in order to promote the use of fair and balanced training data for malware detection [9].
While our study doesn’t introduce a new dataset, we investigated whether the evaluation criteria for state-of-the-art malware detection methods is trustworthy or if finer-grained criteria need to be established by the community.
In the current literature, there is only one other study related to the trustworthiness of malware detectors [19].
In contrast to Kumar et al.
, we chose to evaluate not only hardware-based methods, but also multi-domain methods in order to conduct a comprehensive study across various platforms of malware to investigate the trustworthiness of malware detection model evaluation.
[Table]
Meta-evaluation of malware detectors
Our methodology considers the evaluation criteria for malware detection methods published from 2018-2022.
Our dataset includes methodologies for detecting malware on Android, Windows, Linux,
Mac OS
[Table]
and various IoT operating systems.
The questions investigated in our study were:
• How trustworthy are the current state of evaluations of malware detection across all platforms and data types?
• How should models be evaluated in order to best measure the trustworthiness of their model’s performance in real-world scenarios?
3.1 Datasets
We analyzed the evaluation criteria used across 150 studies proposing machine learning methods for malware detection.
The papers were sourced from top-tier security conferences (USENIX, CCS,
S&P), as well as search engines for scholarly literature (Google Scholar, Springer).
Our search was limited to papers published in the last five years and only included those which proposed methods for malware detection using machine learning methods, excluding review and adversarial studies.
After surveying the studies, we extracted the highest-frequency datasets used by various malware detection methods and classified the datasets based on platform, data quality, timeliness (Table 1).
A significant number of studies used a custom-derived dataset.
In most cases, this was because the study was using a non-standard data representation for malware detection, e.g. ciphertexts [30], memory images [24], steganography [29], microarchitectural side channels (bus cycles, cache misses, branch instructions) [20], or dynamic analysis traces (system logs, etc.)
3.2 Metrics
Among the surveyed studies, the most common metric used for comparing model performance was accuracy, defined with true positive ( TP ), true negative ( TN ), false positive ( FP ), and false negative ( FN ) as
[Equation]
This was followed by the F1 score, which relates the harmonic mean of the precision and recall, which assists in mitigating biases caused by large outliers in either score: [Equation] The next most commonly used metrics were precision, recall, and AUC-ROC, which computes the area under the receiver operating characteristic curve in order to assess the performance of a binary classifier to discriminate across different thresholds.
In a survey study by Kouliaridis et al.
[17], it was shown that due to differences in datasets and evaluation metrics, it was challenging to compare state-of-the-art methods in Android malware detection.
Metrics from surveyed works included true positive rate (TPR), precision, recall, F1 score, accuracy, and AUC-ROC.
This inconsistency resulted in inflated results, such as the case where accuracy is used on an imbalanced dataset where a high volume of true negatives contributes to higher scores.
Of these metrics, AUC-ROC was argued to be the most robust measurement of effectiveness, as it remained conclusive even when datasets were imbalanced and was less sensitive to changes in class balance.
This is because AUC-ROC scales with the total number of detected true values rather than the total number of correctly detected values [8].
Li et al.
[21] proposed that the existence of undetected zero-days in benign datasets may pollute classification and render less robust metrics less meaningful in comparison.
Other metrics that are even more robust to class imbalance are PR-AUC and F1 score, which scale with the total number of correctly detected true values and are better representative of the performance requirements for anomaly detection to maximize true positives over minimizing false positives.
If the size of the estimated positive sample set is much smaller than the size of the overall sample set, the differences across models compared with PR-AUC are much greater than with AUC-ROC.
In particular, where AUC-ROC considers all samples in its scoring, PR-AUC disregards the performance of the classifier over true negatives.
Analysis
Risk Given the capabilities of machine learning systems to assist in large-scale cyberattacks and the relative lack of defense measures to counter such attacks, Hendrycks et al.
[13] recommends research to be steered towards defense measures for machine learning systems.
Our study follows in this direction, as we seek to improve the trustworthiness of large-scale machine learning models aimed at malware detection by critically evaluating the criteria used to measure their efficacy.
Even so, there are still risks associated with improving defense systems.
The unification of datasets for accessing malware, even for the purpose of training more robust models, will ultimately allow attackers to access the same malware.
Furthermore, open source methods will also make it easier for attackers to developer adversarial examples that evade detection from state-of-the-art methods.
Nonetheless, data provenance is necessary for model interpretability, and evidence has shown that attackers will continue to exploit systems regardless of whether they are open-source or reverseengineered.
Trustworthiness In the literature of machine learning, trustworthiness entails reliability on repeat decisions, fairness, robustness against input changes, explainability, and privacy preservation of sensitive data [32].
Our survey considers the trustworthiness of model evaluation in the context of malware detection methods.
Data from the survey shows that the most commonly cited datasets are not necessary the most well-balanced across time and class distributions.
Though well-calibrated models can resist differences in uncertainty over class distribution shifts, evidence shows that models trained on imbalanced datasets tend to have a detection bias towards the majority class[28].
Therefore, to ensure the reliability of models on independent and identically distributed (i.i.d.) data and robustness towards out-of-distribution (OOD) data, models should be trained on more balanced datasets.
The explainability of evaluations also varied, with many studies using metrics such as overall accuracy.
Accuracy tends to obscure the ability of a model to detect malware in real-world scenarios, where the data may comprise a small percentage of the total online data.
Furthermore, when various metrics are used for the evaluation of models, it can be more difficult to empirically compare their performance.
A common, interpretable benchmark that compares model performance in real-world scenarios addresses this challenge.
Regarding privacy, while evaluations of models do not tend to regard data privacy as a quantitative measure, privacy concerns do arise in the data acquisition of benign files, which is not always present in standardized malware datasets.
Applications scraped from various marketplaces and search engines may contain sensitive or confidential data, so efforts to circumvent the use of private information should be taken to ensure the trustworthiness of models.
Conclusion
We analyzed the criteria used for evaluating multi-domain malware detection methods and provided a risk assessment for datasets and metrics used in evaluation.
In addition, we devised guidelines for future practitioners to ensure the trustworthiness of models evaluation results.
By unifying results found across platforms and domains, we hope to steer future malware detection studies towards greater trustworthiness. Acknowledgments
We would like to thank our anonymous reviewers for their valuable feedback, which has helped to improve the quality of our work.
We also gratefully acknowledge constructive input from Dr. Marc
Fyrbiak and Dr. Erwin Quiring.
A Appendix A.1 Evaluating trustworthiness
Within the machine learning literature, trustworthiness of a model is understood to be measured by the quality of the model’s predictive uncertainty.
When measuring the uncertainty of a classification model, the following equation may be used [16]:
[Equation]
where y is the classification label, x ∗ is the test data, D is the training dataset, and Θ is the model parameters for the distribution over which a Bayesian model learns (the posterior).
Our investigation was concerned with the impact of different dataset compositions on the trustworthiness of model evaluation.
We were less concerned with the impact of dataset distribution shifts on the empirical performance of individual models, although a study by Ovadia et al.
[28] found that for all models, regardless of method, performance was consistently worse with increasing distribution shift (though deep ensembles were the least affected).
A.2 Robustness through model calibration
In practice, especially for online models which observe unpredictable shifts in data distribution, robustness under out-of-distribution (OOD) inputs is important for safe deployment.
Calibration on a hold-out set is a common practice to enable accurate assessment of risk, as it allows practitioners to evaluate how accuracy degrades over time and allows a model to abstain from making decisions with low confidence.
Probability scoring can be computed with the Brier Score [5]:
[Equation]
where p ( y | x n , θ ) is the predicted probability vector and y n is the one-hot encoded true response, or with the Expected Calibration Error (ECE) [27]:
[Equation]
which measures the correspondence between predicted probabilities and empirical probabilities.
Findings by Ovadia et al.
showed that calibration on an independent and identically distributed (i.i.d.) dataset, e.g. on a test set, did not result in better calibration under dataset shift.
A.3 Comparison of malware datasets
In our evaluation, we compared characteristics of commonly used datasets for malware detection.
To expand upon the differences between these datasets, several distinguishing factors are discussed.
Platform There are numerous platforms for which malware can be targeted.
Some of the datasets in our study, including Drebin, AndroZoo, and AMD, specifically contain files formatted to one class of platforms, namely Android operating systems.
Others, such as VirusTotal, contain a larger selection of file formats, including Microsoft Advanced Streaming/Systems Format (ASF) files, Java .class bytecode files, compressed files, ISO image files, and Apple MachO files.
Date range Malware is most often deployed for currently supported platforms.
Although system architectural vulnerabilities may persist through updates and users may continue to use older versions of platforms, training on older malware datasets poses the risk of data drift.
Some datasets that not actively maintained, such as the Malicia Project, are explicitly deprecated for this reason.
Metadata Along with file contents, file metadata is often used in static analysis.
The method of data aggregation for datasets affects the availability of this metadata.
For instance, Android files on
VirusTotal may contain API data, but are missing app descriptions, user ratings, and install statistics that are available for files downloaded from Google Play.
Dataset size Obtaining a large corpus of balanced data is important for training accurate detection models.
Commercial malware detection models can be trained on tens to hundreds of millions of samples [12].
From the datasets we surveyed, the largest corpus was VirusTotal with 2.4 billion samples, and the smallest was theZoo with 351 samples.
A full table of dataset sizes is available here.
Reliability For crowd-sourced datasets, such as Koodous, tag reliability is a potential risk.
The platform ranks files based on user-generated descriptions and votes, with no safety measures for data pollution from malicious actors.
A.4 Takeaways and recommendations
From our study, the following best practices are recommended for future practitioners to ensure the trustworthiness of evaluation results:
• Disclose the origins of datasets or to make available custom derived datasets whenever possible, i.e. when privacy measures are taken to protect sensitive data used for training.
• Disclose the constraints of the data being detected by a model (e
g. Windows file binaries or Android APKs) rather than broadly classifying generally malicious files.
Even if the detection method is aimed at detecting malicious files agnostic of platform, composition should still be detailed in the methodology to aid in the evaluation of class balance.
• Disclose how benign files were obtained, especially in the case of custom datasets extracted from the wild, in order to reduce the risk of accessing sensitive or confidential data.
• Balance datasets used for training and testing over time and classes (e.g. benign/malicious or adware/banking/SMS malware/riskware/benign for a more fine-grained classification) to minimize the effects of distribution shift.
• Compare the performance of models using ranking metrics such as F1 score or PR-AUC, especially if datasets are imbalanced.
A.5 Artifacts
The dataset of papers surveyed in this study is available at this link.