# Debiasing Android Malware Datasets: How Can I Trust Your Results If Your Dataset Is Biased?

Tomás Concepción Miranda, Pierre-François Gimenez, Jean-François Lalande, Valérie Viet Triem Tong, Pierre Wilke

► To cite this version:

Tomás Concepción Miranda, Pierre-François Gimenez, Jean-François Lalande, Valérie Viet Triem Tong, Pierre Wilke. Debiasing Android Malware Datasets: How Can I Trust Your Results If Your Dataset Is Biased?. IEEE Transactions on Information Forensics and Security, 2022, 17, pp.2182-2197.
10.1109/tifs.2022.3180184 . hal-03700082

HAL Id: hal-03700082

https://inria.hal.science/hal-03700082

Submitted on 20 Jun 2022

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Debiasing Android Malware Datasets: How Can I Trust Your Results If Your Dataset Is Biased?

Tomás Concepción Miranda¹, Pierre-François Gimenez¹, Jean-François Lalande¹,
Valérie Viet Triem Tong¹, and Pierre Wilke¹

**Abstract**—Android security has received a lot of attention over the last decade, especially malware investigation. Researchers attempt to highlight applications’ security-relevant characteristics to better understand malware and effectively distinguish malware from benign applications. The accuracy and the completeness of their proposals are evaluated experimentally on malware and goodware datasets. Thus, the quality of these datasets is of critical importance: if the datasets are outdated or not representative of the studied population, the conclusions may be flawed. We specify different types of experimental scenarios. Some of them require unlabeled but representative datasets of the entire population. Others require datasets labeled with valuable characteristics that may be difficult to compute, such as malware datasets. We discuss the irregularities of datasets used in experiments, questioning the validity of the performances reported in the literature. This article focuses on providing guidelines for designing debiased datasets. First, we propose guidelines for building representative datasets from unlabeled ones. Second, we propose and experiment a debiasing algorithm that, given a biased labeled dataset and a target representative dataset, builds a representative and labeled dataset. Finally, from the previous debiased datasets, we produce datasets for experiments on Android malware detection or classification with machine learning algorithms. Experiments show that debiased datasets perform better when classifying with machine learning algorithms.

**Index Terms**—Datasets, malware, experiments.

## I. INTRODUCTION

DUE to the growing number of devices equipped with the Android operating system, attackers have developed a large number of malicious applications that may steal personal information, demand ransoms, or make smartphones unavailable. In response to that phenomenon, researchers have tackled the question of determining whether an application is malicious or benign and the question of classifying an application inside malware families. Most of the work in this area follows an experimental approach, and the proposed solutions must be evaluated on datasets mirroring the possible set of Android applications.

Over time, some datasets become extensively used and at some point, become outdated even if they are the latest to provide good information [1]–[3].

Results are often impressive, with detection ratios greater than 99% for some articles [4]–[6]. However, for these results to have any practical significance, the evaluation methodology and the nature of the datasets should be thoroughly inspected. In particular, special care should be taken to ensure that the experimental datasets are representative of the overall population of Android applications in the wild. Pendlebury et al. [7] identified the problem of bias in test datasets used to evaluate machine learning classifiers, which leads to overestimating the performance of an algorithm compared to its use in real-life scenarios. To evaluate an algorithm’s performance on a population of applications, we need a dataset that is representative of the population and that is labeled with the sought answers. Such answers require manual analysis of each application which is often a non-trivial task.

This article presents four main contributions:

* A methodology for building, by random sampling, a non-labeled dataset that is representative of the considered population of Android applications;

* A debiasing algorithm that, given an initial labeled and non-representative dataset and a representative target dataset, outputs a labeled and representative dataset;

* An experimental study on different datasets that have been debiased.

* An evaluation of the usefulness of debiased datasets when classifying malware/goodware samples using machine learning algorithms.

The article is structured as follows. Section II provides background information on the various datasets available to the research community and the type of experiments performed with them. Section III describes related work on the mitigation of irregularities that affect datasets. Section IV introduces the notion of representativity we will be using in the rest of this article. Then, Section V gives a methodology to extract a representative dataset from a population, and Section VI proposes a method to evaluate whether a dataset is representative of a population. Section VII introduces an algorithm that starts from a labeled dataset and builds a representative one. Section VIII experiments this algorithm on benign and malware datasets.

## II. DATASETS FOR EXPERIMENTS

To perform analyses on Android applications, researchers use published datasets as well as online repositories. To clarify

Manuscript received June 23, 2021; revised November 22, 2021 and March 15, 2022; accepted May 18, 2022. Date of publication June 3, 2022; date of current version June 15, 2022. The associate editor coordinating the review of this manuscript and approving it for publication was Dr. Chunyi Peng. (Corresponding author: Jean-François Lalande.)

The authors are with the CentraleSupélec, Inria, CNRS, University of Rennes 1, IRISA, 35000 Rennes, France (e-mail: jean-francois.lalande@irisa.fr).

Digital Object Identifier 10.1109/TIFS.2022.3180184
this landscape, we discuss the different datasets and categorize
their corresponding experimental scenarios.

A. *Sources, Archives and Specific Datasets*

Websites called “markets” are the different Android applica-
tion distribution platforms available to the public, e.g., Google
Play, Aptoide, Cafe Bazaar, F-Droid, MiKandi, Appland.
Google Play, the first and largest of them, is supposed to
be a source of new benign applications (goodware). Indeed,
Google uses services such as the bounc器<fnref n="1" /> and Google Play
Protect,<fnref n="2" /> which analyze applications before publishing them.
There are also country-specific markets, particularly several
Chinese markets (e.g., Baidu, Tencent, 360, 25PP, HiAPK,
Wandoujia), and vendor-specific markets (e.g., Huawei, OPPO,
Xiaomi, Lenovo). These markets evolve through time, as new
applications get published and old ones are withdrawn.

There are also archives datasets where applications are
never removed. AndroZoo [8], APKMirror<fnref n="3" /> and APKPure,<fnref n="4" />
are examples of archives.

Lastly, specific datasets provide a variety of applications that
share a common characteristic, for example, being a malware,
being repackaged applications, or being a specific type of
malware (ransomware, dropper, etc.). Genome [2], Drebin [1],
Andrubis [9], Kharon [10], AMD [3], CICAndMal2017 [11],
VirusShare [12], UpDroid [13], Koodous [14], and VirusTo-
tal [15] are examples of specific datasets (and mainly malware
datasets). Some datasets are mostly static, meaning they are
not regularly updated and therefore become rapidly obsolete.

B. *Experimental Scenarios*

We have identified three types of scenarios typically expe-
rienced by security researchers. Each of these scenarios has
different issues and therefore requires the use of datasets with
different expected properties.

1) *Markets Profiling:* Researchers may want to estimate a statistical characteristic of a market [8], [16], [17], e.g., “What proportion of Google Play applications have an API version lower than some threshold?.” For this case, one needs to obtain a dataset which is *representative* of the market, where results of analysis on this dataset can be extrapolated to the whole market. Section V gives more details on how such a representative dataset should be extracted.

2) *Algorithm Evaluation:* Some experiments aim to compute a characteristic of interest from an application. Examples of interesting characteristics include detecting malicious behavior in an application [18], whether the sample is repackaged [19], identifying its malware family [20], detecting similar applications [21], whether it leaks personal information to the network [22]. To answer these questions, researchers implement algorithms using various analysis techniques, and run them on a dataset to estimate their precision. This dataset should be representative and *labeled*, i.e., the characteristic of interest for each application should be known beforehand.

3) *Machine Learning:* In supervised learning methods, researchers learn a model that classifies inputs into a set of classes. Two datasets are used in this setting: a training dataset and a test dataset. The training set is used during the *learning* phase, and the test set is used to evaluate the learned model on new data. Both datasets should be labeled. The training set needs to be selected carefully to avoid biases, as discussed in more detail in Section III. The test set should be representative, as for the *Algorithm evaluation* scenario.

For any of the three kinds of experiments above, building
an appropriate dataset is a non-trivial task. In the rest of the
paper we take as hypothesis that a researcher needs a dataset
for evaluating a detection algorithm.

III. RELATED WORK

The main irregularities in experimental datasets have
already been studied [7], [23]–[25] and are listed below.
We reviewed the recent literature related to Android malware
experiments and confirm that these irregularities are still
present. Detailed results are reported in Appendix C.

A. *Class Imbalance*

Class imbalance occurs when a dataset contains significantly
more units of one class than of others [23]. An overrepresented
class may push a learning algorithm towards preferring this
class. For malware detection, if there are more samples of
goodware, this has the effect of decreasing the recall but
improving the precision [7].

This problem has been addressed in the machine learning
community [24]: the overrepresented class is undersampled,
i.e., some elements are removed while the underrepresented
class is oversampled. The undersampling can be random or
rely on a more sophisticated method, such as removing ele-
ments in dense clusters [26]. The most popular oversampling
technique is SMOTE [27] that creates new instances from the
interpolation of close samples. This technique cannot be used
for malware datasets, as one cannot forge a new malware
sample with specific characteristics automatically. Besides,
class balancing is mostly restricted to binary classes [24],
while we are interested in balancing more diverse classes.

B. *Presence of Clones*

Another irregularity is the presence of clones, where the
abundance of several very similar apps in the training set pre-
vents the algorithm from learning unique units [25]. Because
experiments do not usually share the list of used applications,
it is not possible to check the presence of clones in their
datasets. Pan et al. [28] show a method to reduce redundancy
in a dataset by extracting *representatives* of cloned units using
mutual information and relative entropy.

C. *Time Incoherence*

This irregularity concerns the dates of the samples in the
test and learning sets used in machine learning algorithms.
It was shown that, when evaluating the accuracy of state-of-
the-art algorithms, using a test set of applications newer than

<footnote n="1">¹http://googlemobile.blogspot.com/2012/02/android-and-security.html</footnote>

<footnote n="2">²https://www.blog.google/products/android/google-play-protect/</footnote>

<footnote n="3">³https://www.apkmirror.com/</footnote>

<footnote n="4">⁴https://apkpure.com</footnote>
those in the training set, the performance of these algorithms decreases [7], [29]. In [30], [31] and [32], Cai states that the main reason that existing Android malware detection techniques do not sustain well lies in their failure to account for the evolutionary dynamics of the Android ecosystem. Android malware classifiers may not be sustainable – they would need to be retrained constantly for later use, or their performance could downgrade over time. A regular re-training can mitigate this problem, however, it is not always a solution, as new samples are not always available as quickly as needed or in sufficient diversity. In [33], Zhang et al. observes that malware samples often implement the same functionalities over time but change their implementation because of the evolution of APIs or changes of third-party libraries. These changes allow such malware to avoid detection by older classifiers. One way to limit this problem is to choose more semantic than syntactic features, as stated by Zhang et al. in [33]. Finally, in [34], Xu et al. propose to enhance malware detection over time by making necessary updates to their detection models with evolving feature sets. For that purpose they maintain a pool of different detection models initialized with a set of labeled applications using various online learning algorithms.

The previously cited works cannot solve the problem of researchers having to evaluate their algorithm on an up-to-date dataset. Indeed, this problem is different than adapting a classifier to an evolving market – even if this problem is of independent interest. The researcher would primarily focus on getting a dataset that mimics the whole set of applications that he targets. Additionally, the dataset should be labeled, for example with a malware/goodware label if the study concerns a detection algorithm. These labels can be missing in the source from which applications are picked. As a consequence, we propose in the following to define the notion of representativity of a dataset against the population it represents. A representative dataset has to be used to validate an experimental results at the time of the experiment. The representativity property is linked to a population at a period of time. A dataset that is representative at a date will not necessarily be so in the future because the population may change significantly. Such a representative dataset will help the researcher to assess results for real-world scenarios [35].

## IV. REPRESENTATIVITY

In this section, we describe the notion of representativity of a population that we will be aiming for in the rest of this article.

The notion of population refers to a set of applications that a user would want to consider for his use case. As an example, a user who wants to detect malware in the Google Play store would want to take the Google Play store as population. Nevertheless, it is very difficult to get a snapshot of such a market. Other markets of applications could be considered [17] but the same problem remains. As a consequence we chose in this paper AndroZoo as population. It is not an hypothesis that reduces the contributions of this paper: the proposed techniques work more generally beyond a particular reference dataset.

From a general perspective, we consider a population $\mathcal{P}$ of $N$ applications. Intuitively, a dataset $\mathbf{D}$ of $n$ elements is representative of $\mathcal{P}$ if $n$ is significantly smaller than $N$ and if $\mathbf{D}$ looks like $\mathcal{P}$ for the following purposes: 1) evaluating the proportion $\hat{p}$ of applications exhibiting a specific characteristic on $\mathbf{D}$ should yield similar results as evaluating the same proportion $p$ on $\mathcal{P}$; 2) evaluating an algorithm on $\mathbf{D}$ should yield similar results as evaluating the same algorithm on $\mathcal{P}$.

Intuitively, we could consider that a dataset $\mathbf{D}$ is representative of $\mathcal{P}$ according to a single characteristic if the proportion of this characteristic is equal in $\mathcal{P}$ and $\mathbf{D}$. However, the exact equality of the proportions in $\mathcal{P}$ and $\mathbf{D}$ (i.e., $p = \hat{p}$) cannot always be achieved in practice. For example, if $p = 1/N$ (i.e., only one element has this characteristic in $\mathcal{P}$), then the only way to have exactly the same proportion in $\mathbf{D}$ is by selecting all $N$ elements from $\mathcal{P}$. This is in contradiction with the first desired property of a representative dataset. Hence a margin of error $\delta$ must be taken into account, which is the maximum difference between $p$ and $\hat{p}$ that we allow considering that $\mathbf{D}$ is representative of $\mathcal{P}$. Intuitively, a dataset is representative of a population if it has the same proportions, up to a maximum difference of $\delta$.

This notion can be easily extended to multiple characteristics. Consider a set of $d$ characteristics $\{c_1, \dots, c_d\}$ that can take respectively $v_1, \dots, v_d$ values. This generates $v_1 \times \dots \times v_d$ different combinations. In the following, we call these combinations *classes*, and we denote $\mathcal{K}$ this set of classes. Intuitively again, we could consider that a dataset $\mathbf{D}$ is representative of $\mathcal{P}$ according to a set of classes $\mathcal{K}$ if the proportion of each class $k \in \mathcal{K}$ is equal in $\mathcal{P}$ and $\mathbf{D}$.

**Definition 1:** Let $0 \le \delta \le 1$. A dataset $\mathbf{D}$ is $\delta$-representative of a population $\mathcal{P}$ for a finite set of classes $\mathcal{K}$ if $\forall k \in \mathcal{K}, |p_{\mathbf{D},k} - p_{\mathcal{P},k}| \le \delta$, where $p_{\mathbf{D},k}$ (resp. $p_{\mathcal{P},k}$) is the proportion of class $k$ in $\mathbf{D}$ (resp. $\mathcal{P}$).

Characteristics can represent various properties of an application. For example, file size, number of classes in the code or whether the application reads SMS messages are examples of characteristics. Note that a class is simply a set of applications sharing the same set of low-level characteristics, and is not related with the notion of malware families.

## V. SAMPLING REPRESENTATIVE DATASETS

In this article, we consider the population of Android applications to be AndroZoo, because this is one of the largest datasets available, it is regularly updated from other markets and malware databases, and no application is removed from it. At the end of 2020, AndroZoo contained 13 696 936 different APK. In this section, we focus on the construction of a non-labeled $\delta$-representative dataset of AndroZoo by sampling, i.e., random drawing.

The standard way to construct a representative dataset consists of drawing $n$ random samples without replacement [36]. Because elements are drawn randomly, a very large fraction of the population would be necessary to ensure, with 100% confidence, an error smaller than $\delta$. In statistics, one is generally more interested in probabilistic guarantees such as “with 95% chance, the error is smaller than $\delta$.” Intuitively, the
more certainty one seeks, the more elements should be drawn from P. This confidence level is usually denoted C.

We recall standard statistical equations to estimate the minimal size n to ensure the randomly drawn dataset D is representative of the population. We show that the minimum size of a representative dataset depends primarily on the number of characteristics for which the dataset must be representative.

### A. Dataset Size for a Single Boolean Characteristic

Once the expected margin of error δ and the expected confidence level C are defined (according to the requirements of the experiment), a lower bound on the size n of the dataset D can be calculated by using Equation (1). This equation is obtained by rearranging the margin of the classical error formula [37] using the finite population correction [36], where z(C) is the standard deviation value at C confidence level for the standard normal distribution with mean 0 and standard deviation of 1 (the derivation of this equation is detailed in the appendix).

$$n \geq N \left( \frac{4(N-1)\delta^2}{z(C)^2} + 1 \right)^{-1} \quad (1)$$

This computation is based on some light assumptions, notably that n is not too small (at least 30 [38]) and that the proportion p is not very close to 0 or 1. In such cases, other confidence interval formulas can be used, like the Hoeffding inequality [39].

The effect of N is limited when it is large. The minimum size of a random sample for 3 014 560 applications in Google Play with the values δ = 0.01 and C = 99% is n = 16 497 while it is n = 16 568 for AndroZoo, a dataset four times larger. For arbitrarily large datasets, n = 16 588 examples are sufficient. The influence of N is more significant for smaller populations. For example, with a population of N = 15 000 elements, one only needs n = 7163 examples.

### B. Dataset Size for one or Multiple Characteristics

The calculation of the size of the dataset given in the previous subsection only holds for a single boolean characteristic, so when there are only two classes. If one is interested in several characteristics or in a non-boolean characteristic, then there are more classes for the dataset to be representative of, and intuitively it means that the minimal size of a dataset representative of a population is larger.

As with a single boolean characteristics, the margin of error can be used to get a sample with an error up to δ. The Bonferroni correction [40] states that correctly representing a set of classes K with a confidence level C using random sampling is more likely than correctly representing one boolean characteristic with a confidence level $1 - (1 - C)/(|K| - 1)$. Therefore, we can modify Equation (1) by replacing z(C) by $z(1 - \frac{1-C}{|K|-1})$. Hence, the dataset D will correctly represent $|K|$ classes with a margin error δ with a probability higher than C if n satisfies the following inequation:

$$n \geq N \left( \frac{4(N-1)\delta^2}{z \left(1 - \frac{1-C}{|K|-1}\right)^2} + 1 \right)^{-1} \quad (2)$$

Remark that, in the case of a single boolean characteristic, $|K| = 2$ and equations (1) and (2) are equal.

In the experiments of Section VI, we will observe 2411 non-empty classes in AZ2030k, the reference dataset. With the values δ = 0.015, C = 99% and $|K| = 36 864$, the minimum size of a random sample is n = 29 099 for Google Play, n = 29 320 for AndroZoo and n = 29 383 for datasets with arbitrarily large N.

## VI. EVALUATION OF EXISTING DATASETS

We now assume that an algorithm or a machine learning model evaluation has to be performed, as described in Section II-B. Such evaluation needs a labeled dataset, i.e., one whose ground truth is known. It should also be representative of the entire population. Otherwise, results could be erroneous due to biases in the dataset. In the case of Android applications, some characteristics like “being malware” are not easily computable, and only small datasets or old datasets contain such information.

For a given hard-to-compute characteristic, we separate the infinite number of other characteristics into two groups. In the first group, the characteristics that are likely to be related to the hard-to-compute characteristic. For example, using a cryptographic library could be related to being a ransomware or not. In the second group, the characteristics are not related to this hard-to-compute characteristic. We would like the datasets used to be homogeneous to the population for the characteristics of this second group.

We now propose a homogeneity test to assess if a dataset is homogeneous to a population for a finite set of characteristics. To determine this finite set, we consider the hard-to-compute characteristic “being a malware” and we choose a set of characteristics that are not related to it. Then, we test the homogeneity of existing datasets with the population for this set.

### A. Choosing Characteristics

The more characteristics are selected, the more combinations there are. Since this number of combinations is exponential in the number of characteristics, it can quickly grow bigger than any dataset size as the number of selected characteristics increases. Therefore, it is necessary to limit the number of characteristics. Besides, numerical characteristics should be reduced to a few values to reduce this combination's number. For example, the APK size can be projected to a few intervals of size to distinguish small, medium, and large applications.

As a consequence, in the following experiments, we choose to rely on the following set of characteristics that are not related to security:

*   APK size (4 values)

0) 1 MB < size

1) 1 MB ≤ size ≤ 5 MB

2) 5 MB ≤ size ≤ 20 MB

3) size > 20 MB

*   year (9 values): { < 2011, 2011-2012, 2012-2013, 2013-2014, 2014-2015, 2015-2016, 2016-2017, 2017-2018, > 2018}
* Internet permission (bool)

* External or internal storage access (bool)

* Uses Google play services (bool)

* Generates UUIDs (bool)

* Vibrate phone permission (bool)

* NFC permission (bool)

* Bluetooth permission (bool)

* Performs HTTP request (bool)

* Uses JSON objects (bool)

* Specifies User-Agent (bool)

These characteristics can be easily extracted using Droidlysis [41],<fnref n="5" /> a property extraction tool. With such characteristics, 36 864 (4 × 9 × 2¹⁰) classes would be possible, but in practice, we observe 2874 non-empty classes in the three studied datasets (AZ20₃₀k, Dmix and AMD) presented later in Figure 1.

These characteristics are diversified and easy to evaluate statically, limiting the computational cost of our experiment. It would be possible to add some dynamic characteristics, such as the number of executed activities or methods, but it would require executing each application. These characteristics will be used to study the homogeneity of the datasets of the literature against an extract of the population, AndroZoo in 2020.

## B. Statistical Test of Homogeneity for Datasets

To evaluate whether a dataset **D** can be considered to be not statistically homogeneous to a population *P* for a finite number of characteristics, we propose to use the $\chi^2$ statistical test of homogeneity [42] and compare **D** with a second dataset $\mathbf{D}_r$, a large dataset randomly drawn from *P* (see Section V). The null hypothesis of the test is $H_0: \mathbf{D}$ and $\mathbf{D}_r$ are drawn from the same population *P* (with replacement). As other statistical tests, the $\chi^2$ test can only reject this null hypothesis but not confirm it. If the null hypothesis is rejected, we will admit that the dataset **D** is not homogeneous to $\mathbf{D}_r$, and therefore dissimilar from the population *P* for the characteristics under consideration.

To apply this test, we consider a finite set of classes $\mathcal{K}$. We denote $m_{\mathbf{D},k}$ (respectively $m_{\mathbf{D}_r,k}$) the number of samples in **D** (respectively in $\mathbf{D}_r$) of class $k \in \mathcal{K}$ and $p_{\mathcal{P},k}$ (respectively $p_{\mathbf{D},k}$ and $p_{\mathbf{D}_r,k}$) the proportion of class k in *P* (respectively in **D**, in $\mathbf{D}_r$).

If the null hypothesis holds true, then the three proportions $p_{\mathcal{P},k}$, $p_{\mathbf{D},k}$ and $p_{\mathbf{D}_r,k}$ of class $k \in \mathcal{K}$ should be close. The proportion $p_{\mathcal{P},k}$ can thus be estimated by $\frac{m_{\mathbf{D},k}+m_{\mathbf{D}_r,k}}{|\mathbf{D}_r|+|\mathbf{D}|}$ and the expected number of occurrences of class k in **D** can be estimated by $E_{\mathbf{D},k,\mathbf{D}_r} = |\mathbf{D}| \times \frac{m_{\mathbf{D},k}+m_{\mathbf{D}_r,k}}{|\mathbf{D}_r|+|\mathbf{D}|}$ and in $\mathbf{D}_r$ by $E_{\mathbf{D},k,\mathbf{D}} = |\mathbf{D}_r| \times \frac{m_{\mathbf{D},k}+m_{\mathbf{D}_r,k}}{|\mathbf{D}_r|+|\mathbf{D}|}$. The $\chi^2$ value of two datasets **D** and $\mathbf{D}_r$ is defined as:

$$ \chi^2 = \sum_{k \in \mathcal{K}} \frac{(m_{\mathbf{D},k} - E_{\mathbf{D},k,\mathbf{D}_r})^2}{E_{\mathbf{D},k,\mathbf{D}_r}} + \frac{(m_{\mathbf{D}_r,k} - E_{\mathbf{D},k,\mathbf{D}})^2}{E_{\mathbf{D},k,\mathbf{D}}} $$

A low value for a $\chi^2$ test means the two sets of data **D** and $\mathbf{D}_r$ have very similar class proportions. On the contrary, a large

value (larger than a critical value that depends on the number of classes and the confidence level) indicates a statistically significant difference. A $\chi^2$ test is associated with a p-value. A low p-value (usually under 0.05) indicates that one can reject the null hypothesis with a certain confidence. A high p-value, however, does not confirm the null hypothesis, but rather produces an inconclusive result.

In the rest of this section, we investigate whether labeled datasets commonly used in the literature can be considered homogeneous to all Android applications according to the $\chi^2$ test and whether they can be considered $\delta$-representative for some low $\delta$.

## C. Studied Android Datasets

In this section, we focus on the following datasets:

* 6 malware datasets: Drebin [1], AMD [3], as well as VS 2015, 2016, 2017 and 2018, containing all the malware collected by VirusShare for each of these years;

* 2 groups of datasets extracted from AndroZoo [8]:

- AZ19₁₀₀k, 100 000 applications randomly drawn in 2019, and the subsets AZ19₁₀₀k 2015, 2016, 2017, 2018 restricted to applications from each year;

- AZ20₁₀k, AZ20₂₀k, and AZ20₃₀k three datasets randomly drawn from AndroZoo in 2020, containing respectively 10 000, 20 000 and 30 000 applications.

* 1 mix dataset: Dmix composed of 5560 malware from Drebin and 11 120 goodware from 2018 extracted from AndroZoo. Dmix mimics the datasets used in previous machine learning papers [43], [44].

In this article, we use AndroZoo as the population, because it is the easiest source to draw a large number of applications. Our goal is to determine which datasets commonly used in the literature can be considered as representative of the population. AndroZoo provides 13 696 936 different APKs applications. Due to its size, it is not reasonable to calculate exactly the distribution of AndroZoo. Thus, we represent here AndroZoo through a representative dataset AZ20₃₀k of 30 000 randomly chosen from it (corresponding to $\delta = 0.015$ and $C = 99\%$ for all 36 864 theoretical classes, cf. Section V-B).

We performed the comparison of AZ20₃₀k with the datasets of the literature, presented in Section II. As an illustration, Figure 1 shows the proportion of applications in each of the 2874 non-empty classes of three datasets: the reference AZ20₃₀k (in blue), the mix dataset Dmix (in orange) and AMD (in green). The classes are sorted in descending order of size for AZ20₃₀k. This figure already suggests that the three datasets are very different. We confirm this intuition by performing a $\chi^2$ test with the AZ20₃₀k dataset. Table I presents the dataset size, the number of classes represented only in AZ20₃₀k (and not in the studied dataset), the number of classes common to AZ20₃₀k and the studied dataset, the number of classes represented only in the studied dataset (and not in AZ20₃₀k), the maximum and average $\delta$ of class proportions, the value of the $\chi^2$ test, the associated p-value, and the conclusion (or lack of) of the test between the studied dataset and AZ20₃₀k. All datasets, except the last extract of AndroZoo, can be considered as not homogeneous to AZ20₃₀k,

<footnote n="5">⁵https://github.com/cryptax/droidlysis</footnote>
<figure><img src="image_2.png" /><figcaption>Fig. 1. Distribution of application classes for the top most 200 over 2874 classes of AZ2030k and compared with Dmix and AMD.</figcaption></figure>

<table><caption>TABLE I
$\chi^2$ TEST FOR HOMOGENEITY OVER AZ2030k FOR A FINITE SET OF NON-SECURITY CHARACTERISTICS</caption><thead><tr><th>Datasets</th><th>Dataset size</th><th># of classes in common</th><th># of classes only in the dataset</th><th># of classes only in AZ2030k</th><th>Max δ</th><th>Avg δ</th><th>χ<sup>2</sup> value</th><th>p-value</th><th>Conclusion</th></tr></thead><tbody><tr><td>AZ2030k</td><td>29 974</td><td>2 411</td><td>-</td><td>-</td><td>0</td><td>0</td><td>0</td><td>1</td><td></td></tr><tr><td>Drebin</td><td>5 304</td><td>181</td><td>141</td><td>2 230</td><td>0.1184</td><td>0.0007</td><td>27 902</td><td>0</td><td>Reject (C=99%)</td></tr><tr><td>AMD</td><td>23 258</td><td>128</td><td>16</td><td>2 283</td><td>0.0974</td><td>0.0007</td><td>39 771</td><td>0</td><td>Reject (C=99%)</td></tr><tr><td>VirusShare 2015</td><td>28 896</td><td>250</td><td>65</td><td>2 161</td><td>0.1683</td><td>0.0007</td><td>44 537</td><td>0</td><td>Reject (C=99%)</td></tr><tr><td>VirusShare 2016</td><td>12 651</td><td>172</td><td>28</td><td>2 239</td><td>0.1595</td><td>0.0007</td><td>30 312</td><td>0</td><td>Reject (C=99%)</td></tr><tr><td>VirusShare 2017</td><td>9 945</td><td>234</td><td>39</td><td>2 177</td><td>0.0898</td><td>0.0006</td><td>22 050</td><td>0</td><td>Reject (C=99%)</td></tr><tr><td>VirusShare 2018</td><td>28 543</td><td>728</td><td>328</td><td>1 683</td><td>0.5057</td><td>0.0006</td><td>42 617</td><td>0</td><td>Reject (C=99%)</td></tr><tr><td>Dmix</td><td>16 424</td><td>982</td><td>447</td><td>1 429</td><td>0.0359</td><td>0.0004</td><td>17 852</td><td>0</td><td>Reject (C=99%)</td></tr><tr><td>AZ19100k</td><td>99 999</td><td>2 086</td><td>2 062</td><td>325</td><td>0.0320</td><td>0.0001</td><td>12 662</td><td>0</td><td>Reject (C=99%)</td></tr><tr><td>DroidBench</td><td>119</td><td>6</td><td>0</td><td>2 405</td><td>0.4773</td><td>0.0008</td><td>14 121</td><td>0</td><td>Reject (C=99%)</td></tr><tr><td>AZ19100k 2015</td><td>5 794</td><td>648</td><td>291</td><td>1 763</td><td>0.0574</td><td>0.0005</td><td>15 390</td><td>0</td><td>Reject (C=99%)</td></tr><tr><td>AZ19100k 2016</td><td>27 516</td><td>1 183</td><td>921</td><td>1 228</td><td>0.0573</td><td>0.0004</td><td>30 725</td><td>0</td><td>Reject (C=99%)</td></tr><tr><td>AZ19100k 2017</td><td>6 524</td><td>688</td><td>274</td><td>1 723</td><td>0.0381</td><td>0.0004</td><td>12 378</td><td>0</td><td>Reject (C=99%)</td></tr><tr><td>AZ19100k 2018</td><td>24 549</td><td>1 216</td><td>547</td><td>1 195</td><td>0.0285</td><td>0.0003</td><td>14 754</td><td>0</td><td>Reject (C=99%)</td></tr><tr><td>AZ2010k</td><td>9 971</td><td>1 370</td><td>423</td><td>1 041</td><td>0.0443</td><td>0.0003</td><td>6 296</td><td>0</td><td>Reject (C=99%)</td></tr><tr><td>AZ2020k</td><td>19 927</td><td>1 361</td><td>397</td><td>1 050</td><td>0.0245</td><td>0.0001</td><td>3 560</td><td>1</td><td>Non-conclusive</td></tr></tbody></table>

rejecting the null hypothesis with a $p$-value < 0.05. We can also remark that extracts of AndroZoo have a low average $\delta$ value, smaller that 0.001, while non-homogeneous datasets have significantly larger average $\delta$ values. This observation is consistent with our definition of representativity, as we can see a link between a low average $\delta$ value and a high $p$-value where the null hypothesis is not rejected.

## VII. DATASET DEBIASING ALGORITHM

The previous section strongly suggests that the datasets regularly used in Android malware scanning experiments are not representative of the entire current population of Android applications. Definitely, they are not for the common characteristics we have studied. Nevertheless, these datasets are valuable because they contain samples for which the characteristic of interest is already computed. We are now interested in producing a representative dataset of a population for a finite set of classes $\mathcal{K}$ from an initial dataset **B**. Our motivation is to not have to recompute computationally expensive characteristics on a randomly drawn dataset but to take maximum advantage of **B**'s labels.

As seen in the previous experiments, the $\chi^2$ test can show that datasets deviate significantly from a population. However, this test cannot conclude that a dataset is representative of a

population, only that it is not homogeneous. Besides, the $\chi^2$ value has no intuitive meaning, and the $p$-value is notoriously prone to cause interpretation error while the maximal error $\delta$ is easier to interpret. For these reasons, we will use the $\delta$-representativity when debiasing datasets.

We propose here two algorithms to limit the bias of an initial dataset **B**, while minimizing the number of modifications. Algorithm 1 constructs a dataset of the same size as **B**. This can lead to suboptimal solutions because the size of the optimal solution and **B** may differ. Algorithm 2 allows the size of the generated dataset to vary. Besides, we prove that this algorithm is optimal in the number of modifications to **B**. These two algorithms take as input an initial base dataset **B**, a target dataset **T** which we would like **B** to resemble, a source labeled dataset **S** from which we will draw applications, and a positive ratio of tolerated difference $\delta$. The objective of both algorithms is to produce a generated dataset **G** which is $\delta$-representative of **T**. We say that a class $k$ is overrepresented in **G** if $p_{G,k} - p_{T,k} > \delta$; and a class $k$ is underrepresented in **G** if $p_{G,k} - p_{T,k} < -\delta$.

### A. Constant-Size Debiasing Algorithm

Algorithm 1 produces a dataset **G** whose size is the same as its input **B**. Intuitively, this algorithm consists in adding
elements from the source **S** to the most underrepresented classes of **B** and to remove elements from **B** for the most overrepresented class. Each time an element is added, another element is removed in order to keep a constant size. With the constraint of constant size, the debiasing may be impossible for one of two reasons: either no element can be added to an underrepresented class (because the source does not contain elements of this class); or it is not possible to obtain for some class *k* a proportion that lies in $[p_{T,k} - \delta; p_{T,k} + \delta]$ because $\delta$ is too small.

<table><caption>**Algorithm 1 Constant Size Debiasing Algorithm.**</caption><thead><tr><th></th><th>Input: <b>B</b>, <b>T</b>, <b>S</b>, <i>&delta;</i></th><th></th></tr></thead><tbody><tr><td>1</td><td><b>S'</b> &larr; <b>S</b>; <b>G</b> &larr; <b>B</b></td><td></td></tr><tr><td>2</td><td>while max<sub>k&isin;&mathcal;K</sub> |<b>p</b><sub><b>G</b>,<b>k</b></sub> &minus; <b>p</b><sub><b>T</b>,<b>k</b></sub>| &gt; <i>&delta;</i> do</td><td></td></tr><tr><td>3</td><td>k<sub><b>h</b></sub> &larr; arg max<sub>k&isin;&mathcal;K</sub>(<b>p</b><sub><b>G</b>,<b>k</b></sub> &minus; <b>p</b><sub><b>T</b>,<b>k</b></sub>)</td><td>&larr; most</td></tr><tr><td>4</td><td>k<sub><b>l</b></sub> &larr; arg min<sub>k&isin;&mathcal;K</sub>(<b>p</b><sub><b>G</b>,<b>k</b></sub> &minus; <b>p</b><sub><b>T</b>,<b>k</b></sub>)</td><td>&larr; most</td></tr><tr><td>5</td><td>underrepresented class</td><td></td></tr><tr><td>6</td><td>if |<b>S'</b>(<b>k</b><sub><b>l</b></sub>)| = 0 then</td><td></td></tr><tr><td>7</td><td>&nbsp;&nbsp;&nbsp;&nbsp;<b>K</b><sub><b>S</b></sub> = {<b>k</b> &isin; &mathcal;K | |<b>S'</b>(<b>k</b>)| &gt; 0}</td><td></td></tr><tr><td>8</td><td>if <b>p</b><sub><b>G</b>,<b>k</b><sub><b>l</b></sub></sub> &ge; <b>p</b><sub><b>T</b>,<b>k</b><sub><b>l</b></sub></sub> &minus; <i>&delta;</i> and <b>K</b><sub><b>S</b></sub> &ne; &empty;</td><td>then</td></tr><tr><td>9</td><td>k<sub><b>l</b></sub> &larr; arg min<sub>k&isin;&mathcal;K</sub><b>p</b><sub><b>G</b>,<b>k</b></sub> <b>p</b><sub><b>T</b>,<b>k</b></sub></td><td></td></tr><tr><td>10</td><td>else return "Impossible";</td><td></td></tr><tr><td>11</td><td><b>r</b> &larr; one element of <b>S'</b>(<b>k</b><sub><b>l</b></sub>) chosen at random</td><td></td></tr><tr><td>12</td><td>Remove <b>r</b> from <b>S'</b> and add it to <b>G</b></td><td></td></tr><tr><td>13</td><td>Remove one element of <b>G</b>(<b>k</b><sub><b>h</b></sub>) chosen at random</td><td></td></tr><tr><td>14</td><td>if <b>p</b><sub><b>G</b>,<b>k</b><sub><b>h</b></sub></sub> &lt; <b>p</b><sub><b>T</b>,<b>k</b><sub><b>h</b></sub></sub> &minus; <i>&delta;</i> or <b>p</b><sub><b>G</b>,<b>k</b><sub><b>l</b></sub></sub> &gt; <b>p</b><sub><b>T</b>,<b>k</b><sub><b>l</b></sub></sub> + <i>&delta;</i> then</td><td>return "Impossible"</td></tr><tr><td>15</td><td>return <b>G</b></td><td></td></tr></tbody></table>

Algorithm 1 is optimal in terms of number of modifications.<fnref n="6" /> If a class is underrepresented in **B**, we need to add elements to increase its proportion and make it correctly represented. If $a_k$ elements are added to this class while the size of **D** remains constant (i.e. $|\mathbf{D}| = |\mathbf{B}|$), the proportion of class *k* in **D** is $p_{\mathbf{B},k} + \frac{a_k}{|\mathbf{B}|}$. A class is not underrepresented if its proportion is greater than $p_{\mathbf{T},k} - \delta$, so we are looking for the smallest $a_k$ such that $p_{\mathbf{B},k} + \frac{a_k}{|\mathbf{B}|} \ge p_{\mathbf{T},k} - \delta$, hence $a_k \ge |\mathbf{B}|(p_{\mathbf{T},k} - \delta - p_{\mathbf{B},k})$. By definition of the ceiling function $\lceil \cdot \rceil$, $\lceil |\mathbf{B}|(p_{\mathbf{T},k} - \delta - p_{\mathbf{B},k}) \rceil$ is the lowest integer that verifies this inequality. So the minimal number of addition for an underrepresented class *k* is $\lceil |\mathbf{B}|(p_{\mathbf{T},k} - \delta - p_{\mathbf{B},k}) \rceil$. If this number is negative, it means that the class is not underrepresented: in that case, no addition should be made. We denote, for any class *k*, the minimal number of addition as $n_{\mathbf{B}}^-(k) = \max(0, \lceil |\mathbf{B}|(p_{\mathbf{T},k} - \delta - p_{\mathbf{B},k}) \rceil)$. By the same reasoning, we define the minimal number of deletions as $n_{\mathbf{B}}^+(k) = \max(0, \lceil |\mathbf{B}|(p_{\mathbf{B},k} - p_{\mathbf{T},k} - \delta) \rceil)$. In fact, we prove that Algorithm 1 makes exactly $2 \max(\sum_k n_{\mathbf{B}}^+(k), \sum_k n_{\mathbf{B}}^-(k))$ modifications between its input **B** and its output **G**.

## B. Optimal Debiasing Algorithm

Algorithm 1 can lead to suboptimal solutions. Algorithm 2 produces a dataset **G** without size constraint which is representative of **T**, with minimal modifications to **B**. This algorithm calls Algorithm 1 multiple times with different dataset sizes and maintains two variables: $\mathbf{G}_{best}$ is the best dataset found so far, i.e., the one that minimizes the number of modifications to **B**; and $d_{min}$ is this distance between $\mathbf{G}_{best}$ and **B**. The algorithm starts with $\mathbf{G}_{best} = "Impossible"$ and $d_{min} = \infty$ (line 1). First, it calls Algorithm 1 (line 2) with the initial dataset **B**. If this succeeds, $d_{min}$ is updated with the appropriate number of additions and deletions, and $\mathbf{G}_{best}$ becomes the result of Algorithm 1. Then, in lines 6 to 11, it tries to remove applications from the most overrepresented classes from **B**. Finally, in lines 14 to 14, it tries to add applications to the most underrepresented classes to **B**. For each modification to **B**, Algorithm 1 is called to update $d_{min}$ and $\mathbf{G}_{best}$.

<table><caption>**Algorithm 2 Optimal debiasing Algorithm**</caption><thead><tr><th></th><th>Input: <b>B</b>, <b>T</b>, <b>S</b>, <i>&delta;</i></th><th></th></tr></thead><tbody><tr><td>1</td><td><b>d</b><sub>min</sub> &larr; &infin;; <b>G</b><sub>best</sub> &larr; "Impossible"</td><td></td></tr><tr><td>2</td><td>if Algo1(<b>B</b>, <b>T</b>, <b>S</b>, <i>&delta;</i>) &ne; "Impossible" then</td><td></td></tr><tr><td>3</td><td><b>d</b><sub>min</sub> &larr; 2 max(&sum;<sub>k</sub> n<sub><b>B</b></sub><sup>+</sup>(<b>k</b>), &sum;<sub>k</sub> n<sub><b>B</b></sub><sup>-</sup>(<b>k</b>))</td><td></td></tr><tr><td>4</td><td><b>G</b><sub>best</sub> &larr; Algo1(<b>B</b>, <b>T</b>, <b>S</b>, <i>&delta;</i>)</td><td></td></tr><tr><td>5</td><td><b>G</b> &larr; <b>B</b></td><td></td></tr><tr><td>6</td><td>for <i>i</i> from 1 to <b>d</b><sub>min</sub> &minus; 1 do</td><td>&larr; Search smaller datasets</td></tr><tr><td>7</td><td>if <b>G</b> = &empty; then break;</td><td></td></tr><tr><td>8</td><td>k<sub><b>h</b></sub> &larr; arg max<sub>k&isin;&mathcal;K</sub>(<b>p</b><sub><b>G</b>,<b>k</b></sub> &minus; <b>p</b><sub><b>T</b>,<b>k</b></sub>)</td><td></td></tr><tr><td>9</td><td>Remove one random element of class k<sub><b>h</b></sub> from <b>G</b></td><td></td></tr><tr><td>10</td><td><b>d</b> &larr; |<b>B</b>| &minus; |<b>G</b>| + 2 max(&sum;<sub>k</sub> n<sub><b>G</b></sub><sup>+</sup>(<b>k</b>), &sum;<sub>k</sub> n<sub><b>G</b></sub><sup>-</sup>(<b>k</b>))</td><td></td></tr><tr><td>11</td><td>if <b>d</b> &lt; <b>d</b><sub>min</sub> and Algo1(<b>G</b>, <b>T</b>, <b>S</b>, <i>&delta;</i>) &ne; "Impossible" then</td><td></td></tr><tr><td>12</td><td><b>d</b><sub>min</sub> &larr; <b>d</b>; <b>G</b><sub>best</sub> &larr; Algo1(<b>G</b>, <b>T</b>, <b>S</b>, <i>&delta;</i>)</td><td></td></tr><tr><td>13</td><td><b>G</b> &larr; <b>B</b></td><td></td></tr><tr><td>14</td><td>for <i>i</i> from 1 to <b>d</b><sub>min</sub> &minus; 1 do</td><td>&larr; Search larger datasets</td></tr><tr><td>15</td><td>k<sub><b>l</b></sub> &larr; arg min<sub>k&isin;&mathcal;K</sub>(<b>p</b><sub><b>G</b>,<b>k</b></sub> &minus; <b>p</b><sub><b>B</b>,<b>k</b></sub>)</td><td></td></tr><tr><td>16</td><td>if |<b>S</b>(<b>k</b><sub><b>l</b></sub>)| = 0 then</td><td></td></tr><tr><td>17</td><td>&nbsp;&nbsp;&nbsp;&nbsp;<b>K</b><sub><b>S</b></sub> = {<b>k</b> &isin; &mathcal;K | |<b>S</b>(<b>k</b>)| &gt; 0}</td><td></td></tr><tr><td>18</td><td>if <b>p</b><sub><b>G</b>,<b>k</b><sub><b>l</b></sub></sub> &ge; <b>p</b><sub><b>T</b>,<b>k</b><sub><b>l</b></sub></sub> &minus; <i>&delta;</i> and <b>K</b><sub><b>S</b></sub> &ne; &empty;</td><td>then</td></tr><tr><td>19</td><td>k<sub><b>l</b></sub> &larr; arg min<sub>k&isin;&mathcal;K</sub><b>p</b><sub><b>G</b>,<b>k</b></sub> &minus; <b>p</b><sub><b>T</b>,<b>k</b></sub></td><td></td></tr><tr><td>20</td><td>else break;</td><td></td></tr><tr><td>21</td><td><b>r</b> &larr; one element of <b>S</b>(<b>k</b><sub><b>l</b></sub>) chosen at random</td><td></td></tr><tr><td>22</td><td>Remove <b>r</b> from <b>S</b> and add it to <b>G</b></td><td></td></tr><tr><td>23</td><td><b>d</b> &larr; |<b>B</b>| &minus; |<b>G</b>| + 2 max(&sum;<sub>k</sub> n<sub><b>G</b></sub><sup>+</sup>(<b>k</b>), &sum;<sub>k</sub> n<sub><b>G</b></sub><sup>-</sup>(<b>k</b>))</td><td></td></tr><tr><td>24</td><td>if <b>d</b> &lt; <b>d</b><sub>min</sub> and Algo1(<b>G</b>, <b>T</b>, <b>S</b>, <i>&delta;</i>) &ne; "Impossible" then</td><td></td></tr><tr><td>25</td><td><b>d</b><sub>min</sub> &larr; <b>d</b>; <b>G</b><sub>best</sub> &larr; Algo1(<b>G</b>, <b>T</b>, <b>S</b>, <i>&delta;</i>)</td><td></td></tr><tr><td>26</td><td>return <b>G</b><sub>best</sub></td><td></td></tr></tbody></table>

**Theorem 1:** Let **B**, **S**, **T** be three datasets and $\delta > 0$. If there exists at least one $\delta$-representative dataset of **T** that is composed of elements of **B** and **S**, then Algorithm 2 produces such a dataset **G** such that the number of additions and

<footnote n="6">⁶Proofs are available in the appendix.</footnote>
deletions from the initial dataset **B** is minimal. If such a dataset doesn't exist, it returns "Impossible."

Its worst-case temporal complexity is $O((1/\delta + |\mathbf{B}|)^2 |\mathcal{K}|)$.

While Algorithm 2 computes the closest dataset to the original one, it may happen that the solution found contains only a few applications from the original dataset. This is an indication that the original dataset was very biased.

If **B** has been debiased towards a target dataset **T** randomly drawn from a population $\mathcal{P}$, with a tolerated difference of $\delta_{\text{debiasing}}$, we can estimate its error with the population $\mathcal{P}$ by taking into account the error $\delta_{\text{sampling}}$ and confidence interval $C$ used to derive the size of **T** with Eq. (2): with confidence level $C$, the error between the debiased dataset and $\mathcal{P}$ is less than $\delta_{\text{sampling}} + \delta_{\text{debiasing}}$.

## VIII. EXPERIMENTS

In this last section, we are interested in producing datasets for experiments concerning Android malware detection or classification.<fnref n="7" /> As identified in Section II-B three types of experimental scenarios can occur. The first scenario (market profiling) requires only unlabeled representative data sets that can be drawn randomly without requiring debiasing (Section V). The two other cases, Algorithm evaluation and Machine learning, require labeled datasets with different proportions of malware/goodware samples. Therefore, we propose in Section VIII-A to debias datasets containing only malware or only goodware and mix them, in Section VIII-B, to obtain mixed datasets directly usable for machine learning algorithms.

### A. Debiasing Datasets

The following details the use of our algorithm to produce representative datasets and in particular the choice of the base, reference and source datasets. Through these objectives, we discuss three main questions:

* Is it always possible to debias a strongly biased dataset such as Drebin or VirusShare?

* How many modifications (del/add) are required?

* Is a debiased malware dataset stable over time?

1) **Debiasing 100% Goodware Datasets:** We have randomly drawn a New AndroZoo Extract (NAZE-18-G) and filtered it by relying on Virus Total to keep only goodware with a date no greater than 2018. This dataset should be close to a representative dataset of the Android app population in 2020. Indeed, by taking as reference dataset AZ2030k and as source dataset AZ19100kG, we successfully build a representative goodware dataset when $\delta$ reaches 0.001, as shown in Table II. Indeed, for $\delta = 0.04$ the dataset is already homogeneous: no addition/deletion is required, but the p-value is 0. Then, we decrease $\delta$ until the p-value becomes close to 1. For $\delta = 0.001$, 19.53% of the resulting dataset are new applications and more than 50% of applications have been removed from the original one. Remark that these values of $\delta$ are far lower than the maximum $\delta$ observed in Table I.

2) **Debiasing 100% Malware Datasets:** We were first interested in investigating whether it is possible to debias a small and old dataset (Drebin) on the one hand and a more recent and larger dataset on the other hand (malware from VirusShare between 2015 and 2018). We want the produced datasets to resemble the population of AndroZoo, extracted in 2020.

The debiasing results show us that Drebin is strongly biased as we need to add from 1.8% to 12.7% of new samples, because a lot of classes of the population are underrepresented or missing in Drebin. Even with these modifications, the resulting dataset is not homogeneous to the population. On the contrary, VirusShare can be debiased with few additions (less than 2.9%) but requires removing more than 90% of the dataset. In that case, the resulting dataset obtains a p-value of 1. We conclude that the VirusShare dataset already contains enough material to be representative of AndroZoo for certain classes but that a lot of these classes are overrepresented.

3) **Discussion:** As seen in the results, Algorithm 2 can generate a new dataset statistically indistinguishable from the target dataset with a margin of error $\delta$. However, the algorithm may fail (cf. grey cells in Table II) when some classes are underrepresented in the source, as discussed in Section VII-A. This limitation can be overcome by providing a large and diverse enough source dataset.

On the other hand, when the algorithm succeeds, the generated datasets' size tends to be much smaller than the base dataset. This is due to a lot of overrepresented classes, from which the algorithm removes elements at line 9. This is especially the case for historical malware datasets.

The number of classes to consider may also limit the debiasing: intuitively, more characteristics would help to resemble the target dataset better, but it spreads the elements into more classes and makes it more difficult to find elements for certain classes.

### B. Building Mixed Datasets for Machine Learning

We finally produced two mixed datasets (goodware/malware) usable for training and testing machine learning algorithms. These datasets are built on top of the debiased malware datasets previously discussed. The goodware are taken from NAZE<sub>Debiased</sub>18-G dataset. These two datasets are:

* **DR-AG<sub>Deb</sub>** (standing for Drebin+AndroZoo Goodware), that mixes the debiased version of Drebin with $\delta = 0.01$ and the goodware.

* **VS-AG<sub>Deb</sub>** (standing for VirusShare+AndroZoo Goodware), that mixes the debiased version of VS<sub>Debiased</sub>15-18 when $\delta = 0.1$ and the goodware.

Because the goal is to use mixed datasets for machine learning algorithms, we have to follow additional recommendations from Pendlebury *et al.* [7]. We build our mixed sets by extracting random samples from goodware and malware and by following the recommendations C1 (APK of the training set should be older than the ones of the test set) and C3 (realistic malware-to-goodware ratio i.e. 5% or 10%). In Table III we call “Pivot year” the year that delimitates the train and test sets. Note also that a consequence of C1 is that applications

<footnote n="7">⁷All materials used in the experiments (SHA256 hashes of samples comprising the datasets and the code) are published in the Dada dataset [45].</footnote>
<table><caption>TABLE II
RESULTS AFTER DEBIASING DATASETS</caption><thead><tr><th>Base Dataset</th><th>Reference</th><th>Source</th><th>Diff</th><th>Del</th><th>Add</th><th>|G|</th><th colspan="3">Debiased dataset G</th><th>Duration validity</th></tr><tr><th>|B|</th><th>|T|</th><th>|S|</th><th>δ</th><th></th><th></th><th></th><th>Add ratio</th><th>χ<sup>2</sup></th><th>p-value</th><th></th></tr></thead><tbody><tr><td colspan="11" style="text-align:center; font-weight:bold;">Debiasing Goodware datasets</td></tr><tr><td rowspan="6">NAZE-18-G<br>11 120</td><td rowspan="6">→<br>AZ20<sub>30k</sub><br>29 974</td><td rowspan="6">AZ19<sub>100k</sub>G<br>72 131</td><td>0.04</td><td>0</td><td>0</td><td>11 120</td><td>0.00</td><td>12 589</td><td>0</td><td rowspan="6"></td></tr><tr><td>0.02</td><td>438</td><td>50</td><td>10 732</td><td>0.47</td><td>12 058</td><td>0</td></tr><tr><td>0.01</td><td>1 026</td><td>223</td><td>10 317</td><td>2.16</td><td>11 291</td><td>0</td></tr><tr><td>0.005</td><td>1 885</td><td>632</td><td>9 867</td><td>6.41</td><td>9 726</td><td>0</td></tr><tr><td>0.0025</td><td>2 685</td><td>1 280</td><td>9 715</td><td>13.18</td><td>7 998</td><td>0</td></tr><tr><td>0.001</td><td>5 808</td><td>1 289</td><td>6 601</td><td>19.53</td><td>5 104</td><td>0.999 44</td></tr><tr><td colspan="11" style="text-align:center; font-weight:bold;">Debiasing Malware datasets</td></tr><tr><td rowspan="4">Drebin<br>5 304</td><td rowspan="4">→<br>AZ20<sub>30k</sub><br>29 974</td><td rowspan="4">VS 15-18, AMD<br>103 293</td><td>0.04</td><td>892</td><td>81</td><td>4 493</td><td>1.80</td><td>25 781</td><td>0</td><td rowspan="4"></td></tr><tr><td>0.02</td><td>3 634</td><td>116</td><td>1 786</td><td>6.49</td><td>20 178</td><td>0</td></tr><tr><td>0.01</td><td>4 596</td><td>103</td><td>811</td><td>12.70</td><td>15 833</td><td>0</td></tr><tr><td>0.005</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="5">VS 15-18<br>80 035</td><td rowspan="5">→<br>AZ20<sub>30k</sub><br>29 974</td><td rowspan="5">Drebin, AMD, AZ-17-18-M<br>32 562</td><td>0.04</td><td>66 475</td><td>21</td><td>13 581</td><td>0.15</td><td>16 709</td><td>0</td><td rowspan="5"></td></tr><tr><td>0.02</td><td>75 825</td><td>31</td><td>4 241</td><td>0.73</td><td>7 406</td><td>0</td></tr><tr><td>0.01</td><td>78 074</td><td>43</td><td>2 004</td><td>2.15</td><td>4 349</td><td>1</td></tr><tr><td>0.005</td><td>79 063</td><td>29</td><td>1 001</td><td>2.90</td><td>915</td><td>1</td></tr><tr><td>0.0025</td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td colspan="11" style="text-align:center; font-weight:bold;">Debiasing the VirusShare dataset over time</td></tr><tr><td rowspan="3">VS 15<br>28 896</td><td rowspan="3">→<br>AZ19<sub>100k</sub><br>15<br>5 792</td><td rowspan="3">Drebin<br>5 304</td><td>0.04</td><td>13 877</td><td>4</td><td>15 023</td><td>0.03</td><td>29 475</td><td>0</td><td>0</td></tr><tr><td>0.02</td><td>19 832</td><td>90</td><td>9 154</td><td>0.98</td><td>21 828</td><td>0</td><td>0</td></tr><tr><td>0.01</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="2">VS 16<br>12 651</td><td rowspan="2">→<br>AZ19<sub>100k</sub><br>16<br>27 516</td><td rowspan="2">Drebin, VS 15<br>34 200</td><td>0.04</td><td>4 549</td><td>1</td><td>8 103</td><td>0.01</td><td>23 204</td><td>0</td><td>0</td></tr><tr><td>0.02</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="2">VS 17<br>9 945</td><td rowspan="2">→<br>AZ19<sub>100k</sub><br>17<br>6 524</td><td rowspan="2">Drebin, VS 15-16, AMD<br>70 109</td><td>0.04</td><td>580</td><td>0</td><td>9 365</td><td>0.00</td><td>21 073</td><td>0</td><td>0</td></tr><tr><td>0.02</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="3">VS 18<br>28 543</td><td rowspan="3">→<br>AZ19<sub>100k</sub><br>18<br>24 549</td><td rowspan="3">Drebin, VS 15-17, AMD<br>80 054</td><td>0.04</td><td>27 644</td><td>7</td><td>906</td><td>0.77</td><td>4 997</td><td>0.356 52</td><td>1</td></tr><tr><td>0.02</td><td>28 189</td><td>7</td><td>361</td><td>1.94</td><td>1 809</td><td>1</td><td>1</td></tr><tr><td>0.01</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr></tbody></table>

in the training and test sets are disjoint (which correspond to an added C4*: Train ∩ Test = ∅). For C2 (temporal window consistency) we do not enforce the constraint as we already balanced the classes on the year when performing the debiasing algorithms. Nevertheless, we can enforce them if needed, at the cost of reducing the output size of the training/tests sets. For example, the set VS-AG-C2<sub>Deb</sub> has fewer samples than VS-AG<sub>Deb</sub>. In the rest of the paper we only use the set that does not enforce C2 to decrease the quantity of results to discuss.

We recomputed the average $\delta$ of these mixed datasets: it is still quite low (worst case of 0.0012 for VS-AG<sub>Deb</sub>). The p-value falls to zero for some sets because applying the constraints of Pendlebury et al. drives the datasets away from the population. Nevertheless, we believe that respecting the time consistency for machine learning algorithms is of higher importance.

Finally, for evaluating these datasets when used for machine learning purposes, Table III includes additional datasets:

* **VS-AG<sub>Deb,δ=0.4</sub>**: a relaxed version of the debiasing of VS<sub>Debiased</sub>15-18 with $\delta = 0.4$. This dataset as a p-value of 0 but contains more samples in the training set.

* **DR-AG**: a mixed version of Drebin and goodware.

* **VS-AG**: a mixed version of VS 15-18 and goodware.

* **ACT14** and **ACT17**: a mixed version of AndroCT [46] with pivot year 2014 and 2017 respectively.

* **AZL14** and **AZL17**: a mixed version of random samples extracted from AZ20<sub>30k</sub> for which we took the goodware/malware label of AndroZoo as an oracle.

All these datasets will be used to compare the efficiency of machine learning algorithms when using our debiased datasets. In particular, AZL14 and AZL17 are the biggest extract of the population. AndroCT [46] is a new recent datasets that cover ten years of applications: it will be used to observe its performances when learning with it and testing on AZL14/17.

### C. Classifying With Machine Learning

We evaluate the performance of various machine learning classifiers depending on their training datasets to assess the contribution of debiased datasets. The datasets used in this experiments are those computed in the previous section, cf. Table III. The machine learning classifiers rely on a total of 262 characteristics. The first 12 characteristics are those used to debiased the datasets that are not related to security. Additionally, we included the characteristics related to security computed from two sources:
<table><caption>TABLE III
MIXED DATASETS FOR MACHINE LEARNING TRAINING AND EVALUATION</caption><thead><tr><th rowspan="2">Malware</th><th rowspan="2">Goodware</th><th rowspan="2">Pivot year</th><th rowspan="2">Set</th><th rowspan="2">Mal prop.</th><th rowspan="2">Good prop.</th><th colspan="4">Constraints [7]</th><th colspan="4">Mixed dataset</th></tr><tr><th>C1</th><th>C2</th><th>C3</th><th>C4*</th><th>Name</th><th>Size</th><th>Avg δ</th><th>p-value</th></tr></thead><tbody><tr><td rowspan="2">Drebin<sub>Debiased</sub><br>(δ = 0.01)</td><td rowspan="2">NAZE<sub>Debiased</sub> 18-G<br>(δ = 0.001)</td><td rowspan="2">2014</td><td>Training</td><td>50%</td><td>50%</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>DR-AG<sub>Deb</sub></td><td>1614</td><td>0.017</td><td>0</td></tr><tr><td>Test</td><td>10%</td><td>90%</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>DR-AG<sub>C2Deb</sub></td><td>572</td><td>0.054</td><td>0</td></tr><tr><td rowspan="2">VS<sub>Debiased</sub> 15-18<br>(δ = 0.01)</td><td rowspan="2">NAZE<sub>Debiased</sub> 18-G<br>(δ = 0.001)</td><td rowspan="2">2017</td><td>Training</td><td>50%</td><td>50%</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>VS-AG<sub>Deb</sub></td><td>3492</td><td>0.012</td><td>1</td></tr><tr><td>Test</td><td>10%</td><td>90%</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>VS-AG<sub>C2Deb</sub></td><td>2858</td><td>0.013</td><td>1</td></tr><tr><td rowspan="2">VS<sub>Debiased</sub> 15-18<br>(δ = 0.04)</td><td rowspan="2">NAZE<sub>Debiased</sub> 18-G<br>(δ = 0.01)</td><td rowspan="2">2017</td><td>Training</td><td>50%</td><td>50%</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>VS-AG<sub>Deb</sub></td><td>1333</td><td>0.057</td><td>0</td></tr><tr><td>Test</td><td>10%</td><td>90%</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>VS-AG<sub>C2Deb</sub></td><td>1333</td><td>0.057</td><td>0</td></tr><tr><td colspan="14"></td></tr><tr><td>Drebin</td><td>AZ19<sub>100k</sub>G</td><td>2014</td><td>Training</td><td>50%</td><td>50%</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>DR-AG</td><td>10608</td><td>-</td><td>-</td></tr><tr><td rowspan="2">VS 15-18</td><td rowspan="2">AZ19<sub>100k</sub>G</td><td rowspan="2">2017</td><td>Training</td><td>50%</td><td>50%</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>VS-AG</td><td>133244</td><td>-</td><td>-</td></tr><tr><td>Test</td><td>10%</td><td>90%</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>VS-AG</td><td>6113</td><td>-</td><td>-</td></tr><tr><td colspan="14"></td></tr><tr><td rowspan="4">AndroCT [46]</td><td rowspan="4"></td><td rowspan="2">2014</td><td>Training</td><td>50%</td><td>50%</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>ACT14</td><td>19351</td><td>0.065</td><td>0</td></tr><tr><td>Test</td><td>10%</td><td>90%</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>ACT14</td><td>12241</td><td>0.057</td><td>0</td></tr><tr><td rowspan="2">2017</td><td>Training</td><td>50%</td><td>50%</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>ACT17</td><td>26389</td><td>0.054</td><td>0</td></tr><tr><td>Test</td><td>10%</td><td>90%</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>ACT17</td><td>4580</td><td>0.093</td><td>0</td></tr><tr><td colspan="14"></td></tr><tr><td rowspan="4">AZ20<sub>30k</sub> with labels</td><td rowspan="4"></td><td rowspan="2">2014</td><td>Training</td><td>50%</td><td>50%</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>AZL14</td><td>4890</td><td>0.015</td><td>0</td></tr><tr><td>Test</td><td>10%</td><td>90%</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>AZL14</td><td>8785</td><td>0.057</td><td>0</td></tr><tr><td rowspan="2">2017</td><td>Training</td><td>50%</td><td>50%</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>AZL17</td><td>9474</td><td>0.019</td><td>0.894</td></tr><tr><td>Test</td><td>10%</td><td>90%</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>AZL17</td><td>2033</td><td>0.059</td><td>0</td></tr></tbody></table>

<table><caption>TABLE IV
MEAN AND MAX AUC OF MACHINE LEARNING CLASSIFIERS DEPENDING ON THEIR TRAINING SET AND THEIR TEST SET WHEN THE PIVOT YEAR IS 2017. IN BOLD FACE: BEST AUC ON A TEST SET</caption><thead><tr><th rowspan="2">Train</th><th rowspan="2">Test</th><th colspan="2">ACT17</th><th colspan="2">AZL17</th><th colspan="2">VS-AG</th><th colspan="2">VS-AG<sub>Deb</sub></th></tr><tr><th>Mean</th><th>Max</th><th>Mean</th><th>Max</th><th>Mean</th><th>Max</th><th>Mean</th><th>Max</th></tr></thead><tbody><tr><td>ACT17</td><td></td><td>0.62</td><td>0.65</td><td>0.67</td><td>0.71</td><td>0.76</td><td>0.85</td><td>0.74</td><td>0.78</td></tr><tr><td>AZL17</td><td></td><td><strong>0.72</strong></td><td><strong>0.81</strong></td><td><strong>0.78</strong></td><td><strong>0.86</strong></td><td>0.71</td><td>0.89</td><td>0.75</td><td>0.80</td></tr><tr><td>VS-AG</td><td></td><td>0.62</td><td>0.68</td><td>0.66</td><td>0.73</td><td><strong>0.96</strong></td><td><strong>0.97</strong></td><td>0.74</td><td>0.84</td></tr><tr><td>VS-AG<sub>Deb</sub></td><td></td><td><strong>0.70</strong></td><td><strong>0.74</strong></td><td><strong>0.71</strong></td><td><strong>0.75</strong></td><td>0.86</td><td>0.89</td><td><strong>0.83</strong></td><td><strong>0.87</strong></td></tr><tr><td>DR-AG</td><td></td><td>0.54</td><td>0.57</td><td>0.53</td><td>0.60</td><td>0.57</td><td>0.63</td><td>0.54</td><td>0.58</td></tr></tbody></table>

<table><caption>TABLE V
MEAN AND MAX AUC OF MACHINE LEARNING CLASSIFIERS DEPENDING ON THEIR TRAINING SET AND THEIR TEST SET WHEN THE PIVOT YEAR IS 2014. IN BOLD FACE: BEST AUC ON A TEST SET</caption><thead><tr><th rowspan="2">Train</th><th rowspan="2">Test</th><th colspan="2">ACT14</th><th colspan="2">AZL14</th><th colspan="2">DR-AG<sub>Deb</sub></th></tr><tr><th>Mean</th><th>Max</th><th>Mean</th><th>Max</th><th>Mean</th><th>Max</th></tr></thead><tbody><tr><td>ACT14</td><td></td><td>0.69</td><td>0.73</td><td>0.71</td><td>0.74</td><td>0.69*</td><td>0.83*</td></tr><tr><td>AZL14</td><td></td><td>0.75</td><td>0.81</td><td>0.76</td><td>0.82</td><td>0.72*</td><td>0.99*</td></tr><tr><td>DR-AG</td><td></td><td>0.57</td><td>0.58</td><td>0.56</td><td>0.58</td><td>0.48*</td><td>0.50*</td></tr><tr><td>DR-AG<sub>Deb</sub></td><td></td><td>0.62</td><td>0.66</td><td>0.61</td><td>0.66</td><td>0.72*</td><td>0.80*</td></tr></tbody></table>

* Droidlysis that computes 214 Booleans about the use of some APIs or behaviors (loading a DEX file, using cryptography, etc.);

* FalDroid [47] that computes 48 characteristics representing score values related to graph-based features (with parameters $\epsilon = 0.8, \theta = 0.1$ [47]). As FalDroid is intended to classify families, we configured the software with only two families (goodware/malware) and we extracted features related to control flow graph that manipulates sensitive APIs.

We base this evaluation on the area under the receiver operating characteristic curve (AUC), a classical metric in machine

learning. This metric can be interpreted as the probability that a classifier considers a randomly chosen goodware to be more probably benign than a randomly chosen malware. A perfect classifier has an AUC of 1 (every goodware is ranked higher than all malware), meaning that there exists a threshold to separate exactly goodware and malware. A random classifier has an AUC of 0.5. Besides, this metric does not depend on some threshold selection.

We evaluate various machine learning techniques: k-nearest neighbors [48] (neighborhood-based technique), decision trees [49], random forest [50] (bagging ensemble technique with decision trees), Gaussian naive Bayes [51] (statistical

<footnote>* due to the very limited size of the test set of DR-AG<sub>Deb</sub>, these results are unreliable</footnote>
model), and AdaBoost [52] (boosting ensemble models with decision trees). We used the scikit-learn [53] implementation with default parameters. Since the decision trees, random forest and AdaBoost learning algorithms are stochastic, we took the average AUC over 25 seeds.

Average and max AUC are presented in Tables IV (pivot year of 2017) and V (pivot year of 2014) where each line corresponds to a different training set and each column to a different test set. The mean AUC over the various classifiers helps to estimate the global quality of a training dataset for machine learning. The max AUC reflects what performances could be obtained with the best classifiers.

The datasets go by pair: there is one training set and one test set (cf. Table III). In Table IV, we can see that generally learning on one training set yields the best results on the associated test set (i.e., the diagonal is in bold). The unique exception is that learning on AZL17 yields better results than learning on ACT17 when testing with ACT17. In fact, we can see that the AZL17 training set allows learning the best classifiers on ACT17 (max AUC: 0.77) and AZL17 (max AUC: 0.84), probably because it is a real randomly sampled dataset, and therefore it is less prone to bias. However, this training set was built using VirusTotal as an oracle, so it is, in general, not available for training.

Moreover, the most interesting results are the comparison of the performances of VS-AG and VS-AG<sub>Deb</sub> on ACT17 and AZL17 test sets. We see that the **debiasing dataset always performs better than the original one**. Another important result is that **it is notably better to train on the debiased dataset VS-AG<sub>Deb</sub> than on ACT17** (AndroCT [46]), which is the most recent dataset we have. Finally, **the debiased test dataset VS-AG<sub>Deb</sub> is more difficult to predict than the test set VS-AG**. It is a sign that the bias in the test set of VS-AG makes it easier for a model to correctly classify it, probably because some difficult cases are absent.

Finally, we can remark that the DR-AG training set yields poor results on all test sets, which is an expected result, as no applications in this dataset after 2015. We tried to confirm this result with an additional experiment with the pivot year of 2014, in Table V. In this experiment, we evaluate if the debiased version of Drebin can obtain good results, even if it is old and quite small. The row in Table V concerning DR-AG<sub>Deb</sub> shows that this dataset is far from being effective compared to ACT14. This is due to its size and the failing of debiasing when $\delta$ reaches 0.05 (cf. Table II).

## D. Discussion

This section discusses the validity of the datasets through time. We also compare these results with related works [32], [54], [55] that have studied this question and the relation with Android history.

First, Cai *et al.* [54] focus on the evolution of usage of the system's API over time, using a dedicated analysis framework [32]. They conclude three main points: the decrease of usage of callbacks for methods of the app activities' lifecycle, a stable and small usage of inter-component communication, and a stable distribution of source/sink categories of callbacks.

Second, Cai *et al.* [55] characterize this evolution for good-ware and malware applications. Newer malware tend to access system APIs more often through third-party libraries than older malware. The use of activities and services has increased over time, while the use of ICC components has decreased. The diversity of callback categories has also increased.

Our experiments confirm this evolution at a higher level. We study the evolution of representativeness of our debiased dataset over the years. More precisely, we debiased several extracts of VirusShare 2015, 2016, 2017 and 2018, and checked whether these debiased datasets stay representative of an evolving population over time. The target datasets are extracts of applications from AndroZoo 2019 (AZ19<sub>100k</sub>) filtered on the corresponding period of time ($T = AZ19_{100k} i$, with $i = 15, 16, 17, 18$). The source datasets are known datasets of malware such as Drebin and AMD, available at date *i*.

When the *p*-value confirms that the dataset is not statistically different, we tested the produced debiased version $VS_{Debiased} i$ with the next years' population, i.e., AZ19<sub>100k</sub> *i* + 1, *i* + 2, .... The last column indicates contains 0 if the *p*-value is 0: no discussion can be done for such debiased datasets. The only debiased dataset of the parts of VirusShare is VS 18. As it is the last set of applications we have, we only conclude that this set can be used during 1 year.

## IX. CONCLUSION

In this article, we presented the different types of Android datasets used by researchers to evaluate their experiments. We argued that these datasets have problems with their construction depending on the type of experiment. For surveys on bigger datasets, we proposed the use of a margin of error to obtain a lower bound on the size of the sample to be drawn. We also proposed a methodology based on the $\chi^2$ test to evaluate the homogeneity of a dataset with respect to a given population for a finite set of characteristics. When starting from a dataset that does not resemble the population, we introduced a debiasing algorithm to push this biased dataset towards the population's characteristics.

The choice of characteristics for which the dataset is debiased is of high importance. We propose here to debias datasets for a finite set of computable characteristics unrelated to the characteristic under study in order to keep only the relevant differences with the population.

Finally, using this algorithm, we proposed directly usable datasets for the evaluation of machine learning algorithms. We evaluated these debiased datasets using machine learning algorithms and we show that they outperform biased ones and in particular one recent dataset of the literature. Additionally, our debiased dataset, VS-AG<sub>Deb</sub>, would be usable for further works as a difficult test set for classification experiments.

Future works concern the study of the influence of the considered characteristics over the performances of detection algorithms. In particular, limiting the study to a small number of characteristics when debiasing, prevents combinations from exploding. As any characteristic could introduce a bias in the dataset, it would be unrealistic to enumerate and balance
them when building a dataset. An additional study about the influence of the characteristic over the bias is needed. It would help to understand what are the characteristics that are important when evaluating an algorithm. Additionally, studying more characteristics, especially the ones that can be extracted from dynamic analysis, is of independent interest.

# APPENDIX

## A. Equation (1) Proof

The margin of error with finite population correction [37] for a boolean characteristic of estimated probability $\hat{p}$ is:

$$ \delta \le z(C) \sqrt{\frac{\hat{p}(1-\hat{p})}{n}} \sqrt{\frac{N-n}{N-1}} $$

The term $\hat{p}(1-\hat{p})$ is maximized when $\hat{p} = 0.5$, so we can bound this value by $0.5(1 - 0.5) = \frac{1}{4}$, i.e.:

$$ \delta \le z(C) \sqrt{\frac{\hat{p}(1-\hat{p})}{n}} \sqrt{\frac{N-n}{N-1}} \le z(C) \sqrt{\frac{N-n}{4n(N-1)}} $$

Rearranging this inequality brings Equation (1). □

## B. Algorithm Proofs

For the sake of brevity and clarity, we introduce the notation $\Delta_T p_{G,k} = p_{G,k} - p_{T,k}$. So a class $k$ is overrepresented w.r.t. $T$ if $\Delta_T p_{G,k} > \delta$ and underrepresented if $\Delta_T p_{G,k} < -\delta$.

**Theorem 2: Algorithm 1 always halts.**

*Proof:* First, remark that if a class $k$ is correctly represented at some iteration, i.e., if $|\Delta_T p_{G,k}| \le \delta$, then it will be correctly represented in the next iteration. Besides, a class that is overrepresented in $G$ cannot become underrepresented in the following iteration (and vice versa) without the algorithm halting. Both remarks are ensured by the “If” block on line 13 that halts the program by verifying whether a class with an added element is overrepresented or a class with a removed element is underrepresented.

Let's assume that this case never happens. At each iteration of the algorithm, there is at least one class $k$ such that $|\Delta_T p_{G,k}| > \delta$ that is modified in $G$: one element is added if $k$ is underrepresented and one element is removed if $k$ is overrepresented in $G$. So $\sum_k |\Delta_T p_{G,k}|$ is reduced by at least $\frac{1}{|B|}$ in the updated value of $G$. With enough iterations, $\sum_k |\Delta_T p_{G,k}|$ will so low that every class will be correctly represented. At that point, the algorithm halts. □

Besides, a representative dataset always exists. This is necessary to prove the termination and the complexity of Algorithm 2.

**Theorem 3: Let $T$ be a dataset, $\delta > 0$ and $n > 0$ such as $n \ge \frac{1}{2}$. There exists a dataset $D$ of size $n$ such as $D$ is $\delta$-representative of $T$**.

*Proof:* To prove the existence of such dataset of size $n$, we construct it. More precisely, we associate to each class $k$ a number of element $b_k$ such as this class is $\delta$-representative of $T$, i.e., $\frac{b_k}{n} \in [p_{T,k} - \delta; p_{T,k} + \delta]$. For each class $k$, define $a_k$ such as $\frac{a_k}{n} < p_{T,k} \le \frac{a_k+1}{n}$. Remark that $p_{T,k} - \delta \le \frac{a_k}{n} < p_{T,k} \le \frac{a_k+1}{n} \le p_{T,k} + \delta$ since $\delta \ge \frac{1}{n}$. As $\sum_k p_{T,k} = 1$,

$\sum_k a_k < n$ and $\sum_k (a_k + 1) \ge n$. So there exist $b_k$ such as for all class $k$ either $b_k = a_k$ or $b_k = a_k + 1$ and that verify $\sum_k b_k = n$.

Denote the dataset $D$ such as $p_{D,k} = \frac{b_k}{n}$. The size of $D$ is $\sum_k b_k = n$ and $D$ is $\delta$-representative of $T$ since $\frac{b_k}{n} \in [p_{T,k} - \delta; p_{T,k} + \delta]$ for every class $k$. It proves the existence of such a dataset. □

**Theorem 4: If there exists a dataset $D$ consisting of elements of $B$ and $S$ (i.e., $D \subseteq B \cup S$) such as $D$ is representative of $T$ and $|D| = |B|$, then Algorithm 1 returns a representative dataset. If no such dataset exists, the algorithm returns “Impossible.”**

*Proof:* Assume such dataset $D$ exists. Let us first prove that the **return** on line 9 cannot be reached. This line is reached if $S$ is empty or if $\Delta_T p_{G,k} < -\delta$.

Consider the case where $\Delta_T p_{G,k} < -\delta$: we will prove by contradiction that this case is impossible when $D$ exists. Since the algorithm never removes an element of an underrepresented class, at each iteration $p_{G,k} = \frac{B(k_l)+m}{n}$ where $m$ is the number of elements added to the class from $S$. If $S(k_l) = 0$, then $p_{G,k_l}$ is maximized, i.e., $p_{G,k_l} \ge p_{D,k_l}$. So $\Delta_T p_{D,k_l} \le \Delta_T p_{G,k_l} < -\delta$ which means that $D$ is not representative of $T$. This is a contradiction: therefore this case is not possible.

Consider the case where $S$ is empty and $\Delta_T p_{G,k} \ge -\delta$. Once again, we will prove by contradiction that this case is impossible when $D$ exists. The fact that $\Delta_T p_{G,k} \ge -\delta$ implies that $\Delta_T p_{G,k_h} \ge \delta$, otherwise the “While” condition would not have been true. Denote $m$ the number of completed iterations so far. Since one element of $S$ is removed at each iteration, we conclude that initially $|S| = m$. Denote $B_D \subseteq B$ and $S_D \subseteq S$ such as $D = B_D \cup S_D$. Due to the definition of $k_h$ and the fact that $\Delta_T p_{G,k_h} \ge \delta$, it means that for each previous iterations one element has been removed from an overrepresented class. So there are at least $m+1$ elements of $B$ that are absent in $B_D$, i.e., $|B_D| \le |B| - m$. However, since $|B| = |D| = |B_D| + |S_D|$, it means that $|S| \ge m + 1$. This is in contradiction with the fact that $|S| = m$. So $S$ cannot be empty. Finally, the **return** on line 9 cannot be reached when $D$ exists.

We now prove that the “If” condition on line 13 cannot be true in Algorithm 1 when $D$ exists. For algorithm 1 to return an error on line 14, the condition $\max_{k \in K} |\Delta_T p_{G,k}| > \delta$ must be true. Let us assume (without loss of generality) that there exists $k'$ such as $\Delta_T p_{G,k'} > \delta$ (the case $\Delta_T p_{G,k'} < -\delta$ is similar).

Consider now the value of $k_h$. Since $k_h \leftarrow \arg \max_{k \in K} \Delta_T p_{G,k}$, we can conclude that $\Delta_T p_{G,k_h} \ge \Delta_T p_{G,k'} > \delta$. Denote $n = |B|$ and $G'$ the dataset obtained from $G$ by removing one element from $k_h$ and adding one element to $k_l$. The proportions of the classes of $G'$ are:

$$ p_{G',k} = \begin{cases} p_{G,k} - \frac{1}{n} & \text{if } k = k_h \\ p_{G,k} + \frac{1}{n} & \text{if } k = k_l \\ p_{G,k} & \text{otherwise} \end{cases} $$

Our goal is to prove that the “If” condition will never be true, i.e., that $\Delta_T p_{G',k_h} \ge -\delta$ and $\Delta_T p_{G',k_l} \le \delta$.
Let $a_1$ and $a_2$ be two naturals such as $p_{G,k_h} = \frac{a_1}{n}$ and $p_{D,k_h} = \frac{a_2}{n}$. Since $\Delta_T p_{G,k_h} > \delta$ and $\Delta_T p_{D,k_h} \le \delta$ (due to $\mathbf{D}$ being representative), we can conclude that $p_{G,k_h} > p_{D,k_h}$, i.e., $a_1 > a_2$. Since $p_{G',k_h} = p_{G,k_h} - \frac{1}{n} = \frac{a_1-1}{n}$ and $a_1 - 1 \ge a_2$, we conclude that $p_{G',k_h} \ge p_{D,k_h}$ and $\Delta_T p_{G',k_h} \ge \Delta_T p_{D,k_h}$. Since $\mathbf{D}$ is itself representative of $\mathbf{T}$, we know that $\Delta_T p_{D,k_h} \ge -\delta$ and therefore that $\Delta_T p_{G',k_h} \ge -\delta$.

Let us show that there exists $k'_l$ such as $\Delta_T p_{G,k_l} < \Delta_T p_{D,k'_l} \le \delta$. Since $\Delta_T p_{G,k_h} > \Delta_T p_{D,k_h}$ and $\sum_k \Delta_T p_{G,k} = \sum_k \Delta_T p_{D,k}$, we can conclude that there exists $k'_l$ such as $\Delta_T p_{G,k'_l} < \Delta_T p_{D,k'_l}$. Furthermore, $k'_l \in \mathcal{K}_S$ because if $\mathbf{S}$ were empty it would not be possible for $\mathbf{D}$ to have more elements than $\mathbf{G}$ for this class.

Due to the definition of $k_l$, $\Delta_T p_{G,k_l} \le \Delta_T p_{G,k'_l} < \Delta_T p_{D,k'_l} \le \delta$. Let $b_1, b_2, b_3, b_4$ four naturals such as $p_{G,k_l} = \frac{b_1}{n}$, $p_{T,k_l} = \frac{b_2}{n}$, $p_{D,k'_l} = \frac{b_3}{n}$ and $p_{T,k'_l} = \frac{b_4}{n}$. We can conclude of the previous inequalities that $b_1 - b_2 < b_3 - b_4$. Therefore $b_1 - b_2 + 1 \le b_3 - b_4$ and $\Delta_T p_{G',k_l} = p_{G,k_l} + \frac{1}{n} - p_{D,k_l} \le \delta$. This proves that the “If” condition is never true if $\mathbf{D}$ exists.

Finally, if $\mathbf{D}$ exists, then Algorithm 1 returns a dataset. This dataset is representative because the “While” condition is true for the last iteration. When the algorithm outputs a dataset, it is representative and its size is the same as $\mathbf{B}$. If such dataset does not exist, the algorithm cannot output one. Since the algorithm always halts, it must return “Impossible.” □

We denote $d_H(\mathbf{A}, \mathbf{B})$ the number of modifications between $\mathbf{A}$ and $\mathbf{B}$. More precisely, we extend the Hamming distance to a distance between two sets by defining $d_H(\mathbf{A}, \mathbf{B})$ as $d_H(\mathbf{A}, \mathbf{B}) = |(\mathbf{A} \setminus \mathbf{B}) \cup (\mathbf{B} \setminus \mathbf{A})|$. $d_H(\mathbf{A}, \mathbf{B})$ is the number of elements present in only one dataset, $\mathbf{A}$ or $\mathbf{B}$. In other words, it is the minimal number of addition and deletions to transform $\mathbf{A}$ into $\mathbf{B}$ (and vice-versa).

**Theorem 5:** Let $\mathbf{B}$ be a biased dataset, $\mathbf{D}$ a dataset representative of $\mathbf{P}$ such as $\mathbf{D} \subseteq \mathbf{B} \cup \mathbf{S}$ and $\mathbf{G} = \text{Algo1}(\mathbf{B})$. If $|\mathbf{B}| = |\mathbf{D}|$, then $d_H(\mathbf{B}, \mathbf{D}) \ge d_H(\mathbf{B}, \mathbf{G}) = 2\max(\sum_k n_{\mathbf{B}}^+(k), \sum_k n_{\mathbf{B}}^-(k))$. So Algorithm 1 outputs a representative dataset with minimal distance to $\mathbf{B}$.

*Proof:* Let $\mathbf{A} = \mathbf{D} \setminus \mathbf{B}$ be the set of elements added to $\mathbf{B}$ and $\mathbf{R} = \mathbf{B} \setminus \mathbf{D}$ be the set of elements removed from $\mathbf{B}$. So $\mathbf{D} = (\mathbf{B} \setminus \mathbf{R}) \cup \mathbf{A}$ and $d_H(\mathbf{B}, \mathbf{D}) = |\mathbf{A}| + |\mathbf{R}|$. Due to the hypothesis $|\mathbf{A}| = |\mathbf{R}|$, we know that $|\mathbf{A}| = |\mathbf{R}|$ (there are as many additions as removals).

For any class $k$ that is overrepresented in $\mathbf{B}$, $n_{\mathbf{B}}^+(k)$ is the minimal number of removal of elements of class $k$ such as $k$ is not overrepresented anymore. $n_{\mathbf{B}}^-(k) = \max(0, -|\mathbf{B}|(\Delta_T p_{\mathbf{B},k} + \delta))$ is similar for underrepresented classes.

To make a representative dataset from $\mathbf{B}$, one must remove at least $\sum_k n_{\mathbf{B}}^+(k)$ elements and add $\sum_k n_{\mathbf{B}}^-(k)$, i.e., $|\mathbf{A}| \ge \sum_k n_{\mathbf{B}}^+(k)$ and $|\mathbf{R}| \ge \sum_k n_{\mathbf{B}}^-(k)$. However, since $|\mathbf{A}| = |\mathbf{R}|$, we conclude that $|\mathbf{A}| = |\mathbf{R}| \ge \max(\sum_k n_{\mathbf{B}}^+(k), \sum_k n_{\mathbf{B}}^-(k))$. It means that $d_H(\mathbf{B}, \mathbf{D}) \ge 2\max(\sum_k n_{\mathbf{B}}^+(k), \sum_k n_{\mathbf{B}}^-(k))$.

Let now show that $d_H(\mathbf{B}, \mathbf{G}) = 2\max(\sum_k n_{\mathbf{B}}^+(k), \sum_k n_{\mathbf{B}}^-(k))$. At each iteration of the algorithm, if $\sum_k n_{\mathbf{G}}^+(k) > 0$, then an overrepresented class has one of its elements removed, so $\sum_k n_{\mathbf{G}}^+(k) < \sum_k n_{\mathbf{G}}^-(k)$. For the same reason, at each iteration, $\sum_k n_{\mathbf{G}}^-(k) < \sum_k n_{\mathbf{G}}^-(k)$. When $\sum_k n_{\mathbf{G}}^+(k) = \sum_k n_{\mathbf{G}}^-(k) = 0$, then the algorithm halts.

So there are $\max(\sum_k n_{\mathbf{G}}^+(k) = \sum_k n_{\mathbf{G}}^-(k))$ iterations. Since each iteration makes two modifications of the dataset, we can conclude that $d_H(\mathbf{B}, \mathbf{G}) = 2\max(\sum_k n_{\mathbf{B}}^+(k), \sum_k n_{\mathbf{B}}^-(k))$.

So $d_H(\mathbf{B}, \mathbf{G}) \le d_H(\mathbf{B}, \mathbf{D})$. □

*Proof of theorem 1:* Let us first prove that at each step of the algorithm, $d$ is equal to the number of additions and deletions between $\mathbf{B}$ and $\text{Algo1}(\mathbf{G})$. This number can be decomposed as the sum of the number of additions and deletions between $\mathbf{B}$ and $\mathbf{G}$ and between $\mathbf{G}$ and $\text{Algo1}(\mathbf{G})$. The number of modifications between $\mathbf{G}$ and $\mathbf{B}$ is simply the difference of their size, i.e., $|\mathbf{B}| - |\mathbf{G}|$. The number of modification between $\mathbf{G}$ and $\text{Algo1}(\mathbf{G})$ is $2\max(\sum_k n_{\mathbf{G}}^+(k), \sum_k n_{\mathbf{G}}^-(k))$ (cf. the previous proof algorithm). So the number of modifications between $\mathbf{B}$ and $\text{Algo1}(\mathbf{G})$ is $|\mathbf{B}| - |\mathbf{G}| + 2\max(\sum_k n_{\mathbf{B}}^+(k), \sum_k n_{\mathbf{B}}^-(k))$.

First, let us show that the algorithm always halts. This algorithm halts either if $d_{min} \ne \infty$ (so both loops have a finite number of iterations) or if the source dataset lacks some element (lines 16 to 16). If a solution $\mathbf{G}$ exists, then at some point Algorithm 2 will call Algorithm 1 with a dataset whose size is the same as $\mathbf{G}$ (either at line 2 if $|\mathbf{G}| = |\mathbf{B}|$, at line 10 if $|\mathbf{G}| < |\mathbf{B}|$ or at line 26 if $|\mathbf{G}| > |\mathbf{B}|$) and Algorithm 1 will find a solution (due to Theorem 5). If at some point it is not possible to add an element to a underrepresented class because there is no such element in the source dataset, then the algorithm halts (lines 5 to 9).

Second, we show that doing $d_{min} - 1$ iterations per loop is sufficient (lines 6 and 15). Let $\mathbf{G}_{best}$ be the best dataset found so far, associated to a distance $d_{min}$, and a better solution $\mathbf{G}'$ associated to a distance $d'$ such as $d' < d_{min}$. Remark that at each step, $i = |\mathbf{B}| - |\mathbf{G}|$ as one element is modified per iteration. Denote $j$ the iteration at which $\mathbf{G}'$ could be found. Since $d' = |\mathbf{B}| - |\mathbf{G}| + 2\max(\sum_k n_{\mathbf{G}}^+(k), \sum_k n_{\mathbf{G}}^-(k)) \ge j$ and $d' < d_{min}$, we conclude that $j \le d_{min}$ and therefore that $\mathbf{G}'$ will be found by the algorithm. □

**Theorem 6:** The worst-case temporal complexity of Algorithm 2 is $O((\frac{1}{\delta} + |\mathbf{B}|)^2|\mathcal{K}|)$.

*Proof:* Remark that for all datasets $\mathbf{A}$ and $\mathbf{B}$, the number of additions and deletions required to obtain $\mathbf{B}$ from $\mathbf{A}$ is lower than or equal to $|\mathbf{A}| + |\mathbf{B}|$. Theorem 3 shows that there exists a representative dataset $\mathbf{D}$ of size $\lceil \frac{1}{\delta} \rceil$. So the minimal distance $d_{min}$ to reach a representative dataset $\mathbf{D}$ from the initial base dataset $\mathbf{B}$ is bounded by $\lceil \frac{1}{\delta} \rceil + |\mathbf{B}|$. So Algo. 2 will do at most $2(\lceil \frac{1}{\delta} \rceil + |\mathbf{B}|)$ iterations. Each iteration calls Algo. 1, whose temporal complexity is $O(|\mathbf{G}| \cdot |\mathcal{K}|)$ where the size of $\mathbf{G}$ is at most $|\mathbf{B}| + d_{min} \le \lceil \frac{1}{\delta} \rceil + 2|\mathbf{B}|$. So the worst-case temporal complexity of Algo. 2 is $O((\frac{1}{\delta} + |\mathbf{B}|)^2|\mathcal{K}|)$. □

### C. Datasets Used in the Literature

Table VI evaluates the dataset usages of 28 articles on Android malware detection using machine learning techniques. Column “Paper” gives the name of the tool or the authors’ names. Next, column “Goodware source” contains the datasets used in the experiment. Sometimes, several sets are used in a paper: we note these subsets $\mathbf{D}_i$ and we give their size in other columns accordingly. Next, columns “GS year” and “Nb Goodware” are the year range and number of goodware according to the article, if available. The next three columns give the same information for the used malware.
<caption>TABLE VI</caption>

DETAILS ON THE DATASET USAGE IN DETECTION EXPERIMENTS. GP = GOOGLE PLAY, VT = VIRUSTOTAL, GM = GM, DB = DREBIN, AZ = ANDROZOO, CT = CONTAGIO, VS = VIRUSSHARE, PD = PLAYDRONE, MSL = MOBISEC LAB, TZ = THEZOO, MS = MALSHARE, N/A = NOT APPLICABLE

<table><thead><tr><th>Paper</th><th>Goodware source</th><th>Goodware source year</th><th>Goodware size</th><th>Malware source</th><th>Malware source year</th><th>Malware Size</th><th>Training set (M/G)</th><th>Test set (M/G)</th><th>Class Imbalance</th><th>Time Incoherence</th></tr></thead><tbody><tr><td>MADAME [56]</td><td></td><td></td><td></td><td>GM<br>CT<br>VS</td><td>2010-2011<br>2016<br>2012-2015</td><td>1 242<br>1 923<br>500</td><td>2,784/0 (Total)</td><td></td><td>X</td><td></td></tr><tr><td>DroidDetector [57]</td><td>GP</td><td>2016</td><td>20 000</td><td>CT<br>GM</td><td>2016<br>2010-2011</td><td>1 260<br>5 264</td><td>880/800</td><td>880/880</td><td></td><td>X</td></tr><tr><td>ICCDetector [44]</td><td>GP</td><td>2014</td><td>12 026</td><td>DB</td><td>2010-2012</td><td>5 560</td><td>15 561 (mixed)</td><td>1 203/526</td><td>X</td><td>X</td></tr><tr><td>Androdiyalysis [43]</td><td>GP</td><td>2016</td><td>1 846</td><td>DB</td><td>2010-2012</td><td>5 560</td><td>10-fold cross-validation</td><td>5 560/1 846</td><td>X</td><td>X</td></tr><tr><td>MaMaDroid [58]</td><td>PD<br>GP</td><td>2013-2014<br>2016</td><td>8 447<br>2 843</td><td>DB<br>VS<br>"<br>"<br>"</td><td>2010-2012<br>2013<br>2014<br>2015<br>2016</td><td>5 560<br>6 228<br>15 417<br>5 314<br>2 974</td><td>10-fold cross-validation</td><td>DB/PD<br>2013/PD<br>2014/PD<br>2014/GP<br>2015/GP<br>2016/GP</td><td>X</td><td></td></tr><tr><td>StormDroid [59]</td><td>GP</td><td>2015</td><td>4 350</td><td>MSL<br>GM<br>CT<br>CT</td><td>2015<br>2010-2011<br>2012-2015<br>2012-2015</td><td>2 000<br>1 260<br>360<br>60</td><td>(900+500+100)<br>1500</td><td>(600+300+100)<br>1000</td><td></td><td>X</td></tr><tr><td>Plndroid [4]</td><td>GP<br>AppBrain<br>F-Droid<br>Getjar<br>Aptoid<br>Mobango</td><td>up to 2016</td><td>445</td><td>DB<br>GM<br>VT<br>TZ[60]<br>MS[61]<br>VS</td><td>2010-2012<br>1 000<br>70<br>70<br>25<br>25</td><td>100<br>1 000<br>70<br>70<br>25</td><td>80% (from 1300/445 total)</td><td>20% (from 1300/445 total)</td><td>X</td><td>X</td></tr><tr><td>Vinod et al. [5]</td><td>GP (D<sub>1</sub>, D<sub>2</sub>, D<sub>3</sub>, D<sub>4</sub>, D<sub>5</sub>)<br>Baidu (D<sub>1</sub>, D<sub>2</sub>)<br>Koodous(D<sub>1</sub>, D<sub>2</sub>)(1,514)<br>1Mobile (D<sub>1</sub>, D<sub>2</sub>)<br>9apps (D<sub>1</sub>, D<sub>2</sub>)</td><td>2016</td><td>3 130</td><td>DB (D<sub>1</sub>, D<sub>2</sub>)<br>Koodous (D<sub>2</sub>)<br>(62) (D<sub>4</sub>)<br>GM (D<sub>5</sub>)</td><td>2010-2012<br>2011-2015<br>2010-2011</td><td>1 514<br>1 000<br>1 206</td><td>10-fold cross-validation</td><td>D<sub>1</sub> 2 520/3 130<br>D<sub>2</sub> 1 514/3 130<br>D<sub>3</sub> 2 474/2 474<br>D<sub>4</sub> 1 000/14 000<br>D<sub>5</sub> 1 206/1 206</td><td>X</td><td>X</td></tr><tr><td>MalDozer [6]</td><td>GP</td><td>2016</td><td>38 000</td><td>GM<br>DB<br>Own<br>"Merged"</td><td>2010-2011<br>2010-2012</td><td>1 000<br>5 500<br>20 000<br>33 000</td><td>Each malware dataset individually/37 627</td><td></td><td>X</td><td>X</td></tr><tr><td>IntelliAV [63]</td><td>VT<br>"</td><td>2011-2016<br>2017</td><td>10 058<br>2 898</td><td>VT<br>"</td><td>2011-2016<br>2017</td><td>9 664<br>2 311</td><td>9 664/10 058<br>2 311/2 898</td><td></td><td>X</td><td></td></tr><tr><td>Martín et al. [64]</td><td>GP</td><td>2015</td><td>~ 49 000</td><td>GP</td><td>2015</td><td>69 000</td><td>10-fold cross validation with 9 subsets of 50 000 samples, varying malware concentration from 2%, 25%, 50% of the whole subset, and the number of antivirus alerts: 1-AV, 2-AV, 4-AV (4-AV, 50% subset only contains 36 000)</td><td></td><td>X</td><td></td></tr><tr><td>HinDroid [65]</td><td>Comodo Cloud Security Center</td><td>2017</td><td>920 (D<sub>1</sub>, Tr)<br>198 (D<sub>1</sub>, Te)<br>15 000 (D<sub>2</sub>)</td><td>Comodo Cloud Security Center</td><td>2017</td><td>914 (D<sub>1</sub>, Tr)<br>302 (D<sub>1</sub>, Te)<br>15 000 (D<sub>2</sub>)</td><td>914/920 (D<sub>1</sub>)<br>15 000/15 000 (D<sub>2</sub>)</td><td>500 (D<sub>1</sub>)<br>10-fold CV (D<sub>2</sub>)</td><td></td><td></td></tr><tr><td>RevealDroid [66]</td><td>GP (AZ)</td><td>2016</td><td>24 600</td><td>VS<br>DB</td><td>2016<br>2010-2012</td><td>22 592<br>5 538</td><td>10-fold CV Training-Test with date separation (Training date &lt; Test date)</td><td></td><td>X</td><td>X</td></tr><tr><td>Demontis et al. [67]</td><td>DB</td><td>2010-2012</td><td>121 329</td><td>DB<br>Mod DB and CT</td><td>2010-2012<br>2010-2012, 2016</td><td>5 615 10<br>500</td><td>10 runs, where 30 000 for training, and the rest for test</td><td></td><td>X</td><td>Too Old</td></tr><tr><td>McLaughlin et al. [68]</td><td>GM (GP)<br>McAfee Labs<br>"</td><td>2010-2011<br>2016<br>2016<br>"</td><td>863<br>3 627<br>9 268</td><td>GM<br>McAfee Labs<br>"</td><td>2010-2011<br>2016<br>2016<br>"</td><td>1 260<br>2 475<br>9 902</td><td>10-fold CV, with same GW/MW ratio In training and test</td><td></td><td>X</td><td></td></tr><tr><td>Alzaylaee et al. [69]</td><td>Intel Security (McAfee Labs)</td><td>2016<br>"</td><td>1 222</td><td>GM</td><td>2010-2011</td><td>1 222</td><td>2/3 of GW/MW<br>1/3 of GW/MW</td><td></td><td>X</td><td>X</td></tr><tr><td>Melis et al. [70]</td><td>DB</td><td>2010-2012</td><td>121 329</td><td>DB</td><td>2010-2012</td><td>5 615</td><td>5 runs, where 60 000 units are randomly selected for training, And the rest for testing<br>CE1: 70% of DR + GP1<br>CE2: 70% of VS + GP2<br>CE3: DR + GP1<br>WEX: DR + VS + GP1 + GP2<br>MCLEX: 1/2 of MYST + GP1</td><td></td><td>X</td><td>Too Old</td></tr><tr><td>MKLDroid [71]</td><td>GP1<br>GP2<br>AndroidDrawer (AD)<br>AnZhi (AZ)<br>AppsApk (AA)<br>F-Droid (FD)<br>SlideMe (SM)</td><td>2012-2014<br>2012-2014<br>2013-2016<br>2013-2016<br>2013-2016<br>2013-2016<br>2013-2016<br>2013-2016</td><td>5 000<br>10 000<br>2 399<br>3 027<br>2 481<br>1 007<br>5 770</td><td>DR<br>VS<br>MYST</td><td>2010-2012<br>2013-2014<br>2015</td><td>5 560<br>24 317<br>3 000</td><td>CE1: 30% of DR + GP1<br>CE2: 30% of VS + GP2<br>CE3: VS + GP2<br>WEX: AD + AZ + AA + FD + SM<br>MCLEX: 1/2 of MYST + GP1</td><td></td><td>X</td><td>X</td></tr><tr><td>Li et al. [72]</td><td>apkpure<br>360<br>HKUST</td><td>2019</td><td>16 753</td><td>DB<br>AMD</td><td>2010-2011<br>2010-2016</td><td>5 560<br>16 753</td><td>5-fold CV</td><td></td><td>X</td><td>X</td></tr><tr><td>SMART [73]</td><td>GP<br>GP</td><td>2015<br>2016<br>"</td><td>5 600<br>3 536</td><td>DB<br>DB</td><td>2010-2012<br>2010-2013</td><td>5 560<br>3 567</td><td>10-fold CV<br>10-fold CV</td><td></td><td>X</td><td>X</td></tr><tr><td>Xiao et al. [74]</td><td>GP (AZ)<br>"</td><td>2016-2017<br>2014-2015<br>2012-2013<br>2009-2011</td><td>5 346<br>6 545<br>5 035<br>439<br>2010-2012</td><td>VS, AZ<br>VS, AZ<br>VS, AZ, GM, DB<br>VS, AZ, GM, DB</td><td>2016-2017<br>2014-2015<br>2012-2013<br>2009-2011</td><td>3 450<br>3 190<br>9 084<br>1 254<br>5 560</td><td>70% of older apps<br>30% of newer apps</td><td></td><td>X</td><td></td></tr><tr><td>DroidCat [75]</td><td>"</td><td>"</td><td>"</td><td>"</td><td>"</td><td>"</td><td>80% training and 20% testing<br>Everything divided in 10 parts, Then 10-CV for each part<br>10-fold CV</td><td></td><td>Too Old</td><td></td></tr><tr><td>Rana et al. [76]</td><td>DB</td><td>2016<br>"</td><td>5 560</td><td>DB</td><td>2010-2012</td><td>5 560</td><td>80% training and 20% testing<br>Everything divided in 10 parts, Then 10-CV for each part<br>10-fold CV</td><td></td><td>Too Old</td><td></td></tr><tr><td>Ma et al. [77]</td><td>AZ</td><td>"</td><td>10 010</td><td>AMD</td><td>2010-2016</td><td>10 683</td><td>5-fold CV</td><td></td><td>?</td><td>?</td></tr><tr><td>Huang et al. [78]</td><td>GP</td><td>2018<br>"</td><td>3 312</td><td>VS<br>"</td><td>2010-2012</td><td>3 312</td><td>10-fold CV</td><td></td><td>?</td><td>N/A</td></tr><tr><td>Liu et al. [79]</td><td>DB</td><td>2010-2012</td><td>123 453</td><td>DB</td><td>2010-2012</td><td>5 560</td><td>83 784/106 912<br>13 313/4 433</td><td></td><td>X</td><td>Too Old</td></tr><tr><td>HG-Learning [80]</td><td>Tencent Security<br>Lab (Tr)<br>" (Te)</td><td>2018<br>2018<br>"</td><td>83 784<br>13 313</td><td>Tencent Security<br>Lab (Tr)<br>" (Te)</td><td>2018<br>2018<br>"</td><td>106 912<br>4 433</td><td>2 974 applications tested</td><td></td><td>X</td><td></td></tr><tr><td>BRIDEMAID [81]</td><td>GP</td><td>2016</td><td>9 804</td><td>DB<br>GM<br>CT</td><td>2010-2012<br>2010-2011 2016</td><td>2 794</td><td>2 974 applications tested</td><td></td><td>X</td><td>X</td></tr></tbody></table>

Next, columns “Training set” and “Test set” correspond to the number of applications used in the respective sets. The notation “(M/G)” reports the Malware/Goodware for each experiment. The use of “k-fold cross validation” will be specified in these columns. Any other special cases are reported in these two columns. Finally, columns “Class imbalance” and “Time Incoherence” indicate whether the datasets used in the experiments suffer from these irregularities, introduced in Section III.

REFERENCES

[1] D. Arp, M. Spreitzenbarth, M. Hübner, H. Gascon, and K. Rieck, “DREBIN: Effective and explainable detection of Android malware in your pocket,” in *Proc. Netw. Distrib. Syst. Secur. Symp.*, Reston, VA, USA, 2014, pp. 720–731.

[2] Y. Zhou and X. Jiang, “Dissecting Android malware: Characterization and evolution,” in *Proc. IEEE Symp. Secur. Privacy*, New York, NY, USA, May 2012, pp. 95–109.

[3] F. Wei, Y. Li, S. Roy, X. Ou, and W. Zhou, “Deep ground truth analysis of current Android malware,” in *Detection of Intrusions and Malware, and Vulnerability Assessment*. Cham, Switzerland: Springer, 2017, pp. 252–276.
[4] F. Idrees, M. Rajarajan, M. Conti, T. M. Chen, and Y. Rahulamathavan, "Pindroid: A novel Android malware detection system using ensemble learning methods," *Comput. Secur.*, vol. 68, pp. 36–46, Jul. 2017.

[5] P. Vinod, A. Zemmari, and M. Conti, "A machine learning based approach to detect malicious Android apps using discriminant system calls," *Future Gener. Comput. Syst.*, vol. 94, pp. 333–350, May 2019.

[6] E. B. Karbab, M. Debbabi, A. Derhab, and D. Mouheb, "MalDozer: Automatic framework for Android malware detection using deep learning," *Digit. Invest.*, vol. 24, pp. S48–S59, Mar. 2018.

[7] F. Pendlebury, F. Pierazzi, R. Jordaney, J. Kinder, and L. Cavallaro, "TESSERACT: Eliminating experimental bias in malware classification across space and time," in *Proc. 28th USENIX Secur. Symp. Berkeley, CA, USA: USENIX Association*, 2019, pp. 729–746.

[8] K. Allix, T. F. Bissyandé, J. Klein, and Y. Le Traon, "AndroZoo: Collecting millions of Android apps for the research community," in *Proc. 13th Int. Conf. Mining Softw. Repositories*, New York, NY, USA, May 2016, pp. 468–471.

[9] M. Lindorfer, M. Neugschwandtner, L. Weichselbaum, Y. Fratantonio, V. V. D. Veen, and C. Platzer, "ANDRUBIS—1,000,000 apps later: A view on current Android malware behaviors," in *Proc. 3rd Int. Workshop Building Anal. Datasets Gathering Exper. Returns Secur. (BADGERS)*, New York, NY, USA, Sep. 2014, pp. 3–17.

[10] N. Kiss, J.-F. Lalande, M. Leslous, and V. V. T. Tong, "Kharon dataset: Android malware under a microscope," in *Proc. Workshop, Learn. Authoritative Secur. Exp. Results*. Berkeley, CA, USA: USENIX Association, 2016, pp. 1–12.

[11] A. H. Lashkari, A. A. F. Kadir, L. Taheri, and A. A. Ghorbani, "Toward developing a systematic approach to generate benchmark Android malware datasets and classification," in *Proc. 52nd Int. Carnahan Conf. Secur. Technol. (ICCST)*, New York, NY, USA, Oct. 2018, pp. 1–7.

[12] VirusShare. (2021). *VirusShare.com—Because Sharing is Caring*. [Online]. Available: https://virusshare.com/

[13] K. Aktas and S. Sen, "Updroid: Updated Android malware and its familial classification," in *Proc. Nordic Conf. Secure IT Syst*. Cham, Switzerland: Springer, 2018, pp. 352–368.

[14] Koodous Team. (2021). *Koodous Project*. [Online]. Available: https://koodous.com/about

[15] Hispasec Sistemas. (2021). *Virus Total*. [Online]. Available: https://www.virustotal.com

[16] S. Dong et al., "Understanding Android obfuscation techniques: A large-scale investigation in the wild," in *Security and Privacy in Communication Networks*. Cham, Switzerland: Springer, 2018, pp. 172–192.

[17] H. Wang et al., "Beyond Google play: A large-scale comparative study of Chinese Android app markets," in *Proc. Internet Meas. Conf.*, New York, NY, USA, Oct. 2018, pp. 293–307.

[18] M. Grace, Y. Zhou, Q. Zhang, S. Zou, and X. Jiang, "RiskRanker: Scalable and accurate zero-day Android malware detection," in *Proc. 10th Int. Conf. Mobile Syst., Appl., Services (MobiSys)*, New York, NY, USA, 2012, pp. 281–294.

[19] H. Gonzalez, A. A. Kadir, N. Stakhanova, A. J. Alzahrani, and A. A. Ghorbani, "Exploring reverse engineering symptoms in Android apps," in *Proc. 8th Eur. Workshop Syst. Secur.*, New York, NY, USA, Apr. 2015, pp. 1–7.

[20] P. Battista, F. Mercaldo, V. Nardone, A. Santone, and C. A. Visaggio, "Identification of Android malware families with model checking," in *Proc. 2nd Int. Conf. Inf. Syst. Secur. Privacy*, Rome, Italy, 2016, pp. 542–547.

[21] J. Crussell, C. Gibler, and H. Chen, "AnDarwin: Scalable detection of semantically similar Android applications," in *Proc. Comput. Secur. (ESORICS)*. Berlin, Germany: Springer, 2013, pp. 182–199.

[22] L. Li et al., "IccTA: Detecting inter-component privacy leaks in Android apps," in *Proc. IEEE/ACM 37th IEEE Int. Conf. Softw. Eng.*, New York, NY, USA, May 2015, pp. 280–291.

[23] G. M. Weiss and F. Provost, "The effect of class distribution on classifier learning: An empirical study," Dept. Comput. Sci., Rutgers Univ., Fredericton, NM, Canada, Tech. Rep. ML-TR-44, 2001.

[24] S. Susan and A. Kumar, "The balancing trick: Optimized sampling of imbalanced datasets—A brief survey of the recent state of the art," *Eng. Rep.*, vol. 3, no. 4, 2020, Art. no. m12298.

[25] D. Pfaff, M.-T. Walter, and M. Backes, "Code clones considered harmful? Quantifying and exploiting the effects of code clones in static malware classifiers for Javascript," in *Proc. 14th Int. Conf. Malicious Unwanted Softw.*, 2019, pp. 111–118.

[26] I. Tomek, "Two modifications of CNN," *IEEE Trans. Syst., Man, Cybern.*, vol. SMC-6, pp. 769–772, 1976.

[27] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, "SMOTE: Synthetic minority over-sampling technique," *J. Artif. Intell. Res.*, vol. 16, pp. 321–357, Jun. 2002.

[28] F. Pan, W. Wang, A. K. H. Tung, and J. Yang, "Finding representative set from massive data," in *Proc. 5th IEEE Int. Conf. Data Mining (ICDM)*, New York, NY, USA, Jun. 2005, pp. 338–345.

[29] X. Fu and H. Cai, "On the deterioration of learning-based malware detectors for android," in *Proc. IEEE/ACM 41st Int. Conf. Softw. Eng., Companion (ICSE-Companion)*, May 2019, pp. 272–273.

[30] H. Cai and J. Jenkins, "Towards sustainable Android malware detection," in *Proc. 40th Int. Conf. Softw. Eng., Companion*, New York, NY, USA, May 2018, pp. 350–351.

[31] H. Cai, "Assessing and improving malware detection sustainability through app evolution studies," *ACM Trans. Softw. Eng. Methodol.*, vol. 29, no. 2, pp. 1–28, Apr. 2020.

[32] H. Cai, "Embracing mobile app evolution via continuous ecosystem mining and characterization," in *Proc. IEEE/ACM 7th Int. Conf. Mobile Softw. Eng. Syst.*, New York, NY, USA, Jul. 2020, pp. 31–35.

[33] X. Zhang et al., "Enhancing state-of-the-art classifiers with API semantics to detect evolved Android malware," in *Proc. ACM SIGSAC Conf. Comput. Commun. Secur.*, New York, NY, USA, Oct. 2020, pp. 757–770.

[34] K. Xu, Y. Li, R. Deng, K. Chen, and J. Xu, "DroidEvolver: Self-evolving Android malware detection system," in *Proc. IEEE Eur. Symp. Secur. Privacy (EuroSP)*, Jun. 2019, pp. 47–62.

[35] K. Allix, T. F. Bissyandé, J. Klein, and Y. Le Traon, "Are your training datasets yet relevant?" in *Engineering Secure Software and Systems. Cham, Switzerland: Springer, 2015*, pp. 51–67.

[36] W. G. Cochran, *Sampling Techniques*, 3rd ed. New York, NY, USA: Wiley, 1977.

[37] R. D. De Veaux, P. F. Velleman, and D. E. Bock, *Stats Data and Models*. London, U.K.: Pearson, 2016.

[38] R. V. Hogg, E. A. Tanis, and D. L. Zimmerman, *Probability and Statistical Inference*. London, U.K.: Pearson, 2015.

[39] R. Bardenet and O.-A. Maillard, "Concentration inequalities for sampling without replacement," *Bernoulli*, vol. 21, no. 3, pp. 1361–1385, 2015.

[40] O. J. Dunn, "Multiple comparisons among means," *J. Amer. Statist. Assoc.*, vol. 56, no. 293, pp. 52–64, 1961.

[41] L. Apvrille and A. Apvrille, "Pre-filtering mobile malware with heuristic techniques," in *Proc. 2nd Int. Symp. Res. Grey-Hat Hacking (GreHack)*, 2013, pp. 44–59.

[42] K. Pearson, "X. On the criterion that a given system of deviations from the probable in the case of a correlated system of variables is such that it can be reasonably supposed to have arisen from random sampling," *Philos. Mag.*, vol. 50, no. 320, pp. 157–175, 1900.

[43] A. Feizollah, N. B. Anuar, R. Salleh, G. Suarez-Tangil, and S. Furnell, "AndroDialysis: Analysis of Android intent effectiveness in malware detection," *Comput. Secur.*, vol. 65, pp. 121–134, Mar. 2017.

[44] K. Xu, Y. Li, and R. H. Deng, "ICCDetector: ICC-based malware detection on Android," *IEEE Trans. Inf. Forensics Security*, vol. 11, no. 6, pp. 1252–1264, Jun. 2016.

[45] T. C. Miranda, P.-F. Gimenez, and J.-F. Lalande. (2021). *Dada: Debiased Android Datasets*. [Online]. Available: https://dx.doi.org/10.21227/0frv-zb46

[46] W. Li, X. Fu, and H. Cai, "AndroCT: Ten years of app call traces in Android," in *Proc. IEEE/ACM 18th Int. Conf. Mining Softw. Repositories (MSR)*, May 2021, pp. 570–574.

[47] M. Fan et al., "Android malware familial classification and representative sample selection via frequent subgraph analysis," *IEEE Trans. Inf. Forensics Security*, vol. 13, no. 8, pp. 1890–1905, Aug. 2018.

[48] E. Fix and J. L. Hodges, Jr., "Discriminatory analysis. Nonparametric discrimination: Consistency properties," *Int. Stat. Rev.*, vol. 57, no. 3, pp. 238–247, 1989.

[49] W.-Y. Loh, "Classification and regression trees," *Wiley Interdiscipl. Rev., Data Mining Knowl. Discovery*, vol. 1, no. 1, pp. 14–23, 2011.

[50] T. K. Ho, "Random decision forests," in *Proc. 3rd Int. Conf. Document Anal. Recognit.*, vol. 1, Aug. 1995, pp. 278–282.

[51] D. J. Hand and K. Yu, "Idiot's Bayes—Not so stupid after all?" *International Stat. Rev.*, vol. 69, no. 3, pp. 385–398, 2001.

[52] Y. Freund and R. E. Schapire, "A decision-theoretic generalization of on-line learning and an application to boosting," *J. Comput. Syst. Sci.*, vol. 55, no. 1, pp. 119–139, Aug. 1997.

[53] F. Pedregosa et al., "Scikit-learn: Machine learning in Python," *J. Mach. Learn. Res.*, vol. 12, pp. 2825–2830, Oct. 2011.
[54] H. Cai and B. Ryder, "A longitudinal study of application structure and behaviors in android," *IEEE Trans. Softw. Eng.*, vol. 47, no. 12, pp. 2934-2955, Dec. 2021.

[55] H. Cai, X. Fu, and A. Hamou-Lhadj, "A study of run-time behavioral evolution of benign versus malicious apps in android," *Inf. Softw. Technol.*, vol. 122, Jun. 2020, Art. no. 106291.

[56] A. Saracino, D. Sgandurra, G. Dini, and F. Martinelli, "MADAM: Effective and efficient behavior-based Android malware detection and prevention," *IEEE Trans. Dependable Secure Comput.*, vol. 15, no. 1, pp. 83-97, Jan. 2018.

[57] Z. Yuan, Y. Lu, and Y. Xue, "Droiddetector: Android malware characterization and detection using deep learning," *Tsinghua Sci. Technol.*, vol. 21, no. 1, pp. 114-123, Feb. 2016.

[58] E. Mariconti, L. Onwuzurike, P. Andriotis, E. De Cristofaro, G. Ross, and G. Stringhini, "MaMaDroid: Detecting Android malware by building Markov chains of behavioral models," in *Proc. Netw. Distrib. Syst. Secur. Symp.*, San Diego, CA, USA, 2017, pp. 297-311.

[59] S. Chen, M. Xue, Z. Tang, L. Xu, and H. Zhu, "StormDroid: A streamlined machine learning-based system for detecting Android malware," in *Proc. 11th ACM Asia Conf. Comput. Commun. Secur.*, May 2016, pp. 377-388.

[60] Yuval Nativ YTISF. *TheZoo—A Live Malware Repository Web Site*. Accessed: Mar. 15, 2022. [Online]. Available: http://ytisf.github.io/theZoo/

[61] Silas Cutler. *MalShare*. Accessed: Mar. 15, 2022. [Online]. Available: https://malshare.com/about.php

[62] S. Bhandari, R. Panihar, S. Naval, V. Laxmi, A. Zemmari, and M. S. Gaur, "SWORD: Semantic aWare Android malwaRe detector," *J. Inf. Secur. Appl.*, vol. 42, pp. 46-56, Oct. 2018.

[63] M. Ahmadi, A. Sotgiu, and G. Giacinto, "IntelliAV: Toward the feasibility of building intelligent anti-malware on Android devices," in *Machine Learning and Knowledge Extraction*. Cham, Switzerland: Springer, 2017, pp. 137-154.

[64] I. Martín, J. A. Hernández, A. Muñoz, and A. Guzmán, "Android malware characterization using metadata and machine learning techniques," *Secur. Commun. Netw.*, vol. 2018, pp. 5749481:1-5749481:11, Jul. 2018.

[65] S. Hou, Y. Ye, Y. Song, and M. Abdulhayoglu, "HinDroid: An intelligent Android malware detection system based on structured heterogeneous information network," in *Proc. 23rd ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining*, New York, NY, USA, Aug. 2017, pp. 1507-1515.

[66] J. Garcia, M. Hammad, and S. Malek, "Lightweight, obfuscation-resistant detection and family identification of Android malware," *ACM Trans. Softw. Eng. Methodol.*, vol. 26, no. 3, pp. 1-29, Jan. 2018.

[67] A. Demontis et al., "Yes, machine learning can be more secure! A case study on Android malware detection," *IEEE Trans. Dependable Secure Comput.*, vol. 16, no. 4, pp. 711-724, Jul. 2019.

[68] N. McLaughlin et al., "Deep Android malware detection," in *Proc. 7th ACM Conf. Data Appl. Secur. Privacy*, New York, NY, USA, 2017, pp. 301-308.

[69] M. K. Alzaylaee, S. Y. Yerima, and S. Sezer, "EMULATOR vs REAL PHONE: Android malware detection using machine learning," in *Proc. 3rd ACM Int. Workshop Secur. Privacy Anal.*, Scottsdale, AZ, USA, Mar. 2017, pp. 65-72.

[70] M. Melis, D. Maiorca, B. Biggio, G. Giacinto, and F. Roli, "Explaining black-box Android malware detection," in *Proc. 26th Eur. Signal Process. Conf. (EUSIPCO)*, Sep. 2018, pp. 524-528.

[71] A. Narayanan, M. Chandramohan, L. Chen, and Y. Liu, "A multi-view context-aware approach to Android malware detection and malicious code localization," *Empirical Softw. Eng.*, vol. 23, no. 3, pp. 1222-1274, 2018.

[72] C. Li, K. Mills, D. Niu, R. Zhu, H. Zhang, and H. Kinawi, "Android malware detection based on factorization machine," *IEEE Access*, vol. 7, pp. 184008-184019, 2019.

[73] G. Meng, Y. Xue, Z. Xu, Y. Liu, J. Zhang, and A. Narayanan, "Semantic modelling of Android malware for effective malware comprehension, detection, and classification," in *Proc. 25th Int. Symp. Softw. Test. Anal.*, New York, NY, USA, Jul. 2016, pp. 306-317.

[74] X. Xiao, S. Zhang, F. Mercaldo, G. Hu, and A. K. Sangaiah, "Android malware detection based on system call sequences and LSTM," *Multi-media Tools Appl.*, vol. 78, no. 4, pp. 3979-3999, 2019.

[75] H. Cai, N. Meng, B. G. Ryder, and D. Yao, "DroidCat: Effective Android malware detection and categorization via app-level profiling," *IEEE Trans. Inf. Forensics Security*, vol. 14, no. 6, pp. 1455-1470, Nov. 2019.

[76] M. S. Rana, C. Gudla, and A. H. Sung, "Android malware detection using stacked generalization," in *Proc. 27th Int. Conf. Softw. Eng. Data Eng.*, 2018, pp. 15-19.

[77] Z. Ma, H. Ge, Y. Liu, M. Zhao, and J. Ma, "A combination method for Android malware detection based on control flow graphs and machine learning algorithms," *IEEE Access*, vol. 7, pp. 21235-21245, 2019.

[78] N. Huang, M. Xu, N. Zheng, T. Qiao, and K.-K.-R. Choo, "Deep Android malware classification with API-based feature graph," in *Proc. 18th IEEE Int. Conf. Trust, Secur. Privacy Comput. Commun.*, 13th IEEE Int. Conf. Big Data Sci. Eng. (TrustCom/BigDataSE), Aug. 2019, pp. 296-303.

[79] X. Liu, X. Du, X. Zhang, Q. Zhu, H. Wang, and M. Guizani, "Adversarial samples on Android malware detection systems for IoT systems," *Sensors*, vol. 19, no. 4, p. 974, Feb. 2019.

[80] Y. Ye et al., "Out-of-sample node representation learning for heterogeneous graph in real-time Android malware detection," in *Proc. 28th Int. Joint Conf. Artif. Intell.*, Aug. 2019, pp. 4150-4156.

[81] F. Martinelli, F. Mercaldo, and A. Saracino, "BRIDEMAID: An hybrid tool for accurate detection of Android malware," in *Proc. ACM Asia Conf. Comput. Commun. Secur.*, New York, NY, USA, Apr. 2017, pp. 899-901.