Android malware detection: An in-depth investigation of the impact of theuse of imbalance datasets on the efficiency of machine learning models.
ZAKARIA SAWADOGO ˚: ,JEAN-MARIE DEMBELE: , GERVAIS MENDY ˚ ,SAMUEL OUYA ˚ .
˚ LITA(Laboratory of Computer Science, Telecommunications and Applications), Cheikh Anta Diop University of Dakar, Senegal: LANI(Laboratory of Numerical Analysis and Computer Science), Gaston Berger University,Senegal{sawadogo.zakaria,jean-marie.dembele}@ugb.edu.sn, {gervais.mendy,samuel.ouya}@ucad.edu.sn

Abstract—Machine learning techniques have become an essential part of research into the detection and classificationof malicious applications. There are several approaches oralgorithms that learn from existing data and predict classes.
Machine learning principles recommend a balance of classes
in the training dataset, but the reality on the ground is quite
different. The majority of datasets used for malicious application
detection are unbalanced. Class imbalance degrades classifier
performance, so it is a common problem in classification tasks.
This observation is much more significant in the field of Android
malware detection and classification. There is little work to ourknowledge on the effects of unbalanced datasets in the fieldof Android malware detection. Our contribution focuses on theimpact of unbalanced datasets on the performance of different
algorithms and the relevance of using evaluation metrics inAndroid malware detection. And the state of the databases fromwhich researchers typically draw datasets. We show that formalicious application detection, some classification algorithms are
not suitable for unbalanced datasets. We also prove that someof the most widely used performance evaluation metrics in theliterature (Accuracy, Precision, Recall) are not very well suited tounbalanced datasets. On the other hand, the metrics (BalancedAccuracy, Geometric mean) are more suitable. These results wereobtained by evaluating the performances of eleven classification
algorithms as well as the adequacy of the different evaluation
metrics (Accuracy, Recall, Precision, F1 score, Balanced accuracy, Matthews corrcoef, Geometric mean, Fowlkes mallows).
Also not all databases are accessible by researchers and many ofthese databases are not updated.
Index Terms—imbalanced dataset, Android malware detection,Malware classification, Artificial intelligence, Machine learning.

Zakaria SAWADOGO received
his Master II degree in software
engineering and information systems from Joseph Ki-Zerbo University in Burkina Faso. He iscurrently a researcher in cybersecurity including artificial intelligence and is affiliated with
the Laboratoire d’Informatique,de Télécommunications et Applications ( LITA) of the University
Cheikh Anta Diop of Dakar and the Laboratoire d’Analyse
Numérique et d’Informatique ( LANI) of the University Gaston Berger of Saint Louis. His research interests include
mobile security, system security and artificial intelligence. Heis a member of the IEEE.

Pr. Gervais MENDY is a
researcher-scientist at the ESP
polytechnic school at UCAD university where he was head of theIT department from 2012 to 2016.
Holder of a PhD in Computer
Science from Paris-Sud XI University. His research interests arein Computer Combinatory, Social
Network Analysis, Internet of Things (IoT), Artificial Intelligence and IT Security.He is a member of the LITA Laboratoryof UCAD.
Pr. Jean-Marie DEMEBELE is
Assimilated Professor in Computer Science at the UGB, ex Director of the UFR SAT and member of the UMI UMMISCO Senegal. He has to his credit, several
scientific publications in international peer-reviewed journals. His
research interests include artificial
intelligence, agent-based modeling, dynamical systems, evolutionary algorithms, genetic regulatory networks, machinelearning
Pr. Samuel Ouya is currently
the director of the LITA laboratory at the UCAD. He was from2013 to May 2017 the first Director of Infrastructure and Information System of the first virtual university of Senegal (UVS).
Holder of a thesis in Applied
Mathematics from the Gaston Berger University of SaintLouis Senegal and a Telecommunications Thesis from theUCAD university in Dakar-Senegal, he is interested in he isinterested in Applications of innovative telecom services tovirtual organizations.