  In recent years we have witnessed an increase in cyber threats and malicious
software attacks on different platforms with important consequences to persons
and businesses. It has become critical to find automated machine learning
techniques to proactively defend against malware. Transformers, a category of
attention-based deep learning techniques, have recently shown impressive
results in solving different tasks mainly related to the field of Natural
Language Processing (NLP). In this paper, we propose the use of a Transformers'
architecture to automatically detect malicious software. We propose a model
based on BERT (Bidirectional Encoder Representations from Transformers) which
performs a static analysis on the source code of Android applications using
preprocessed features to characterize existing malware and classify it into
different representative malware categories. The obtained results are promising
and show the high performance obtained by Transformer-based models for
malicious software detection.
