Program analysis and machine learning techniques for mobile security

The rapid rise of the smartphone is mainly due to the availability of mobile applications (apps) that provide a wide range of functionalities. Statistics from International Data Corporation (IDC) has shown that Android is the most popular smartphone platform over the past few years. Due to the openness and low threshold of entering the Android app market, it has the largest collection of mobile apps. Unfortunately, it’s popularity also attracts the unwanted attention of cyber-criminals. McAfee has reported that the number of threats families found in Google Play has increased by 30% in the year 2017, resulting in more than 4 thousand mobile threat families and variants in their sample database. Over the past few years, concerns have been raised with respect to the increasing number of malicious and clone apps infiltrating the Android markets. Android malware may perform a range of malicious activities e.g., leakage of sensitive information or encrypt data and lock the user out of the compromised device for ransom. On the other hand, clone apps are repackaged apps that steal revenue from the original developer of the popular apps. Despite the advances in mobile security, the detection of malicious and clone mobile apps is non-trivial and remains an open problem. In order to differentiate these adversary apps from benign apps, an in-depth understanding of the apps is required. However, due to the arms race between the adversary apps and the detection algorithms, the adversary apps are constantly evolving and becoming more sophisticated. Hence, new and more effective algorithms are imperative. In this thesis, we address the problem of Android security by presenting new program analysis and machine learning approaches we have developed for the vetting of Android apps. The achievements made in this thesis are as follow: 1. We develop a novel approach to detect clone in Android apps by analyzing runtime user interface (UI) information. We take advantage of the unique multiple entry points characteristic of Android apps, to collect the UI information ii efficiently and avoid the need for the tedious process of having to create multiple sets of relevant inputs to navigate through the entire app. An inherent advantageous characteristic of our approach is that is it resilient to code obfuscation since semantics preserving obfuscation techniques do not have any influence on runtime behaviors. The evaluation of the proposed approach was performed on a set of real-world Android apps and the results reveal that it can achieve low false positive rate and false negative rate. We further analyze the results and observe that our approach is effective in detecting different types of repackaging attacks. 2. Third-party libraries (TPLs) are commonly found in Android apps and various reports on the privacy risks and security threats that are brought about by them have surfaced. In addition, there have also been multiple complains of TPLs hindering various program analysis tasks, such as clone detection, static taint analysis, etc. Understandably, since TPLs are typically used as it is, it may include an abundance of unnecessary code to the host app and may dilute the features and increase the complexity of the code analysis, thus affecting the accuracy and scalability of the tasks. A typical and straightforward solution for identifying and excluding the TPLs is to match the name of the packages in the app to a whitelist which holds a list of known TPL package names. However, these whitelists are vulnerable against the commonly employed renaming obfuscation technique and given the fast-paced ecosystem, it is also difficult for the whitelist to be exhaustive. Hence, we propose LibSift, a tool which automates the process of identifying TPLs in Android apps. It identifies TPLs by analyzing the package dependencies of the app allowing it to be resilient to most of the typical obfuscation techniques. 3. In recent years, several promising Machine Learning (ML) based Android malware detection approaches that achieve remarkable results have been proposed in the literature. Most of these approaches are built upon the batch learning model, where a common assumption of such model is that the underlying probability distribution of the observed characteristics belonging to the data source (i.e., malware samples) is stationary. However, apart from the arms race, mobile apps are constantly evolving due to several factors such as environmental changes and adding features. These evolutions cause the distribution of the population to change over time. Moreover, in the real-world use case, the malware detection models are trained on the existing dataset and used to predict forthcoming samples that stream in. Consequently, in the face of malware evolution, the detection accuracy of the model in the real-world scenario will degrade over time. We perform iii a systematic study to examine the challenges faced by state-of-the-art ML based Android malware detection approaches in the presence of concept drift. Furthermore, we propose and evaluate the modification to the approaches that may help them to overcome their limitations in handling streaming features, samples, and classes. 4. Traditionally, extensive feature engineering effort is spent to develop a solution for each of the critical Android program analytics tasks to address the multitude of problems have plagued the Android ecosystem. Hence, we aim to build holistic behavioral profiles of Android apps with rich and multi-modal information (e.g., incorporating several semantic views of an app such as API calls, system calls, etc.). Such profiles could be used to address various downstream program analytics tasks such as malware detection, clone detection, and app recommendation etc. Towards this goal, we design a data-driven Representation Learning (RL) framework named apk2vec which incorporates various state-of-the-art RL paradigm such as semisupervised, multiview and hash embedding techniques to automatically generate succinct and versatile representation (aka profile or embedding) for Android apps. In sum, this thesis proposes three methods and one empirical study with suggestions for Android apps analysis. We address four specific issues that plague Android security, namely, app clone detection, third-party library detection, malware detection, and concept drift. We do so through leveraging on techniques such as program analysis, Machine Learning and Deep Learning.