# Using Machine Learning to Detect Vault (Anti-Forensic) Apps
## 5 Conclusions and Future Work  
Detecting and classifying vault applications on mobile devices remains a challenge due to various forensic and legal considerations. While some vault apps may exhibit characteristics similar to malware, many serve legitimate privacy functions. Given the popularity of phones, any approach that is non-invasive and accurate would be of use to law enforcement bodies. This research undertook the reverse-engineering of Android vault and non-vault applications to uncover and extract their distinct properties. We then applied various machine learning algorithms to classify these properties, aiming to validate the effectiveness of a machine learning-driven approach. Given that machine learning thrives on data, we successfully demonstrated that good results can be obtained even with a modest dataset of around 1000 observations. Notably, our analysis revealed that the support vector machine (SVM), multi-layer perceptron (MLP), and random forest classifiers yielded the most promising outcomes. Looking ahead, we plan to test additional algorithms, expand our sample size, and further investigate other Android permissions, paving the way for more robust findings.  
In particular, using unsupervised machine learning algorithms can divulge meaningful insights, and a robust comparison can be made between supervised, semi-supervised, and unsupervised machine learning algorithms. In addition, the usage of vault apps by cyber criminals such as ransomware groups will be investigated to see whether a correlation exists between the successful ransomware attacks and the vault apps. It would also be worthwhile to check the usage of vault apps by online scammers. These future investigations will enhance our research contributions in this area of digital forensics, which is a growing area, as noted by [PERSON] et al. (2018). We will endeavor to integrate our research findings into industrial practices such as law enforcement and criminal justice systems, leading to more impactful and multidisciplinary research.  
We highlight that false positives in machine learning-based detection of vault (anti-forensic) apps pose significant legal and ethical concerns, as wrongfully flagging a legitimate privacy-focused application as malicious could lead to privacy violations, reputational  
\begin{table}
\begin{tabular}{c c c c c} \hline
**Reference** & **Accuracy** & **F1 Score** & **Sample Size** & **Vault Apps** \\ \hline [PERSON] (2018) & 0.880 & - & 2000 & No \\ \hline [PERSON] (2018) & - & 0.997 & 11,000 & No \\ \hline [PERSON] (2018) & 0.933 & - & 102 & Yes \\ \hline [PERSON] (2018) & 0.988 & 0.990 & 24,911 & No \\ \hline Our Approach & 0.981 & 0.981 & 1040 & Yes \\ \hline \end{tabular}
\end{table}
Table 4: Comparison of Our RF classifier with related work.  
Figure 10: Cross-validation plot for RF classifier.  
damage, and legal liability. As a future research direction, mitigating false positives requires enhancing dataset diversity, incorporating explainable AI techniques, implementing adaptive thresholding, and involving human supervision in decision-making. Ensuring a balanced detection approach minimizes wrongful classifications while maintaining the integrity and reliability of forensic investigations.  
Conceptualisation, M.N.J.; methodology, M.N.J.; implementation, M.N.J.; writing--original draft preparation, M.N.J.; writing--review and editing, M.N.J., W.Y. and M.A. All authors have read and agreed to the published version of the manuscript.  
This research received no external funding.  
All data are available on request.  
We thank [PERSON], [PERSON], [PERSON], [PERSON], and [PERSON] for their assistance with some of the data collection, data curation, pre-processing, and aspects of the algorithms.  
The authors declare no conflicts of interest.