# Using Machine Learning to Detect Vault (Anti-Forensic) Apps
## 3 Method
Default parameters were used for the PCA. To evaluate the effectiveness of the proposed method, we used the TP, TN, FP, and FN standard values to calculate the metrics, these are as follows:  
True positive (TP): indicates that a vault app was correctly classified and labeled as a vault app. True negative (TN): indicates that a non-vault app was correctly classified and labeled as a non-vault app. False positive (FP): indicates that a non-vault app was incorrectly classified and labeled as a vault app. False negative (FN): indicates a vault app sample was incorrectly classified and labeled as a non-vault app. Based on the above, we define the metrics as follows. \[\text{Accuracy}=(\text{TP}+\text{TN})/(\text{TP}+\text{FP}+\text{TN}+\text{ FN})\]  
\[\text{F}_{1}\text{score}=\text{TP}/(\text{TP}+\text{\text{\  
* Vault App: A vault application may request READ_EXTERNAL_STORAGE and WRITE_EXTERNAL_STORAGE permissions to access and conceal user files. These permissions allow the app to hide sensitive data from standard file managers or gallery applications, making it difficult to detect stored content. * Non-Vault App: A gallery app might also request these permissions to display and manage user images, but without any concealment. The requested permissions, in this case, are intended for legitimate media management rather than data obfuscation. Similar analyses can be applied to other properties--such as Activities, Services, Metadata, Action, Receiver, and Provider--to further differentiate between vault and non-vault applications. To address the \"curse of dimensionality\", we use a chi-square test to determine the properties that provided the highest contribution to the detection of vault apps. We also apply PCA to further improve the detection capability. Initial experiments (Figure 4) with a random forest classifier, using a subset of the data and without fine-tuning (i.e., without the chi-square selection and PCA dimensionality reduction), were far from perfect but encouraging. Approximately 1/3 of vault apps and 1/3 of non-vault apps were misclassified. Figure 5 shows the confusion matrix for the random forest classifier, using chi-square and PCA. Note that no non-vault apps are misclassified as vault apps, and very few vault apps are misclassified as non-vault apps. Principal component analysis was employed to reduce the dimensionality of the feature space, as shown in Figure 6. The data points are projected across one or two principal components to test the variance between classes (vault and non-vault apps). The PCA visualization depicts how the data is distributed in two-dimensional space. On the plot, the separation into clusters of points shows that feature selection and dimensionality reduction effectively captures the difference between vault and non-vault applications. The clusters clearly distinguish between the classes, which suggests that our model has high discriminative power. Figure 4: Initial confusion matrix for RF classifier. The loading plot identifies the properties that have the largest effect on each principal component. The coefficient range is \(\pm 1\). Large coefficients indicate that a property strongly influences a component. Coefficients close to 0 indicate that a property has little influence on the component. The coefficients for the apR.1 property are 0.171 and 0.406 for the PC1 and PC2 components, respectively, indicating that this property influences PC2 relatively strongly (and PC1 much less so). Conversely, the coefficients for the apW property are 0.336 and 0.156 for PC1 and PC2, respectively, indicating that this property influences PC1 relatively strongly compared to PC2. Further, the angle between the properties indicates their influence on one another; 90 degrees (orthogonal) would mean that the properties are 0.171 and 0.406 for PC1 and PC2 components, respectively. Figure 5: Confusion matrix for RF classifier. Figure 6: PCA plot for RF classifier. independent.