# A Risk Assessment Framework for Mobile Apps in Mobile Cloud Computing Environments
## 3 Results
### Ensemble ML Model  
A series of experiments were conducted to identify the most suitable algorithms for inclusion in an ensemble classifier to build an effective ML-based app classification model. The selected ML classification algorithms (shown in Table 3) have been used extensively in related prior research [31]. Each algorithm was applied to each of the three study datasets. The outputs were compared to identify the best-performing classification algorithm based on the following confusion matrix:  
1. True Positive (TP): The number of malicious apps classified correctly as malicious (in a given dataset). 2. True Negative (TN): The number of benign apps classified correctly as benign (in a given dataset). 3. False Negative (FN): The number of malicious apps classified wrongly as benign (in a given dataset). 4. False Positive (FP): The number of benign apps classified wrongly as malicious apps (in a given dataset. The performance metrics used are listed below. The performance metric values obtained are shown in Table 4.  
1. Classification Accuracy (CA): The percentage of all correctly classified apps (TP + TN) out of all apps in a given dataset (TP + TN + FP + FN). 2. Error Rate (ER): The percentage of all misclassified apps (FP + FN) out of all apps in a given dataset (TP + TN + FP + FN). 3. Precision (PR): The percentage of the correctly classified malicious apps (TP) out of all apps classified as malicious in a given dataset (TP + FP). 4. Recall (RC): The percentage of the correctly classified malicious apps (TP) out of all malicious apps in a given dataset (TP + FN). 5. False Positive Rate (FPR): The percentage of the wrongly classified malicious apps (FN) out of all benign apps in a given dataset (TN + FP). 6. False Negative Rate (FNR): The percentage of the wrongly classified benign apps out of all malicious apps in a given dataset (TP + FN). 7. False Alarm Rate (FAR): The mean of FPR and FNR. 8. F-Measure (FM): The harmonic mean of PR and RC. Consistent with the data about intent usage by malicious and benign apps in the study sample, ML models using intents as the only features performed poorly compared to models using permissions only or those using both intents and permissions. Moreover, combining both permissions and intents as features consistently produced better results compared to using permissions only in terms of accuracy (CA) and false alarm rate (FAR). These results lend support to considering both app permissions and intents in the proposed risk assessment framework. \begin{table}
\begin{tabular}{c c} \hline \hline
**ID** & **Classification Algorithm** \\ \hline C1 & Decision Tree \\ C2 & Random Forest \\ C3 & AdaBoost \\ C4 & Naive Bayes \\ C5 & Stochastic Dual Coordinate Accent \\ C6 & Multi-Layer Perceptron \\ C7 & K-Nearest Neighbor \\ C8 & Linear Discriminant Analysis \\ C9 & Logistic Regression \\ C10 & Support Vector machine \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Classification algorithms used in this study. The three best-performing classifiers in terms of CA, FAR, and false negative rate (FNR) were C2 (Random Forest), C1 (Decision Tree), and C7 (K-Nearest Neighbor). Applied to the hybrid dataset t, the top classifier (C2) achieved a CA of 94.59%, precision rate (PR) of 96.05%, and recall rate (RC) of 95.58%. The outcomes of the tenfold cross-validation (Table 5) indicated that C1, C2, and C7 performed better than most, apart from C9 when using the Permissions dataset having the highest maximum CA overall (96.11%). However, C2 had the highest minimum CA (91.17%) overall. When applied to the hybrid dataset, C1 and C9 achieved the same maximum CA overall (95.41%); the highest minimum CA overall also belonged to C2 (90.46%). Considering these results, C1, C2, and C7 retained their position as the top-performing algorithms amongst the ten evaluated. The top best-performing algorithms discussed above (Decision Tree, Random Forest, and K-Nearest Neighbor) were used to construct the ensemble ML classifier.