# Deep Learning Based Android Anomaly Detection Using a Combination of Vulnerabilities Dataset
## 4 Experiments
### Results  
In this section, we present obtained results. The factors below explain why our approach was able to outperform other approaches. These include hyper-parameters tuning as well as the combination of vulnerability features in our dataset. * Performance of DL model: Initially, we used 11,814 apps; in this experiment, an app which has 0 flag labelled as benign, whereas an app which has two or more flags labelled as malware, and an app with one flag are excluded. The training phase performance results were higher than the testing phase, which caused over-fitting in the model. To solve this situation,we increased the size of the dataset by adding the malware apps from VirusShare and apps with one flag as benign. Experimentally, we observe that the performance improved. From Table 4, we can observe that the size of the dataset has increased from 11,814 to 18,780 to avoid over-fitting and improve the performance. Figures 7 and 8 illustrate the history model's accuracy and loss for 11,814 and 18,526 samples, respectively. From the figures, it is clear that when we increase the size of the dataset, the accuracy and loss curve lines in the training phase are very close to the accuracy and loss curve lines in the testing phase. In the previous experiment, the difference between the accuracy and loss curve lines was huge. The performance improved when we increased the size of the dataset and the over-fitting problem was solved. If the output layer's Sigmoid result was greater than or equal to 0.5, the application was categorised as malware. Values below 0.5 were considered benign. As Table 5 indicates, the performance improved when we increased the size of the dataset and the over-fitting problem was solved. If the output layer's Sigmoid result was greater than or equal to 0.5, the application was categorised as malware. Values below 0.5 were considered benign. Table 5 thus shows that the confusion matrix correctly \begin{table}
\begin{tabular}{l c c c} \hline \hline
**Size of Dataset** & **Accuracy** & **F1** & **AUC\_score** \\ \hline
11,814 samples & 89\% & 90\% & 88\% \\ \hline
18,780 samples & 99.33\% & 99\% & 99.15\% \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison between the results of datasets with 11,814 samples, and 18,780 samples. Figure 8: Comparison between history models loss for 11,814 samples and 18,526 samples. (**a**) Loss for 11,814 samples. (**b**) Loss for 18,526. Figure 7: Comparison between history models Accuracy for 11,814 samples and 18,526 samples. (**a**) Accuracy for 11,814 samples. (**b**) Accuracy for 18,526.  
classifies the 2406 benign samples as benign and 1275 malware samples as malware. Out of 3706 app samples, 3681 samples were predicted accurately and only 25 samples were wrongly predicted. Performance of the SVM model:  
Table 3 illustrates the experimental results. When we were tuning the hyperplane parameters, we noticed that when gamma is smaller than 0.01 and C is higher than 1000, the results improve, i.e., both parameters increase the values of AUC, F1, and the accuracy. With such parameter values, we can therefore get the correctly separating hyperplane and improve the performance of the model. Table 6 shows that the confusion matrix correctly classifies the 2425 benign samples as benign and 1235 malware samples as malware. Out of 3706 app samples, 3660 were predicted accurately and only 46 were wrongly predicted. Challenges and discussion  
Improving the performance of the SVM classifier was challenging and involved some fine tuning with respect to the two parameters: C and gamma. Our results showed that tuning C correctly is a vital step in the use of SVMs for structural risk minimization In RBF kernel, both C and gamma parameters need to be optimized simultaneously. If gamma is large, the effect of C becomes negligible.. When gamma gets smaller, the results improve.