# Dynamic IoT Malware Detection in Android Systems Using Profile Hidden Markov Models
## 5 Evaluation and Results
Equation (7) shows the formula of F-measure: \[\text{F-measure}=2*\frac{\text{precision}*\text{Recall}}{\text{precision} +\text{recall}}\] (7)  
We calculated the detection rate for each malware family according to Equation (1). Table 4 shows the detection rate for each malware family. As can be seen, 18 families have a high detection rate which is above 90% excluding Boxer, ExploitLinuxLotoor, Hambo and SMSreg families whose detection rates are between 84.21% and 88.89%. The Geinimi and Jifak family have detection rates of 57.14% and 66.67%, respectively. The YZHC family has the lowest detection rate of 31.58% due to the low number of tested samples. Having said this, some families with a low number of test samples still yielded a worthy detection rate,such as Boxer and Hambo. Notably, DroidKungFu, Fakelnstaller, Opfake and Plankton have the highest detection rates, approaching 99.28%, because they contain a high number of test samples. It can be concluded that determining the detection rate of a family depends on the number of samples used in training and testing, and on the complexity of the family. The results of accuracy, FP rate, FN rate, precision, recall and F1 score are presented in Table 5. DIP showed 96.3% accuracy with 5% FP and 3% FN which shows how successful our model is at predicting the correct category of the code. It has pinpointed malware code with a precision of up to 94.0%. While the 96.0% recall indicates that we have only missed 0.04 of malware code. The F-measure of 94.9% indicates that the model provides a balanced precision and recall score. We compared the results of DIP with rival approaches that used the Drebin dataset for malware detection in Table 6. The missing figures in the comparison are denoted by \(\epsilon\). Based on Table 6, DIP technique scored the highest on accuracy, precision, and recall, with 96.3% for accuracy, 94% for precision, and 96% for recall. On the other hand, the benchmark algorithms' highest accuracy was 96% achieved by the dynamic classifier in [45]. However, that classifier is capable of binary classification only. Among the benchmarks, the static classifier in [16] got the highest precision and recall at 93% and 95%, respectively. As shown by these results, DIP performed better than all state-of-the-art algorithms when it was applied to detect malware on Android IoT platforms. DIP's dynamic nature adds to its advantages since it can detect more malicious features and recognize new malware families easily. \begin{table}
\begin{tabular}{l l c c c c} \hline \hline \multirow{2}{*}{\#} & \multirow{2}{*}{**Malware Family**} & **Detection** & \multirow{2}{*}{**\#**} & \multirow{2}{*}{**Malware Family**} & **Detection** \\  & & **Rate \%** & & & **Rate \%** \\ \hline
1 & Adrd & 96.67 & 13 & Glodream & 94.74 \\
2 & BaseBridge & 94.64 & 14 & Hambo & 87.50 \\
3 & Boxer & 88.89 & 15 & Iconosys & 96.43 \\
4 & DroidDream & 95.24 & 16 & Imlog & 92.86 \\
5 & DroidKungFu & 98.05 & 17 & Jifake & 66.67 \\
6 & ExploitLinkLotoor & 86.96 & 18 & Kwin & 93.94 \\
7 & FakeDoc & 95.83 & 19 & MobileTx & 94.44 \\
8 & Fakelnstaller & 99.28 & 20 & Opfake & 98.34 \\
9 & FakeRun & 93.75 & 21 & Plankton & 98.96 \\
10 & Gappusin & 94.44 & 22 & SendPav & 90.00 \\
11 & Geinimi & 57.14 & 23 & SMSreg & 84.21 \\
12 & Ginmaster & 95.54 & 24 & YZHC & 31.58 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Detection rate of each malware family using DIP. \begin{table}
\begin{tabular}{l l} \hline \hline
**Measure** & **Value** \\ \hline FP Rate & 5\% \\ FN Rate & 3\% \\ Precision & 94.0\% \\ Recall & 96.0\% \\ Accuracy & 96.3\% \\ F-measure & 94.9\% \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Results of malware detection using DIP.