# \(HM^{3}a1D\): Polymorphic Malware Detection Using Program Behavior-Aware Hidden Markov Model
## 5 Experimental Evaluation
### Comparing with Other Work  
In this section, the performance of \(HM^{3}alD\) in terms of DR, FAR, and Accuracy is compared to important previous works. For fair comparison, we consider the number of dataset members same as the compared methods when we train HMMs of \(HM^{3}alD\). To express the detection performance of \(HM^{3}alD\), we compare \(HM^{3}alD\) to dynamic malware detection methods and show the corresponding results in Table 4. Moreover, Table 5 shows the results of \(HM^{3}alD\) in comparison with static malware detection methods. As shown in Tables 4 and 5, \(HM^{3}alD\) outperforms all dynamic and static malware detection methods especially in term of FAR that is a hard work to decrease it without affecting DR.  
Figure 8: The performance comparison of \(HM^{3}alD\) with and without the program behavior-aware topology for \(k=24\) and \(fracrj=0.05\). Figure 9: The performance comparison of \(HM^{3}alD\) (for \(k=24\)), \(HM^{3}alD\_S\) (for \(k=26\)) and \(HM^{3}alD\_S\_R\) (for \(k=26\)). Expressing the behavior of a malware program leads to detect its polymorphic instances, effectively. \(HM^{3}alD\) tries to extract realistic behavior of malware from polymorphic instances. First, \(HM^{3}alD\) abstracts malware behavior by using high-level action sequences instead of system call sequences. Then, it clusters similar malware action sequences leading to integrate polymorphic instances of a malware. Finally, \(HM^{3}alD\) models the realistic behavior of each malware cluster using HMM. Thus, \(HM^{3}alD\) provides a high detection rate and low false rate in comparison with other work. Note that a false positive occurs when \(HM^{3}alD\) erroneously labels a benign program as malware. If a malware detector blocks access or deletes a program or file that is vital to the proper functioning of some system programs, those may become unusable and, in some cases, the deletion may render a system unstable, although the anti-malware methods try to reduce FAR to zero. For Virus Bulletin [42], \"the 'no false positives' rule is one of the main requirements for certification in the VB100 test process\". AV-Comparatives [43] considers false positives \"an important measurement for AV quality\", and an important factor in determining the reliability of a product, besides its detection capabilities. Thus, we introduce new metric, \(PenalizedDR\), as Equation (7). \[PenalizedDR=DR-\alpha*FAR, \tag{7}\]  
We analyze the impact of \(\alpha\) by increasing it from 0 to 10 in increments of 0.5. Figures 10 and 11 show the results of \(PenalizedDR\) for \(HM^{3}alD\) and current dynamic and static methods, respectively. The important findings from the results shown in Figures 10 and 11 are as follows: (1) FAR is the most important and impressive measurement for malware detection methods; (2) high DR is not enough to have a good result and its effectiveness depends on FAR; and (3) high FAR significantly reduces the usefulness and applicability of such methods. According to Tables 4 and 5, \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Approaches** & **Dataset Size (Benign/Malware)** & **DR(\%)** & **FAR(\%)** & **Accuracy (\%)** \\ \hline [PERSON] (2013) [20] & 105/114 & 93.7 & 0 & 96.65 \\ \(HM^{3}alD\) & 105/114 & **100** & **0** & **100** \\ \hline [PERSON] (2014) [21] & 98/416 & 97.57 & 0 & 98.05 \\ \(HM^{3}alD\) & 98/416 & **100** & **0** & **100** \\ \hline [PERSON] (2017) [23] & 1359/3009 & 98.4 & 4.6 & — \\ \(HM^{3}alD\) & 1359/3009 & **98.89** & **1.12** & **98.88** \\ \hline [PERSON] (2015) [22] & 2000/2000 & 97.6 & 2.37 & 96.89 \\ \(HM^{3}alD\) & 2000/2000 & **98.83** & **1.18** & **98.87** \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of \(HM^{3}alD\) with current dynamic malware detection methods.