# Malware Detection Approach Based on Artifacts in Memory Image and Dynamic Analysis
## 4 Results and Discussion  
### Machine Learning Modeling  
Several machine learning algorithms were applied on the integrated vector using the WEKA tool; the support vector machine (SVM), the decision tree, Naive Bayes (NB), K-nearest neighbor (KNN) and the random forest. A 10-fold cross validation was applied to evaluate the classification models. Thus, the data set samples were shuffled and split into 10 groups and the data used in training the classifier was completely separated from the testing one. By using a 10-fold cross validation and by removing all the duplicates in the data set, the performance of a given model was estimated in a better way. Moreover, the following measures are used to evaluate the results:  
* True Positive (TP): number of correctly classified malicious files. Figure 4: Four steps performed by the python script for feature extraction and pre-modeling. Figure 3: Generating global and local vectors for one file. * True Negative (TN): number of correctly classified benign files. * False Positive (FP): number of benign files wrongly classified as malicious files. * False Negative (FN): number of malicious files wrongly classified as benign files. * Detection Rate (DR): \(=\frac{TP}{TP+FN}\). * False Positive Rate (FPR): \(=\frac{FP}{FP+TN}\). * Accuracy: \(=\frac{TP+TN}{TP+TN+FP+FN}\). In order to examine the accuracy of the proposed approach, three experiments were performed. In the first experiment, the API call features extracted from the memory analysis only were classified. The global vector and local vectors for this experiment were also constructed from the API features extracted from the memory analysis. The experiment result is shown in Table 1. This table shows that the K-nearest neighbor (KNN) machine learning algorithm performed better with accuracy close to 95% and a false positive rate equal to 5.1%. In the next experiment, the API call features extracted during the dynamic analysis were only classified. Likewise, the global vector and local vectors were created from the API features obtained in dynamic analysis. The result is shown in Table 2. The performance of SVM is the highest among others with an accuracy rate of 97.4% and false positive rate of slightly less than 5%. In the third experiment, the proposed integrated approach was examined. The combination of the API call features from the memory and dynamic analyses was used to create the global vector and local vectors, as discussed earlier in the feature extraction and pre-modeling subsections. The results (shown in Table 3) display a significant improvement in the accuracy rate as well as an impressive decrease in the false positive rate. The integrated approach applying the support vector machine (SVM) algorithm outperformed both the memory analysis and dynamic analysis with an accuracy rate equal to 98.5% and false positive rate as low as 1.7%. Figure 5 shows the evaluation rates for the malware and benign files when applying the SVM algorithm. \begin{table}
\begin{tabular}{c c c c c} \hline
**Classifier** & **DR (\%)** & **FPR (\%)** & **Accuracy (\%)** & **Time (ms)** \\ \hline
**Naïve Bayes** & 94.4 & 5 & 94.6 & 3.7 \\
**SVM** & 93.8 & 6.4 & 93.9 & 11.4 \\
**Decision Tree** & 88.2 & 13.3 & 88.2 & 3.4 \\
**Random Forest** & 93.8 & 6.7 & 93.8 & 27.2 \\
**KNN** & 94.7 & 5.1 & 94.8 & 1.0 \\ \hline \end{tabular}
\end{table} 
Table 1: The result of classifying memory features. \begin{table}
\begin{tabular}{c c c c c} \hline
**Classifier** & **DR (\%)** & **FPR (\%)** & **Accuracy (\%)** & **Time (ms)** \\ \hline
**Naïve Bayes** & 80.8 & 8.2 & 88.6 & 9.2 \\
**SVM** & 97.4 & 4.9 & 97.4 & 22.8 \\
**Decision Tree** & 95.7 & 8.4 & 95.7 & 31.7 \\
**Random Forest** & 96.5 & 5.2 & 96.6 & 47.3 \\
**KNN** & 96.2 & 5.3 & 96.3 & 2.0 \\ \hline \end{tabular}
\end{table} 
Table 2: The result of classifying dynamic features. It is well known that smart malware change their behaviors or terminate themselves when they are placed in a controlled environment. In addition, dynamic analysis can only demonstrate one path of file execution.