# Malware Variants Detection Model Based on MFF-HDBA
## 5 Experimental Results and Discussion
### Comparison of Our Model Performance with Advanced Deep Learning Frameworks  
Since AlexNet [48] won the ImageNet Large Scale Visual Recognition Challenge (ILSVRC) in 2012, a surge of development in deep learning occurred. From the efforts of these experts and scholars came numerous excellent CNN structures that were proposed, one after another, such as VGGNet [49], ResNet [50], and GoogleNet [51]. These CNN models have been utilized to resolve the classification issue of multi-class malware, with good results. In order to validate the effectiveness of the proposed model in this paper, this section first evaluated the training performance of this model and then compared our model with AlexNet8, VGG16, ResNet50, and Inception V3. In this section, we implemented the experiments on the Malimg data set after data augmentation, which contained 51608 samples in the training set. Figure 7 shows the training curve of our model with 50 rounds. The training time of our model was 10,081.05 s and the memory consumption was 3365.03 MB(Mbytes). It can be observed that the model converged when the epoch reached 20, indicating a faster convergence rate. The classification accuracy reached 100% in the training set and 99.36% in the test set. The model performed well on both the training and test sets without overfitting. In addition, the training time for the model was closely related to the performance of the GPU. Using a powerful GPU can greatly reduce the time required to train the model on hundreds of thousands to millions of malware variants in an acceptable time. Table 6 shows the results of the comparison experiments between our model and three other classical neural network models. From Table 6, it can be observed that the accuracy, precision, recall, and F1-score of our model improved significantly compared with the other models. The performance of AlexNet8 was limited to its shallow architecture. The performance of ResNet50 and VGG16 were better than AlexNet8 due to their greater depth of network. VGG16 utilized small-size filters to increase the depth of the network, while ResNet50 used residual block to solve the problem of gradient explosion and gradient disappearance for more convolution layers. Inception V3 increased the width of the neural network architecture. The experimental results indicated that the width of the neural network can improve model performance. Our model achieved the best performance among all of the above models, due to the design of multi-scale feature extraction blocks. It was able to capture more abundant texture information of feature maps from different scales of receptive field (RF), and enhanced the expression of typical features. In order to further observe and analyze the classification performance of the models in each malware family, the distribution of the five models in each malware family in the Malimg data set was plotted in Figure 8. The results indicate that our model significantly improved the insufficient classification accuracy of AlexNet, VGG16, ResNet50, and Inception V3 in some confusable malware families to different degrees, such as C2 LOP.P, C2 LOP.gen!g, Swizzor.gen!E, and Swizzor.gen!, which ultimately improved the overall classification accuracy of our model. The above experimental results demonstrate that a combination of multi-scale feature fusion and channel attention mechanism has excellent feature extraction capability, which can effectively extract representative features from malware and distinguish easily confused malware variants. The mean time to detect (MTTD) reflects the average time taken by the detector to successfully identify a threat. The experiments were implemented by calculating the parameters and MTTDs of the models, and the results are shown in Table 7. The experimental results show that our model obtained the smallest volume, with 1.11 million(M) parameters, which was 1/362 that of AlexNet, 1/134 that of VGG16, 1/23 that of ResNet50, and 1/21 that of Inception V3.