# Detection of Exceptional Malware Variants Using Deep Boosted Feature Spaces and Machine Learning
## 3 Methodology
### Classification Schemes
#### 3.2.3 The Proposed DBFS-MC Framework  
In the aforementioned DFS-MC framework, we have carefully selected the best two customized CNNs, ResNet-18 and DenseNet-201. We have utilized the hybrid learning capacity of these two customized CNN in the proposed deep boosted learning and named the Deep Boosted Feature Space-based Malware Classification (DBFS-MC) framework. In the proposed DBFS-MC approach, ResNet-18 and DensNet-201 are customized and TL based fine-tuned according to our problem space. Deep features arisen in the second last layer classification layer of these customized CNNs are merged to produce a boosted feature space and provided to SVM. An overview of this framework is shown in Figure 4. The proposed deep boosted learning scheme exploited the deep feature learning capability of customized CNN and the discrimination power of SVM. In Equation (3), \(\mathrm{x}_{Boosted}\) shows the boosted feature space that are generated by concatenation of deep feature spaces of \(f_{ResNet}\)(x) and \(f_{DenseNet}\)(x). These boosted features are then fed to SVM for classification. SVM creates an optimal decision boundary by minimizing the misclassification rate and maximizing the margin between the samples [31]. \(\zeta\) repre \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multicolumn{1}{c}{**Pre-Trained**} & \multicolumn{2}{c}{**Trained from Scratch**} & \multicolumn{2}{c}{**Deep Boosted**} \\ \hline Feature Layer & Feature Dim. & Feature Layer & Feature Dim. & Feature Layer & Feature Dim. \\ Last fc & 64 \(\times\) 25 & Last fc & 64 \(\times\) 25 & Last fc & 128 \(\times\) 25 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Deep feature matrix dimensions for each CNN model. sents the misclassified samples, while \(C\) is the cost that establishes the trade-off between misclassification rate and the model's generalization as depicted in Equations (4) and (5):  
\[\mathrm{x}_{Boosted}=f_{b}(f_{ResNet}(\mathrm{x})\mid f_{DenseNet}(\mathrm{x})) \tag{3}\]  
\[w^{\mathrm{T}}\mathrm{x}_{Boosted}+\mathrm{b}=0 \tag{4}\]  
\[\mathop{\mathit{min}}\limits_{w,\delta_{i}}C\sum_{i}^{N}\zeta_{i}+\frac{1}{2} \
orm{\mathrm{w}}^{2} \tag{5}\]  
In the proposed framework, we exploited the potential of residual and block-based learning to generate prominent features from malware visual images as shown in Figures 5 and 6. The residual and block learning attributes of both the architectures are highly appreciated in the form of experimental results. These two architectures are described below. Figure 4: Flow diagram of the proposed DBFS-MC malware classification approach. Figure 5: Skip Connection Building Block. ResNet-18  
Residual network (ResNet-18) for the first time solved vanishing gradient problem by introducing skip connections, also known as identity connection [32], as shown in Figure 5. Skip connections help the gradient to flow through an alternate path and allow the model to learn identity hypothesis function. They provide a mean for information flow from earlier to later layers of the model. ResNet-18 uses a bottleneck residual block design to increase the performance of the network. ResNet-18 is 18 layers deep:  
\[\mathrm{x}_{s,t}=\sum_{a=1}^{\mathrm{m}}\sum_{b=1}^{n}\mathrm{x}_{s+a-1,t+b-1} \mathrm{w}_{a,b} \tag{6}\]  
\[\mathrm{y}=f(\mathrm{x}\,,\,\{\mathrm{w}_{i}\})+\mathrm{x} \tag{7}\]  
\[\mathrm{y}=f(\mathrm{x}\,,\,\{\mathrm{w}_{i}\})+\,\mathrm{w}_{\mathrm{s}} \mathrm{x} \tag{8}\]  
Convolutional operation for the symmetric filter is shown in Equation (6). Here, \(\mathrm{x}\) represents the input and convolved feature map and \(\mathrm{w}\) represents the kernel. Equations (7) and (8) describe the residual operation, where \(\mathrm{w}_{i}\) and \(\mathrm{w}_{\mathrm{s}}\) represents the weight layers of \(3\times 3\) and \(1\times 1\) (skip connection) convolutional operation, respectively. DenseNet-201  
The DenseNet-201 architecture connects each layer in a feed-forward manner.