## 4 Experimental Results and Discussion
### Experimental Results Analysis  
In the above, we obtained six sets of categorical features through feature extraction and selection. To verify the effectiveness of single-category features, this paper uses the tree-boosting-based XGBoost classification algorithm to train and test them, as shown in Table 4. Since the Microsoft Challenge does not provide labels for the test dataset, we used five-fold cross-validation on the training set for testing. As shown in Table 5, the classification accuracy of each proposed six-category feature is over 99.7% and 98.7% on both the test set and the training set. It demonstrates that the selected features can effectively distinguish the families to which the malware belongs. In particular, the Asm_1_gram feature alone achieves the greatest classification accuracy of 99.45% and a low-level logarithmic loss of 0.0207. To significantly increase the suggested system's classification accuracy and decrease memory overhead, we adopted the forward feature stepwise selection algorithm for effective feature fusion to make a trade-off between the number of features and the classification efficiency, as shown in Table 5.  
As shown in Table 6, according to the Asm_1_gram category features with the lowest logloss, we used a forward feature stepwise selection algorithm to add other category features that make the model produce the lowest logloss to form a fusion feature set. The experimental findings demonstrate that the classification accuracy of all fusion feature sets (C1-C5) based on the XGBoost classification algorithm is 100% on the training data. The logloss values of C1-C5 under the test data continue to decrease, especially C5 achieved a classification accuracy of 99.79% and a log loss of 0.009. It demonstrates that the feature set fused by the forward selection algorithm can gradually improve classification accuracy. In particular, the C5 fusion feature set can help to build a reliable malware classification scheme. The previous experiments show that the fusion feature set (C5) can achieve 99.79% classification accuracy on the original XGBoost classifier. The XGBoost algorithm based on tree structure has the advantage of adjustable parameters. We used a grid search algorithm to adjust the parameters of the XGBoost to obtain the optimal parameter combination, which is most conducive to distinguishing malware families. The optimization \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Category of Feature** & **Numbers** & **Train** & \multicolumn{2}{c}{**5-CV**} \\ \hline  & & **Accuracy** & **Logloss** & **Accuracy** & **Logloss** \\ \hline C1: Asm\_1\_gram + Asm\_Pixel & 2348 & 1.0 & 0.00042 & 0.9959 & 0.0163 \\ C2: C1 + Byte\_1\_gram & 2605 & 1.0 & 0.00036 & 0.9966 & 0.0129 \\ C3: C2 + Opcodes\_2\_gram & 3170 & 1.0 & 0.00035 & 0.9976 & 0.0097 \\ C4: C3 + Opcodes\_3\_gram & 5170 & 1.0 & 0.00035 & 0.9978 & 0.0092 \\
**CS: C4 + Byte\_2\_gram** & **5530** & **1.0** & **0.00035** & **0.9979** & **0.0090** \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Adding features categories gradually based on feature fusion. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Category of Feature** & **Numbers** & **Train** & \multicolumn{2}{c}{**5-CV**} \\ \hline  & & **Accuracy** & **Logloss** & **Accuracy** & **Logloss** \\ \hline
**Hex dump file** & & & & & \\ \hline Byte\_1\_gram & 257 & 1.0 & 0.0007 & 0.9873 & 0.0477 \\ Byte\_2\_gram & 360 & 1.0 & 0.0005 & 0.9940 & 0.0234 \\ \hline
**Disassembled file** & & & & & \\ \hline Asm\_Pixel & 2300 & 0.9999 & 0.0010 & 0.9902 & 0.0399 \\ Asm\_1\_gram & 48 & 0.9982 & 0.0054 & 0.9945 & 0.0207 \\ Opcodes\_2\_gram & 565 & 0.9971 & 0.0082 & 0.9914 & 0.0335 \\ Opcodes\_3\_gram & 2000 & 0.9971 & 0.0083 & 0.9921 & 0.0307 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: List of feature categories and their evaluation with XGBoost.