# SFCWGAN-BiTCN with Sequential Features for Malware Detection
## 4 Experiments and Analysis of Results
### Experimental Results and Analysis
#### 4.3.4 Ablation Experiments  
Model ablation experiments were established to validate the impact of the proposed model on the detection ability of a few malware family samples. The BiTCN, GAN [23]-BiTCN, CWGAN [38]-BiTCN, and the models in this paper were compared on the Kaggle and DataCon datasets under the same experimental conditions. The traditional algorithms performed poorly for Vundo, Simda, Tracur and Kelihos_ver1, which are families. Among them, for the Simda family with only 42 samples, the classification accuracy was poor, mainly because the sample size was too small to obtain accurate family features. The accuracy for each model was evaluated for difference malware families of the Kaggle and DataCon datasets and are shown in Tables 7 and 8. As can be seen from Tables 7 and 8, the accuracy of the unbalanced malware family samples based on feature selection and this paper's model is significantly improved. The main reason is that the actual samples include many invalid features, and using feature selection to remove redundant features simplifies the structure and improves the accuracy. The GAN, CWGAN, and SFCWGAN were used with sample expansion for the families \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Algorithm** & **Method** & **Accuracy/\%** & **Precision/\%** & **Recall/\%** & **F1-Score/\%** \\ \hline \multirow{2}{*}{BOS-BiTCN} & \multirow{2}{*}{Borderline random synthesis and balance} & \multirow{2}{*}{96.63} & \multirow{2}{*}{96.63} & \multirow{2}{*}{96.63} & \multirow{2}{*}{96.63} \\  & the samples & & & & \\ \multirow{2}{*}{ROS-BiTCN} & \multirow{2}{*}{Resample the sample} & \multirow{2}{*}{96.53} & \multirow{2}{*}{96.50} & \multirow{2}{*}{96.51} & \multirow{2}{*}{96.55} \\ ADASYN-BiTCN & & & & & \\ \multirow{2}{*}{SMOTE-BiTCN} & \multirow{2}{*}{KNN for create samples based on density} & \multirow{2}{*}{96.72} & \multirow{2}{*}{96.72} & \multirow{2}{*}{96.72} & \multirow{2}{*}{96.72} \\  & the samples & & & & \\ \multirow{2}{*}{CWGAN-BiTCN} & \multirow{2}{*}{CWGAN samples} & \multirow{2}{*}{96.84} & \multirow{2}{*}{96.84} & \multirow{2}{*}{96.84} & \multirow{2}{*}{96.84} \\ Model in this paper & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Detection results under the influence of enhanced algorithms on DataCon dataset. \begin{table}
\begin{tabular}{c c c c c c} \hline \hline
**Algorithm** & **Method** & **Accuracy/\%** & **Precision/\%** & **Recall/\%** & **F1-Score/\%** \\ \hline \multirow{2}{*}{BOS-BiTCN} & \multirow{2}{*}{Borderline random synthesis and balance} & \multirow{2}{*}{99.13} & \multirow{2}{*}{99.13} & \multirow{2}{*}{99.13} & \multirow{2}{*}{99.13} \\  & the samples & & & & \\ \multirow{2}{*}{ROS-BiTCN} & \multirow{2}{*}{Resample the sample} & \multirow{2}{*}{99.03} & \multirow{2}{*}{99.03} & \multirow{2}{*}{99.03} & \multirow{2}{*}{99.03} \\ ADASYN-BiTCN & KNN for create samples based on density & & & & \\ \multirow{2}{*}{SMOTE-BiTCN} & \multirow{2}{*}{KNN for random synthesis and balance} & \multirow{2}{*}{99.22} & \multirow{2}{*}{99.22} & \multirow{2}{*}{99.22} \\ SMOTE-BiTCN & & & & & \\ \multirow{2}{*}{CWGAN-BiTCN} & \multirow{2}{*}{KNN for random synthesis and balance} & \multirow{2}{*}{99.35} & \multirow{2}{*}{99.35} & \multirow{2}{*}{99.35} \\ CWGAN-BiTCN & & & & & \\ \multirow{2}{*}{CWGAN-BiTCN} & \multirow{2}{*}{CWGAN samples} & \multirow{2}{*}{99.46} & \multirow{2}{*}{99.46} & \multirow{2}{*}{99.46} & \multirow{2}{*}{99.46} \\ Model in this paper & & & & & \\ \multirow{2}{*}{Model in this paper} & \multirow{2}{*}{Based on feature selection and solves the} & \multirow{2}{*}{99.55} & \multirow{2}{*}{99.57} & \multirow{2}{*}{99.54} & \multirow{2}{*}{99.53} \\  & & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Detection results under the influence of enhanced algorithms on Kaggle dataset. Vundo, Simda, Tracur, and Kelihos_ver1. By combining with the original training set, the unbalanced malware family samples are increased to expand the malware family samples and solve the gradient disappearance problem.