# SSCL-TransMD: Semi-Supervised Continual Learning Transformer for Malicious Software Detection
## 4 Experiment
### Experimental Analysis
#### 4.2.1 Comparison Models  
The comparison models selected in this section are mainly related to semi-supervised models and continual learning models, aiming to objectively verify the effectiveness of the proposed SSCL-TransMD model compared to existing continual learning models and semi-supervised models:  
(1) SSL-MD [28]: SSL-MD is based on the LLGC algorithm and aims to construct a machine learning classifier using labeled malicious software, benign software, and unlabeled instances;  
(2) DroidDL [16]: The DroidDL framework is a malware detection method based on online learning methods, capturing security-sensitive behaviors from applications using the graph neural network algorithm;  
(3) DroidEvolver [29]: DroidEvolver employs a model pool which consists of five different kinds of linear online learning algorithms, including Passive-Aggressive (PA), Online Gradient-Descent (OGD), Adaptive Regularization of Weight Vectors (AROW), Regularized Dual Averaging (RDA), and Adaptive Forward-Backward Splitting (Ada-FOBOS), to process necessary light model updates through computing pseudo-labels;  
(4) PLDNN [30]: PLDNN is an efficient Android malware classification system based on a semi-supervised deep neural network.  
The experiment results are shown below. Tables 9 and 10 shows the binary and multi-classification results of the methods above respectively.  
Among them, when conducting binary classification detection, the F1-score has an average improvement of 1.27% compared to other models. Furthermore, after inputting hybrid samples four times, which include historical labeled samples and new unlabeled samples, the F1-score has an average improvement of 1.96%.  
When performing multi-class classification using the CICMalDroid 2020 dataset for labeled training, compared to other models, the Micro-F1 metric saw an average improvement of 2.7%. Furthermore, after inputting a mixture of samples containing historical labeled samples and new unlabeled samples four times, the Micro-F1 metric saw an average improvement of 1.7%.  
Through comprehensive analysis, it can be concluded that the proposed SSCL-TransMD detection model in this paper achieves good results in both binary and multiclass classifica  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{**Labeled**} & **First Input of** & **Second Input of** & **Third Input of** & **Fourth Input of** \\  & & **Mixed Samples** & **Mixed Samples** & **Mixed Samples** & **Mixed Samples** \\ \hline SSL–MD & 0.7261 & 0.7223 & 0.7198 & 0.7166 & 0.7153 \\ DroidOL & 0.7407 & 0.7379 & 0.7340 & 0.7301 & 0.7264 \\ DroidEvolver & 0.7396 & 0.7365 & 0.7321 & 0.7287 & 0.7255 \\ PLDNN & 0.7255 & 0.7214 & 0.7185 & 0.7112 & 0.7076 \\ SSCL-TransMD & 0.7528 & 0.7489 & 0.7415 & 0.7355 & 0.7309 \\ \hline \hline \end{tabular}
\end{table}
Table 10: Multi-classification of CICMalDroid2020.  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline \multirow{2}{*}{**Model**} & \multirow{2}{*}{**Labeled**} & **First Input of** & **Second Input of** & **Third Input of** & **Fourth Input of** \\  & & **Mixed Samples** & **Mixed Samples** & **Mixed Samples** \\ \hline SSL–MD & 0.8375 & 0.8316 & 0.8279 & 0.8225 & 0.8193 \\ DroidOL & 0.8498 & 0.8463 & 0.8416 & 0.8347 & 0.8278 \\ DroidEvolver & 0.8483 & 0.8413 & 0.8379 & 0.8314 & 0.8298 \\ PLDNN & 0.8412 & 0.8379 & 0.8341 & 0.8278 & 0.8209 \\ SSCL–TransMD & 0.8549 & 0.8537 & 0.8492 & 0.8482 & 0.8406 \\ \hline \hline \end{tabular}
\end{table}
Table 9: Binary classification of CICMalDroid2020.  
tion. Compared to other semi-supervised classification methods SSL-MD and DroidEvolver based on pseudo-labeling, the pseudo-labeling determination method used in our proposed SSCL-TransMD is more effective in improving the model. Compared to other deep learning models DroidOL and PLDNN, the Transformer-based model performs better in the scenario of semi-supervised continual learning.