# SSCL-TransMD: Semi-Supervised Continual Learning Transformer for Malicious Software Detection
## 3 Methodology  
The SSCL-TransMD detection model is mainly composed of five layers, namely, the memory replay layer, information mapping layer, similarity calculation layer, information encoding layer, and classification detection layer. Each layer is responsible for receiving the input of weighted historical sample sequences and new sample sequences, mapping the input sequence, providing pseudo-labels for the input sequence, feature encoding, as well as training the model and outputting the final classification results.  
In the SSCL-TransMD detection model, the input data consist of data sequences generated during the operation of different categories of malware, and the output data are the identification result of malware. The overall structure of the SSCL-TransMD detection model is shown in Figure 1.  
Figure 1: SSCL–TransMD.  
The following sections provide a detailed introduction to the model structure:  
Memory Replay Layer: The inputs of this layer are the malware data after static analysis. The required sub-sample B for each learning iteration is divided in advance. At the first layer, they are sequence data \(A_{1}\). At the i-th layer, the data from \(A_{1}\) to \(A_{i}\) layers are combined through the memory replay sampling method and given to the information mapping layer;  
Similarity Calculation Layer: The input of this layer is the feature matrix generated by the information mapping layer. The feature matrix is composed of countless vectors. The LLGC algorithm is iteratively used in the similarity calculation layer to calculate the similarity matrix between labeled vectors and unlabeled vectors, thus obtaining similarity scores. The similarity scores are used as input for the unlabeled vectors, and the feature matrix obtained from the information mapping layer in the information coding layer;  
Information Mapping Layer: The inputs of this layer are the sequence data \(A_{i}\) obtained from the memory replay layer. After preprocessing the sequence data \(A_{i}\), the malware data size and annotation format are adapted to the model. During each training iteration, a sub-sample \(A_{i}\) is obtained from the dataset, and the input data size is A \(\times\) L \(\times\) W, where L represents the number of malware features in the input and W represents the dimension of the data after mapping by the input layer;  
Information Coding Layer: The input of this layer is the feature matrix generated by the similarity calculation layer and its corresponding similarity labels. The information coding layer consists of three layers: multi-head attention mechanism layer, residual connection and normalization layer, and fully connected layer. The feature matrix is learned using multi-head attention, then passed through the residual connection and normalization layer to perform residual connection on the sequence matrix weighted by the feature, followed by a linear transformation to map the data, which are finally mapped by the fully connected layer to learn more abstract features;  
Classification Detection Layer: The input of this layer is the matrix obtained from the information coding layer. The classification detection layer consists of an MLP basic neural network, which adjusts the matrix dimension provided by the information coding layer through the MLP fully connected layer, then performs classification and outputs the classification results through the softmax activation function. The specific model and parameters are detailed in the next chapter. After training, the model parameters converge to achieve optimal prediction performance. The test dataset is input into the trained model to output the recognition results of the test dataset.