# SSCL-TransMD: Semi-Supervised Continual Learning Transformer for Malicious Software Detection
## 3 Methodology
### Model Components
#### 3.1.1 Memory Replay Component  
The memory replay component is one of the core components of semi-supervised continual learning. The purpose of this component is to mix historical labeled malicious software sample sequences and unlabeled new malicious software sample sequences according to the principles of continual learning [22], using certain weights. This subsection discusses how the principles of continual learning are applied in the field of malicious software and how the memory replay method for malicious software samples is utilized in this chapter. (1) Continuous Learning  
When the malware detection model transitions into batch learning mode, it is easy to forget the old malware classifications. This means that, after updating the model with new malware data, the classification performance achieved by the malware detection model in historical tasks will rapidly decline, leading to catastrophic forgetting. The root cause of catastrophic forgetting is that the training process for the new malware classification task requires changing the weights of the historical neural network. This inevitably modifies certain weights that are crucial for the historical malware detection task, rendering the malware detection model no longer suitable for historical tasks. To overcome catastrophic forgetting, the malware detection model needs to not only demonstrate the ability to acquire new knowledge in new classification tasks, but also prevent significant interference from new malware data on the existing model [23]. In this section, we use a continuous learning approach to continuously train the malware detection model. The process of continuous learning for a malware model, denoted as \(P\), is shown in Equations (1) and (2):  
\[P=\{P_{1},P_{2},\ldots,P_{n},\ldots\} \tag{1}\]  
\[P_{n}:\langle M_{n-1},T_{n}\rangle\to M_{n} \tag{2}\]  
The initial malware detection model before continuous learning is represented as \(M_{0}\), and the continuous input of newly added malware behavior data sequence is represented as \(T=T_{1},T_{2},\ldots,T_{n},\ldots,T_{n}=(X_{n}^{i},Y_{n}^{i})|i=1,2,\ldots,N_{n}\), where \(X_{n}^{i}\) and \(Y_{n}^{i}\) are the dataset instances and corresponding labels of the first \(i\) malware sequences at task \(T_{n}\). The malware detection model after learning from \(T_{n}\) is represented as \(M_{n}\). The process of semi-supervised continual learning is shown in Figure 2.  
(2) Memory Replay Method  
The LUMP model is able to sample between the current batch and the historical sequences of malicious software inputs, effectively mitigating the catastrophic forgetting problem. It is a lifelong unsupervised learning method. The training idea of the LUMP model is to mixup two random samples \((x_{i},y_{i})\) and \((x_{j},y_{j})\) together with the weight \(\lambda\) to form a new sample, as shown in Equation (3):  
\[ \begin{cases}\bar{x}=\lambda\cdot x_{i}+(1-\lambda)\cdot x_{j}\\ \bar{y}=\lambda\cdot y_{i}+(1-\lambda)\cdot y_{j}\end{cases}  \tag{3}\]  
The standard mixup [24] training contrutually virtual training examples based on the principle of Vicinal Risk Minimization. Let \((x_{i},y_{i})\) and \((x_{j},y_{j})\) denote two random feature  
Figure 2: Semi-supervised continual learning process. target pairs sampled from the training data distribution, and let \((\bar{x},\bar{y})\) denote the interpolated feature-target pair in the vicinity of these examples; mixup then minimizes the following objective:  
\[ \begin{split} L_{mixup}(\bar{x},\bar{y})&=CE(h_{\psi} (f_{\odot}(\bar{x})),\bar{y}),\\ &\text{where }\bar{x}=\lambda x_{i}+(1-\lambda)x_{j}\text{ and }\bar{y}= \lambda y_{i}+(1-\lambda)y_{j}\end{split}  \tag{4}\]  
where \(f_{\odot}\) is an encoder network which is composed of a backbone network and \(h_{\psi}\) is prediction MLP head, \(\lambda\sim Beta(\alpha,\alpha)\), for \(\alpha\in(0,\infty)\). LUMP utilizes mixup for UCL (unsupervised continuous learning) by incorporating the instances stored in the replay buffer from the previous task into the vicinal distribution.