# SSCL-TransMD: Semi-Supervised Continual Learning Transformer for Malicious Software Detection
## 3 Methodology
### Model Components
#### 3.1.1 Memory Replay Component
More specificly, LUMP interpolates between the examples of the current task \((x_{i,\tau})\in U_{\tau}\) and random examples selected using uniform sampling from the replay buffer, which encourages the model to behave linearly across a sequence of tasks. For current task \(\tau\), LUMP minimizes the objective on the following interpolated instances \(x_{i,\tau}\):  
\[\bar{x}_{i,\tau}=\lambda\cdot x_{i,\tau}+(1-\lambda)\cdot x_{i,M} \tag{5}\]  
where \(x_{i,M}\sim M\) denotes the example selected using uniform sampling from replay buffer M. The interpolated example not only augments the past tasks' instances in the replay buffer, but also approximates regularized loss minimization [24]. During unsupervised continuous learning, LUMP enhances the robustness of learned representation by revisiting the attributes of the past task that are similar to the current. As a result, LUMP successively mitigates catastrophic forgetting. Although the LUMP model partially avoids the catastrophic forgetting problem, there still exists an issue regarding the proportion between historical sample sequences and new input malicious software sample sequences. In the LUMP model, the weight \(\lambda\) for the historical sample sequence and the new input malicious software sample sequence is fixed, which introduces a new problem. As the number of continuous learning training increases, the number of historical sample sequences also increases. At this time, the imbalance between the new task data and the historical sample data causes a data imbalance problem [25]. Specifically, in the training algorithm based on memory replay, only a few old class samples are seen, while there are more new class samples. In this case, the focus of the training process clearly shifts towards the new class, resulting in many detrimental effects on class-specific weights, as shown in Figure 2(a), where the weights of new sequence data are significantly higher than those of old sequence data. From Figure 2(b), it can be seen that the relationship between the labels of old data and their corresponding weights is not well preserved. The combination of these effects severely misleads the classifier, leading to decision biases towards confusion between new and old classes. These effects seriously cause the model to forget malicious behaviors in the old sequences during the analysis of malicious behavior. To address these problems, this subsection proposes improvements to the training idea of the LUMP model in the memory replay component. First, assume a memory replay buffer M that provides historical malicious software sample sequences. It mixes the historical input sequences stored in the memory replay buffer with new input malicious software detection samples. In other words, for the current input sequence \(x_{i,\tau}\in T\) and the sequence sampled from the memory replay buffer, a sampled sequence \(\hat{x}_{i,\tau}\) is created, as shown in Equation (5). Here, \(x_{i,M}\) represents an example sampled from the memory replay buffer M, and the obtained \(\hat{x}_{i,\tau}\) is used as a training sample. Next, to address the issue of data imbalance, an adaptive weight \(\lambda_{new}\) is proposed for improvement, as shown in Equation (6):  
\[\lambda_{new}=\lambda_{base}\sqrt{C_{n-1}/x_{n}} \tag{6}\]  
Here, \(C_{n-1}\) is the number of old samples in the \((n-1)_{th}\) training, \(x_{n}\) is the number of new input samples in the nth training, and \(\lambda_{base}\) is the base weight for each dataset. Therefore, as the ratio between the number of historical sequences and the number of new input sequences increases, \(\lambda_{new}\) increases, thereby mitigating the detrimental effects caused by data imbalance, which allows for more effective preservation of previously learned knowledge and reduction of ambiguity between new and old classes.