# SSCL-TransMD: Semi-Supervised Continual Learning Transformer for Malicious Software Detection
## 3 Methodology
### Model Components
#### 3.1.2 Similarity Calculation Component  
The purpose of the similarity calculation component is to calculate the similarity between new input unlabeled malware samples and historical labeled malware samples iteratively, based on the historical labeled malware samples. This calculation is used to assist the training of the information encoding layer by providing pseudo-labels. Most pseudo-labeling methods train models on labeled data and then use the trained models to predict labels for unlabeled data in order to create pseudo-labels. However, due to the complexity of training the Transformer-based malware training model, most pseudo-labeling methods are not suitable for the scenario in this paper. The LLGC(Learning with Local and Global Consistency) algorithm is a classic label propagation algorithm, which provides smooth classification based on the intrinsic similarity between labeled and unlabeled data. It allows for simple iteration to provide pseudo-labels for unlabeled data. The basic idea of the LLGC algorithm is to iteratively propagate the label information of each point to its neighboring points until a globally stable state is reached. This allows unlabeled malware samples to obtain corresponding pseudo-labels based on similarity scores, based on the assumption of prior consistency: (1) nearby points may have the same label; (2) points with similar structures may have the same label, thus constructing a smoothing function [26]. Since the LLGC algorithm requires very few computing resources and the accuracy of pseudo-label calculation is high after repeated iterations, this subsection uses the LLGC algorithm as the similarity calculation component. Below is a detailed explanation of the calculation process of the similarity calculation component. Figure 3: Examples of unbalanced data. First, given a set of malware vectors \(X=\{x_{1},\ldots,x_{l},x_{l+1},\ldots,x_{n}\}\subset R^{m}\) and a corresponding set of labels \(L=\{1,\ldots,c\}\), the labels of the first \(l\) malware vectors \(x_{i}\)\((i\leq l)\) are marked as \(y_{i}\in L\), and the remaining malware vectors \(x_{u}\)\((l+1\leq u\leq n)\) are unlabeled. Next, when \(i\
eq j\) and \(W_{i,j}=0\), the similarity matrix \(W\) between different malware feature vectors is calculated using the formula shown in Equation (8):  
\[W_{i,j}=exp\left(\frac{-\|x_{i}-x_{j}\|^{2}}{2\cdot\sigma^{2}}\right) \tag{8}\]  
At this point, the similarity matrix \(W\) is used to calculate the diagonal matrix \(D\), where the diagonal elements are the sums of the \(i\)-th row of \(W\), i.e., \(D_{ii}=\sum_{j}W_{ij}\). A new matrix \(S=D^{-\frac{1}{2}}WD^{-\frac{1}{2}}\) is constructed using the diagonal matrix \(D\) and the similarity matrix \(W\). Then, for each \(x_{i}\), assume a non-negative score vector \(F_{i}\in R_{+}^{1\times C}\) with positive direction, where \(F_{i}=(f_{1},\ldots,f_{iC})\) represents the similarity scores for different labels. The vector \(F=(F_{1}\ldots F_{l+u})^{T}\) is propagated through labels to obtain \(Y=(Y_{1}\ldots Y_{l+u})^{T}\). Finally, iterate \(F\) until convergence, using the following iteration formula:  
\[F(t+1)\leftarrow\alpha SF(t)+(1-\alpha)Y \tag{9}\]  
where \(\alpha\in(0,1)\) and \(F(0)=Y\). Let \(F^{*}\) represent the limit of the sequence \(F(t)\), and assign each point \(x_{i}\) with the label \(y_{i}=argmax_{j\leq c}F_{ij}\). First, the similarity matrix \(W\) is defined on the dataset \(X\), with the diagonal elements set to zero. Assume a graph \(G=(V,E)\) defined within \(X\), where the vertex set \(V\) is equal to \(X\), and the edge set \(E\) is weighted by the values in \(W\). The algorithm then performs symmetric normalization on the matrix \(W\) of the graph \(G\). This step is mandatory to ensure the convergence of the iteration. During each iteration, each instance receives information from its neighboring instances while retaining its initial information.