# A Self-Supervised Learning Model for Unknown Internet Traffic Identification Based on Surge Period
## 3 Methods
### The Self-Supervised Learning-Based Identification Model for Unknown Internet Traffic
#### 3.2.1 The JigClu-Based Flow Feature Pre-Training Model  
As far as we know, JigClu is an efficient self-supervised learning approach. It aims to enable the model to restore a batch of scrambled data to the original state through self-supervised learning training. The main procedures are outlined as follows:  
* Batch division and reorganization: This is the prerequisite step before further procedures. It allows the division of traffic data into batches with the same length of \(b\). For the data in each batch, the algorithm will segment and reorganize the fragments from the same batch to form a \"new\" batch of data. In our proposed model, we divide the input data with a feature size of \((n,k)\) into \(b\times 4\) data blocks with the same size. The fragments in the same batch are recombined to form input with a size of \((b,n,k)\). This input is new data with the same size as the original data while the content is disordered. At the same time, we record the scrambled fragments as labels to calculate the training loss functions.
* Pre-training model: This step mainly focuses on establishing and optimizing the pre-training model based on the \"new\" batch of data. The pre-training model adopts a deep residual network (ResNet) to extract features, which is composed of multiple residual units. In each residual unit, there are multiple convolutional layers and shortcut connections. As shown in Figure 4, we assume that the input of the residual unit is \(x\), and its expected output is \(H(x)\), which is a complex potential map. If we want to learn such a complicated model, the training would be more difficult. With the help of shortcut  
Figure 3: The self-supervised learning-based identification model for unknown Internet traffic.  
connections, ResNet's learning goal could be calculated as \(F(x):=H(x)-x\), and it helps solve the gradient disappearance problem of deep neural networks. The ResNet will finally generate two kinds of output, used to determine the original data block to which the included data belongs and the location to which the data belongs. The output is transformed into \((b\times 2\times 2)\) vectors, represented as \(L=L_{1},L_{2},\ldots L_{4b}\), which will be further used for data positioning and reorganization operations.
* JigSaw: For the data positioning module, its goal is to find the original location of multiple data blocks in the data. For example, if the input data are spliced into four data blocks, the first block of the spliced data is from the first small block of some original data, then the tag corresponding to this block could be marked as 0. Similarly, the second one is marked with the tag 1, and so on. Here, we regard the process of data positioning as a problem of multi-classification and adopt the function of CrossEntropy to calculate the loss of JigSaw. The details can be seen in Equation (3). \[loss_{jigsaw}=-\sum_{i=1}^{M}y_{i}\log\left(p_{i}\right)\] (3) where \(p_{i}\) refers to the predicated distribution and \(y_{i}\) is the actual distribution. \(M\) represents the number of types.
* Clu: The purpose of the data reorganization module is to find as many data blocks from the same original data. For the input of each data batch,  
By adopting the self-supervised learning algorithm of JigClu, the original features acquired at the pre-processing stage would be pre-trained and learned. It further generates the pre-trained feature matrix that can be used for further identification.  
To be noted, the JigSaw and Clu steps in the JigClu method would not appear in the final identification model. They are only adopted here by calculating loss functions and, in turn, help optimize the model parameters. Before the final procedure of identification, we need to remove the modules of JigSaw and Clu, keep the remaining part in JigClu as the feature pre-training model, and further combine it with the clustering algorithm to realize the identification.