# OpenCBD: A Network-Encrypted Unknown Traffic Identification Scheme Based on Open-Set Recognition

Authors: Xinyi Hu; Chunxiang Gu; Yihang Chen; Xi Chen; Fushan Wei
Journal: Wireless Communications and Mobile Computing
 (2022)

---

## Abstract

The encryption of network traffic promotes the development of encrypted traffic classification and identification research. However, many existing studies are only effective for closed-set experimental data, that is to say, only for traffic of known classes, while there are often lots of unknown classes traffic in the real environment of open sets, and many studies have difficulty identifying the traffic of unknown classes and can only misclassify them as known classes. How to identify unknown traffic and classify known traffic in an open-collection environment is one of the focuses of traffic analysis research. Considering these problems, this paper proposes a novel solution, which applies the open-set recognition method to the unknown traffic identification, and constructs a model based on deep learning and ensemble learning. The method constructs a model based on a convolutional neural network and a transformer encoder and then uses a three-stage training and testing process, combined with a novel loss function, to generalize to the open space to form OpenCBD. Experiments on public datasets show that the proposed method is significantly better than other open-set identification methods. It can not only distinguish known traffic from unknown traffic but also identify specific classes of known traffic.


---

## Body

## 1. Introduction

With the wide application of encryption technology in network traffic, it becomes more and more challenging to effectively monitor and analyze network traffic, and encrypted traffic analysis technology has also become an important research topic in the field of network security [1–3]. To analyze encrypted traffic, it is first necessary to divide the traffic into different sets according to specific goals, that is, to classify and identify network traffic. Most of the existing researches are implemented in a closed environment; that is, many researches can efficiently and accurately classify and identify traffic of known classes. For example, Aceto et al. [4, 5] proposed a practical mobile traffic classification model based on deep learning which can automatically extract features. Liu et al. [6] proposed an encrypted traffic classification model FS-Net based on the recurrent neural network. Wang et al. [7] proposed App-Net, a mobile application recognition model based on RNN and CNN. Nascita et al. [8] improved a multimodal deep learning traffic classification model based on explainable artificial intelligence techniques. But some studies, while valid in closed settings, often cannot really be applied to the real world. Because the real-world environment is open, network traffic includes encrypted and nonencrypted, known and unknown, benign and malicious, standard protocols and private protocols, etc. There are many practical problems to be considered and many difficulties to overcome. If a classification method trained in a closed set is simply generalized to an open set, it is easy to misclassify samples of unknown classes into known classes [9–11]. In order to solve this problem, researchers need to develop models that can support both the classification of known class samples and the discovery of unknown class samples, so as to actively manage and prevent abnormal traffic and further create and maintain a good network environment.In real-world identification and classification tasks, it is often difficult to obtain labels for all samples during training, but it is desirable to have the ability to identify unknown classes during testing. Therefore, open-set recognition describes such a scenario. During testing, samples of unknown classes that have not appeared in training will appear. The classifier can not only accurately classify known samples but also identify unknown classes. In a real network environment, there are known classes of encrypted traffic and unknown classes of encrypted traffic. This is similar to the scenario of open-set recognition. Therefore, some open-set recognition methods can be used to solve the unknown traffic identification. This paper proposes an unknown traffic identification method OpenCBD based on open-set recognition. It first performs a pretraining process based on self-supervised learning on unlabeled data, so that the CBD model (CBD model was proposed in [12]; it is a self-supervised learning model containing three modules; the three modules are the CNN module—based on a convolutional neural network, BERT module—based on a transformer model encoder, dense module—based on a fully connected network) has a certain understanding of the basic characteristics of encrypted traffic. Then, the training and testing process based on ensemble learning is designed in the open set. During training, the individual model is trained based on a specific loss function on some known classes. Then, all known classes are trained through the ensemble strategy, so that the ensemble model can achieve accurate classification of known classes and identification of unknown classes in the process of open-set testing. The contributions of this paper are summarized as follows:
(i)
A novel unknown traffic identification model OpenCBD is designed. It uses the idea of open-set recognition, combines deep learning and ensemble learning, learns the basic characteristics of encrypted traffic from unlabeled data, and then trains on known classes of traffic to classify and identify traffic in an open environment(ii)
A general training method suitable for open-set recognition is proposed. The method only needs to be trained on the data of the known classes and can identify the data of the unknown classes. The method adopts two-stage training: first, it randomly selects part of the known classes data to train the individual model and then integrates the individual model to train with all known classes data, so that the model can learn the special knowledge between classes(iii)
A loss function that combines the cross-entropy loss function commonly used in classification models and the II-loss function proposed for open-set recognition is proposed. The combination of the two loss functions can train the model more efficiently, making the model fit faster and more accurately, so that the classification of known classes and the identification of unknown classes can be better completed at the same time(iv)
The OpenCBD model achieves good results in unknown traffic identification and known traffic classification tasks. Furthermore, the OpenCBD model outperforms significantly compared to the baseline methodsThe rest of the paper is organized as follows. Section2 summarizes the basic knowledge and related work of open-set recognition and unknown traffic identification. Section 3 details the structure and methodology of the overall model. Section 4 introduces the specific details of the experiments and evaluates and compares the experimental results. Finally, Section 5 concludes the paper.


## 2. Preliminary and Related Work

This section mainly introduces the basic knowledge and researches in recent years of open-set identification and unknown traffic identification.


### 2.1. Open-Set Recognition

First, some definitions in open-set recognition are given.Definition 1 (open space [13]).
Given a label setK, the known class label is a positive integer, and the unknown class label is 0. For the feature x∈ℝd, define f∈H to be a measurable recognition function, fyx>0 means that the class y can be recognized, and fyx≤0 means that y is not recognized, where H:ℝd⟶ℝ is the appropriate smooth space for the recognition function. For any known class of training samples xi∈K,i=1,⋯,N, the open space O is defined as
(1)O=SO−⋃i∈NBrxi,
whereBrxi is a closed sphere with the training sample xi as the center and radius r. SO is a sphere of radius ro, including all known positive training samples x∈K and open space O.Definition 2 (open space risk [13]).
Open space risk is defined as the relative measure of positive labeled open space compared to the overall measure of positive labeled space. Then, the probabilistic open space riskROf of class y is
(2)ROf=∫Ofyxdx∫SOfyxdx.Definition 3 (openness [14]).
Openness refers to the degree of openness of an open space, which consists of training classes, target classes, and test classes,(3)O=1−2×CTRCTA+CTE,
whereCTR,CTA,CTE represent training classes, target classes, and testing classes, respectively. Sometimes O<0 may occur, so the openness after calibration is only related to training classes and test classes,
(4)O∗=1−2×CTRCTR+CTE.Definition 4 (open-set risk [13]).
The open-set recognition problem is to minimize both traditional empirical risk and open space risk. Given an empirical risk functionRε, the open-set risk is defined as
(5)argminf∈HROf+λrRεf,
whereλr is the regularization constant.Definition 5 (open world recognition [13]).
The solution for open world recognition is represented by the quintupleF,φ,ν,L,I. Fx:Rd⟶N is a multiclass open-set recognition function. The i-th class in Fx is identified by the vector function φx of the measurable recognition function fix, and the detector νφ:Rito0,1 is determined, where i∈Kt, νφ determine whether the output vector of the recognition function is from an unknown class. Lx:Rd⟶N+ is the labeling process. Lx applies new unknown data Ut to time t, resulting in label data Dt=yj,xj, where yj=Lxj∀xj∈Ut. Assuming that the label finds m new classes, the set of known classes becomes Kt+1=Kt∪i+1,⋯,i+m. Itφ;Dt:Hi⟶Hi+m is the incremental learning function. Itφ;Dt extensibly learns and adds new measurable functions fi+1x⋯fi+mx to the measurable recognition function vector φ. Each measurable function minimizes the corresponding open space risk.Suppose that eachfkx outputs the probability of belonging to the k classes. And assume that fkx is normalized across each class. Let φ=f1x,⋯,fkx, then the multiclass open-set recognition function is
(6)Fx=0,ifνφx=0,y∗,otherwise,where(7)y∗=argmaxy∈K,fyx∈φxfyx.At this point, an easy way to detect is to set an acceptable minimum thresholdτ and minimize the open space risk, i.e.,
(8)νφx=fy∗x>τ.Since open-set recognition was proposed, many research methods have been proposed, mainly including discriminative models and generative models. Among them, the discriminative models are mostly constructed by traditional machine learning methods and deep neural network methods, and the generative models are divided into methods based on instance generation and methods based on noninstance generation according to whether there is instance generation. There are also some studies using neural networks in these two methods. The following mainly introduces some methods based on deep neural networks, which include not only discriminative models but also generative models.In 2016, Bendale and Boult [15] proposed OpenMax, the first open-set recognition method based on deep neural networks. OpenMax was used as a new model layer that estimated the probability that the input came from an unknown class and provided a bounded open space risk.In 2017, Ge et al. [16] proposed a multiclass open-set recognition method Generative OpenMax (G-OpenMax). Unlike some existing studies, the unknown class was not inferred from the features of the known classes or the decision distance. It extended OpenMax by employing Generative Adversarial Networks (GANs) for new classes of image synthesis.In 2018, Yoshihashi et al. [17] proposed an open-set recognition classification reconstruction learning method CROSR, which used latent representations for reconstruction and achieved robust unknown detection without reducing the classification accuracy of known classes.In 2019, Oza and Patel [18] proposed an open-set recognition algorithm C2AE, which used a class-conditional autoencoder, as well as closed-set classification training and open-set recognition training. The encoder and decoder were trained in two stages, and the reconstruction error was modeled using the extreme value theory of statistical modeling to find a threshold that identified samples of known and unknown classes.In 2020, Hassen and Chan [19] proposed a representation method based on a neural network and used this representation method to propose an open-set recognition mechanism. In this representation, instances from the same class were close to each other, while instances from different classes were further apart.In 2020, Liu et al. [20] proposed an algorithm that uses the meta-learning technique PEELER to solve the problem of open-set recognition. It combines randomly selecting a new set of classes per episode, maximizes the loss of postentropy of these class instances, and then learns a new metric formula based on the Mahalanobis distance.In 2021, Joseph et al. [21] proposed an open-world object detector ORE, which was based on contrastive clustering and energy-based unknown identification. Identifying and characterizing unknown instances helped reduce confusion in incremental object detection settings. In this setting, state-of-the-art performance could be achieved without additional methodologies.In 2022, Geng and Chen [22] proposed a batch decision strategy that was aimed at extending existing open-set recognition methods for new class discovery while considering correlations between test instances. By modifying the Hierarchical Dirichlet Process (HDP), a collective decision-based open-set recognition framework CD-OSR was proposed. CD-OSR did not need to define decision thresholds and could realize open-set recognition and new class discovery at the same time.Table1 summarizes several open-set recognition methods mentioned in Section 2.1.Table 1
Different models for open-set recognition.
ModelReferenceOpen-set recognition categoryMethodologyExtreme value theoryAdvantagesOpenMax[15]Discriminative modelEVT-based calibration classification√First deep open-set classifier without using background samplesG-OpenMax[16]Generative modelUnknown generation classification√Combining generative adversarial networks and OpenMaxCROSR[17]Discriminative modelDistance√First neural network architecture which involved hierarchical reconstruction blocksC2AE[18]Discriminative modelReconstruction error√Algorithms using class conditional autoencodersNeural-network-based representation[19]Discriminative modelEVT-free calibration classification×A loss function was proposed such that instances from the same class are close to each other, while instances from different classes are farther apartPEELER[20]Discriminative modelDistance×Combining few-shot classification and open-set recognitionORE[21]Discriminative modelDistance×An incremental object detector is proposedCD-OSR[22]Discriminative modelEVT-free calibration classification×Automatically reserve space for unknown classes under test, naturally bringing new class discovery capabilities


### 2.2. Unknown Traffic Identification

In the field of network traffic analysis, unknown traffic identification has always been an important research direction. Researchers usually use unsupervised learning or semisupervised learning to solve the tasks of identifying and detecting unknown traffic.In 2011, Finamore et al. [23] proposed an unsupervised algorithm to identify traffic classes within aggregates. This algorithm utilized the K-means clustering algorithm and added a mechanism to automatically determine the number of traffic clusters.In 2013, Zhang et al. [24] proposed an approach to address the problem of unknown applications in the critical case of small supervised training sets. The proposed method had a superior ability to detect unknown traffic generated by unknown applications and exploited the correlation information between real-world network traffic to improve the classification performance.In 2013, Zhang et al. [25] proposed an iterative method to extract unknown information from a set of unlabeled traffic flows. The method combined asymmetric bagging and flow correlation to guarantee the purity of the extracted negatives and demonstrated significantly better than state-of-the-art flow classification methods under unknown applications.In 2014, Yu et al. [26] proposed a method to classify elephant traffic using service-based statistical features for cluster analysis. Elephant traffic refers to unknown traffic generated by only a few or some types of applications.In 2015, Shaikh and Harkut [27] proposed a framework that classifies unknown flows in the network, solving the problem of applying unknowns in critical situations with little supervised training data. Flow label propagation was proposed, which automatically and accurately labeled more unlabeled flows to enhance the ability of Nearest Clustering-based Classifiers (NCCs). Composite classification was also proposed, which combines many flow predictions to more accurately classified Bag of Flows (BoF).In 2015, Lin et al. [28] proposed a semisupervised learning method to address the problem of an unknown protocol in critical cases where the labeled training sample set was small. With the help of flow-related information and semisupervised clustering ensemble learning, the method had a superior ability to detect unknown samples generated by unknown protocols to improve classification performance.In 2017, Ma and Qin [29] proposed a method using deep learning techniques to identify unknown protocols in complex network environments. The method identified the protocol in the network flow according to the application layer protocol type and found out the unknown protocol. This method only used the payload information in the captured 200,000 traffic flows and achieved well unknown protocol traffic identification accuracy.In 2018, Fu et al. [30] proposed a scheme, FlowCop, to implement traffic detection that did not belong to any predefined application in network traffic classification. It divided the test traffic into N classes and one unknown class by building multiple one-class classifiers. A feature subspace algorithm was also proposed to select salient features for each class of classifiers.In 2019, Sabeel et al. [31] proposed two methods to predict unknown DoS and DDoS attacks based on DNN and LSTM. The method demonstrated how well deep learning-based methods perform in unknown situations and to what extent deviations from the trained model could be handled. This method can effectively identify unknown attacks.In 2019, Zhang et al. [32] proposed a network intrusion detection method based on open-set recognition. This method fits the recognition results of known classes to Weibull distribution and then builds an Open-CNN model to estimate the probability of unknown classes from the activation scores of known classes, so as to achieve the purpose of detecting unknown attacks.In 2020, Zhang et al. [33] proposed an autonomous learning framework to correctly classify unknown classes. The framework efficiently updated deep learning-based traffic classification models during active operation. The core of the proposed framework consisted of a deep learning-based classifier, a self-learning discriminator, and an autonomous self-labeling model. The discriminator and self-labeling process generated new datasets during active operation to support classifier updates.In 2020, Mohamed et al. [34] proposed a method for handling unknown applications. This method enabled efficient network classification with limited supervised training sets. The proposed model applied multiple neural network algorithms to predict unknown applications. The method improved Internet performance, reduced Internet traffic, and reduced delays in transmitting data.In 2021, Wang et al. [35] proposed an unknown protocol parsing method based on a convolutional neural network. The protocol data was preprocessed into an image, and the converted image was inputted to the convolution layer for convolution. After convolution, the data was flattened, and the flattened data was put into a fully connected neural network to analyze and predict unknown protocols.In 2021, Li et al. [36] proposed a lightweight unknown traffic discovery model, LightSEEN, which realized traffic classification and model update in the open world under practical conditions. The overall structure of the method was based on the Siamese network, and each side used a multihead attention mechanism, a one-dimensional convolutional neural network, and a residual network to facilitate the extraction of deep flow features and the convergence speed of the network.In 2021, Xu et al. [37] proposed the KCC (Known Central Clustering) method to deal with the open-set-based intrusion detection problem. By introducing CD-loss (Class Distance-loss), the centers of different clusters were obtained. By introducing negative samples as unknown classes for training, the threshold of known classes was obtained. Unknown intrusions were rejected by comparing with fuzzy distances.Table2 makes a simple classification and summary according to the specific methods used in the above-mentioned several unknown traffic identification literatures. We combine unknown traffic identification with open-set recognition. According to the characteristics of network encrypted traffic, we design a new open-set recognition method based on a discriminant model. Through self-supervised learning and supervised learning, it can effectively complete known traffic classification and unknown traffic identification tasks at the same time.Table 2
Summary of methods for identifying unknown traffic.
Statistical methodsDeep neural networksSupervised learningSemisupervised learningUnsupervised learningCNN-basedOpen-set recognitionOthers[25][28][23, 24, 26, 27, 30][29, 32, 33, 35, 37][32, 35, 37][31, 34]


## 2.1. Open-Set Recognition

First, some definitions in open-set recognition are given.Definition 1 (open space [13]).
Given a label setK, the known class label is a positive integer, and the unknown class label is 0. For the feature x∈ℝd, define f∈H to be a measurable recognition function, fyx>0 means that the class y can be recognized, and fyx≤0 means that y is not recognized, where H:ℝd⟶ℝ is the appropriate smooth space for the recognition function. For any known class of training samples xi∈K,i=1,⋯,N, the open space O is defined as
(1)O=SO−⋃i∈NBrxi,
whereBrxi is a closed sphere with the training sample xi as the center and radius r. SO is a sphere of radius ro, including all known positive training samples x∈K and open space O.Definition 2 (open space risk [13]).
Open space risk is defined as the relative measure of positive labeled open space compared to the overall measure of positive labeled space. Then, the probabilistic open space riskROf of class y is
(2)ROf=∫Ofyxdx∫SOfyxdx.Definition 3 (openness [14]).
Openness refers to the degree of openness of an open space, which consists of training classes, target classes, and test classes,(3)O=1−2×CTRCTA+CTE,
whereCTR,CTA,CTE represent training classes, target classes, and testing classes, respectively. Sometimes O<0 may occur, so the openness after calibration is only related to training classes and test classes,
(4)O∗=1−2×CTRCTR+CTE.Definition 4 (open-set risk [13]).
The open-set recognition problem is to minimize both traditional empirical risk and open space risk. Given an empirical risk functionRε, the open-set risk is defined as
(5)argminf∈HROf+λrRεf,
whereλr is the regularization constant.Definition 5 (open world recognition [13]).
The solution for open world recognition is represented by the quintupleF,φ,ν,L,I. Fx:Rd⟶N is a multiclass open-set recognition function. The i-th class in Fx is identified by the vector function φx of the measurable recognition function fix, and the detector νφ:Rito0,1 is determined, where i∈Kt, νφ determine whether the output vector of the recognition function is from an unknown class. Lx:Rd⟶N+ is the labeling process. Lx applies new unknown data Ut to time t, resulting in label data Dt=yj,xj, where yj=Lxj∀xj∈Ut. Assuming that the label finds m new classes, the set of known classes becomes Kt+1=Kt∪i+1,⋯,i+m. Itφ;Dt:Hi⟶Hi+m is the incremental learning function. Itφ;Dt extensibly learns and adds new measurable functions fi+1x⋯fi+mx to the measurable recognition function vector φ. Each measurable function minimizes the corresponding open space risk.Suppose that eachfkx outputs the probability of belonging to the k classes. And assume that fkx is normalized across each class. Let φ=f1x,⋯,fkx, then the multiclass open-set recognition function is
(6)Fx=0,ifνφx=0,y∗,otherwise,where(7)y∗=argmaxy∈K,fyx∈φxfyx.At this point, an easy way to detect is to set an acceptable minimum thresholdτ and minimize the open space risk, i.e.,
(8)νφx=fy∗x>τ.Since open-set recognition was proposed, many research methods have been proposed, mainly including discriminative models and generative models. Among them, the discriminative models are mostly constructed by traditional machine learning methods and deep neural network methods, and the generative models are divided into methods based on instance generation and methods based on noninstance generation according to whether there is instance generation. There are also some studies using neural networks in these two methods. The following mainly introduces some methods based on deep neural networks, which include not only discriminative models but also generative models.In 2016, Bendale and Boult [15] proposed OpenMax, the first open-set recognition method based on deep neural networks. OpenMax was used as a new model layer that estimated the probability that the input came from an unknown class and provided a bounded open space risk.In 2017, Ge et al. [16] proposed a multiclass open-set recognition method Generative OpenMax (G-OpenMax). Unlike some existing studies, the unknown class was not inferred from the features of the known classes or the decision distance. It extended OpenMax by employing Generative Adversarial Networks (GANs) for new classes of image synthesis.In 2018, Yoshihashi et al. [17] proposed an open-set recognition classification reconstruction learning method CROSR, which used latent representations for reconstruction and achieved robust unknown detection without reducing the classification accuracy of known classes.In 2019, Oza and Patel [18] proposed an open-set recognition algorithm C2AE, which used a class-conditional autoencoder, as well as closed-set classification training and open-set recognition training. The encoder and decoder were trained in two stages, and the reconstruction error was modeled using the extreme value theory of statistical modeling to find a threshold that identified samples of known and unknown classes.In 2020, Hassen and Chan [19] proposed a representation method based on a neural network and used this representation method to propose an open-set recognition mechanism. In this representation, instances from the same class were close to each other, while instances from different classes were further apart.In 2020, Liu et al. [20] proposed an algorithm that uses the meta-learning technique PEELER to solve the problem of open-set recognition. It combines randomly selecting a new set of classes per episode, maximizes the loss of postentropy of these class instances, and then learns a new metric formula based on the Mahalanobis distance.In 2021, Joseph et al. [21] proposed an open-world object detector ORE, which was based on contrastive clustering and energy-based unknown identification. Identifying and characterizing unknown instances helped reduce confusion in incremental object detection settings. In this setting, state-of-the-art performance could be achieved without additional methodologies.In 2022, Geng and Chen [22] proposed a batch decision strategy that was aimed at extending existing open-set recognition methods for new class discovery while considering correlations between test instances. By modifying the Hierarchical Dirichlet Process (HDP), a collective decision-based open-set recognition framework CD-OSR was proposed. CD-OSR did not need to define decision thresholds and could realize open-set recognition and new class discovery at the same time.Table1 summarizes several open-set recognition methods mentioned in Section 2.1.Table 1
Different models for open-set recognition.
ModelReferenceOpen-set recognition categoryMethodologyExtreme value theoryAdvantagesOpenMax[15]Discriminative modelEVT-based calibration classification√First deep open-set classifier without using background samplesG-OpenMax[16]Generative modelUnknown generation classification√Combining generative adversarial networks and OpenMaxCROSR[17]Discriminative modelDistance√First neural network architecture which involved hierarchical reconstruction blocksC2AE[18]Discriminative modelReconstruction error√Algorithms using class conditional autoencodersNeural-network-based representation[19]Discriminative modelEVT-free calibration classification×A loss function was proposed such that instances from the same class are close to each other, while instances from different classes are farther apartPEELER[20]Discriminative modelDistance×Combining few-shot classification and open-set recognitionORE[21]Discriminative modelDistance×An incremental object detector is proposedCD-OSR[22]Discriminative modelEVT-free calibration classification×Automatically reserve space for unknown classes under test, naturally bringing new class discovery capabilities


## 2.2. Unknown Traffic Identification

In the field of network traffic analysis, unknown traffic identification has always been an important research direction. Researchers usually use unsupervised learning or semisupervised learning to solve the tasks of identifying and detecting unknown traffic.In 2011, Finamore et al. [23] proposed an unsupervised algorithm to identify traffic classes within aggregates. This algorithm utilized the K-means clustering algorithm and added a mechanism to automatically determine the number of traffic clusters.In 2013, Zhang et al. [24] proposed an approach to address the problem of unknown applications in the critical case of small supervised training sets. The proposed method had a superior ability to detect unknown traffic generated by unknown applications and exploited the correlation information between real-world network traffic to improve the classification performance.In 2013, Zhang et al. [25] proposed an iterative method to extract unknown information from a set of unlabeled traffic flows. The method combined asymmetric bagging and flow correlation to guarantee the purity of the extracted negatives and demonstrated significantly better than state-of-the-art flow classification methods under unknown applications.In 2014, Yu et al. [26] proposed a method to classify elephant traffic using service-based statistical features for cluster analysis. Elephant traffic refers to unknown traffic generated by only a few or some types of applications.In 2015, Shaikh and Harkut [27] proposed a framework that classifies unknown flows in the network, solving the problem of applying unknowns in critical situations with little supervised training data. Flow label propagation was proposed, which automatically and accurately labeled more unlabeled flows to enhance the ability of Nearest Clustering-based Classifiers (NCCs). Composite classification was also proposed, which combines many flow predictions to more accurately classified Bag of Flows (BoF).In 2015, Lin et al. [28] proposed a semisupervised learning method to address the problem of an unknown protocol in critical cases where the labeled training sample set was small. With the help of flow-related information and semisupervised clustering ensemble learning, the method had a superior ability to detect unknown samples generated by unknown protocols to improve classification performance.In 2017, Ma and Qin [29] proposed a method using deep learning techniques to identify unknown protocols in complex network environments. The method identified the protocol in the network flow according to the application layer protocol type and found out the unknown protocol. This method only used the payload information in the captured 200,000 traffic flows and achieved well unknown protocol traffic identification accuracy.In 2018, Fu et al. [30] proposed a scheme, FlowCop, to implement traffic detection that did not belong to any predefined application in network traffic classification. It divided the test traffic into N classes and one unknown class by building multiple one-class classifiers. A feature subspace algorithm was also proposed to select salient features for each class of classifiers.In 2019, Sabeel et al. [31] proposed two methods to predict unknown DoS and DDoS attacks based on DNN and LSTM. The method demonstrated how well deep learning-based methods perform in unknown situations and to what extent deviations from the trained model could be handled. This method can effectively identify unknown attacks.In 2019, Zhang et al. [32] proposed a network intrusion detection method based on open-set recognition. This method fits the recognition results of known classes to Weibull distribution and then builds an Open-CNN model to estimate the probability of unknown classes from the activation scores of known classes, so as to achieve the purpose of detecting unknown attacks.In 2020, Zhang et al. [33] proposed an autonomous learning framework to correctly classify unknown classes. The framework efficiently updated deep learning-based traffic classification models during active operation. The core of the proposed framework consisted of a deep learning-based classifier, a self-learning discriminator, and an autonomous self-labeling model. The discriminator and self-labeling process generated new datasets during active operation to support classifier updates.In 2020, Mohamed et al. [34] proposed a method for handling unknown applications. This method enabled efficient network classification with limited supervised training sets. The proposed model applied multiple neural network algorithms to predict unknown applications. The method improved Internet performance, reduced Internet traffic, and reduced delays in transmitting data.In 2021, Wang et al. [35] proposed an unknown protocol parsing method based on a convolutional neural network. The protocol data was preprocessed into an image, and the converted image was inputted to the convolution layer for convolution. After convolution, the data was flattened, and the flattened data was put into a fully connected neural network to analyze and predict unknown protocols.In 2021, Li et al. [36] proposed a lightweight unknown traffic discovery model, LightSEEN, which realized traffic classification and model update in the open world under practical conditions. The overall structure of the method was based on the Siamese network, and each side used a multihead attention mechanism, a one-dimensional convolutional neural network, and a residual network to facilitate the extraction of deep flow features and the convergence speed of the network.In 2021, Xu et al. [37] proposed the KCC (Known Central Clustering) method to deal with the open-set-based intrusion detection problem. By introducing CD-loss (Class Distance-loss), the centers of different clusters were obtained. By introducing negative samples as unknown classes for training, the threshold of known classes was obtained. Unknown intrusions were rejected by comparing with fuzzy distances.Table2 makes a simple classification and summary according to the specific methods used in the above-mentioned several unknown traffic identification literatures. We combine unknown traffic identification with open-set recognition. According to the characteristics of network encrypted traffic, we design a new open-set recognition method based on a discriminant model. Through self-supervised learning and supervised learning, it can effectively complete known traffic classification and unknown traffic identification tasks at the same time.Table 2
Summary of methods for identifying unknown traffic.
Statistical methodsDeep neural networksSupervised learningSemisupervised learningUnsupervised learningCNN-basedOpen-set recognitionOthers[25][28][23, 24, 26, 27, 30][29, 32, 33, 35, 37][32, 35, 37][31, 34]


## 3. Proposed Method

To identify unknown classes that have not appeared in the training set, we propose OpenCBD, a deep learning and ensemble learning-based approach. The CBD model was proposed in [12], but it can only complete the classification task in datasets with known class traffic. In this paper, the CBD model is regarded as an individual model. First, the loss function is used to train the individual model, and then through the ensemble strategy, the individual model is fused into an ensemble model, which is extended to OpenCBD suitable for open-set data. Classify known classes and identify unknown classes in the world. The CBD model includes the CNN module, BERT module, and dense module. The detailed structure is given in Section 3.3. The overall process of OpenCBD is shown in Figure 1.Figure 1
Overall flow chart of proposed method. The orange part represents the pretraining process, the blue part represents the training process, the green part represents the testing process, and the black part represents the process involved in both training and testing processes.First, pretraining is performed on unlabeled data to obtain a pretrained individual model, including a CNN module with a fixed structure and parameters and a BERT module with a fixed structure and no fixed parameters. Then, perform data preprocessing on the raw data participating in the training, then input them into multiple individual models for training after obtaining the training set, then input the training set into the ensemble model for training, and then obtain the training results. Finally, data preprocessing is performed on the raw data participating in the test, and after the test set is obtained, it is inputted into the trained ensemble model, and the prediction results are the outputs.The training process includes two stages, namely, closed-set individual training and closed-set ensemble training. The testing process is implemented in the open set. The block diagram of the three stages is shown in Figure2.Figure 2
Block diagram of proposed method. (1) Closed-set individual training: randomly select a part of the known class data to train the individual model including the encoder and the classifier. 2) Closed-set ensemble training: using all known class data to train encoders and classifiers inw individual models, where the weights of the encoders are locked. (3) Open-set testing: the ensemble model produces M results to be identified. If the largest to-be-identified result is greater than or equal to the threshold, the test sample is classified into one of the M classes; otherwise, it is classified as unknown.In the following, we will introduce the data preprocessing process, pretraining process, and detailed process of the three stages.


### 3.1. Data Preprocessing

Data preprocessing is an essential step to achieve the goals of classification and identification. Data preprocessing mainly includes four parts: traffic split, traffic cleaning, traffic conversion, and time interval integration. Following the description in [12], we summarize the preprocessing process as Algorithm 1.Algorithm 1: Preprocessing algorithm for traffic data.
Input: the original network traffic dataset D, the number of classes of traffic c.Output: packet vector set X;1:Fori=1,⋯,cdo2:  Randomly selectni consecutive 10 packets to form ni flows;3:Forj=1,⋯,10nido4:  Read the payload part of the packetxi,j;5:Ifxi,j<2048then6:xi,j=xi,j,zeros1,2048−xi,j;7:Else8:xi,j=xi,j1:2048;9:End if10:xi,j=intxi,j′,10;11:couti,j=timexi,j+1−timexi,j;12:Ifcouti,j<1 second then13:   Continue;14:  Else15:   Addx0 between xi,j and xi,j+1;16:End if17:End for18: Generate packet vector setXi=xi,1,⋯,x0,⋯,xi,10ni;19:End for20:Return packet vector set X=X1⋃X2⋃⋯⋃Xc.Line 2 is the traffic split. For bidirectional flow, randomly intercept 10 consecutive data packets and define these ten packets as a flow. A total ofn flows are intercepted, that is, a total of 10n data packets.Lines 4-9 are the traffic cleaning. Read the payload part of each packet, then unify the length, truncate the first 256 bytes of each packet, and add 0 to the insufficient, to obtain the raw sequencex,
(9)x=b1,b2,⋯,b8×256,bi∈F2,i=1,⋯,2048.Line 10 is the traffic conversion. Convert the elements in the raw sequence into decimals according to bytes to obtain a 256-dimensional vectorx,
(10)x=x1,x2,⋯,x256,xi∈F28,i=1,⋯,256.Lines 11-16 are the time interval integration, which was first proposed in [38]. It means that according to the statistical results of the time interval of two adjacent data packets in different classes, for the interval of more than 1 s, insert a blank data packet and ignore it within 1 s. The blank data packet is represented as a 256-dimensional all-one vector x0=1,⋯,1,⋯,1256.The payload vectorx and the time interval vector x0 are formed into a set X in chronological order, and the model can be directly inputted in the next step.


### 3.2. Pretraining

Pretraining adopts a common pretraining method in the field of encrypted traffic analysis proposed in [12], which starts from the packet level and the flow level, and can directly deepen the model’s understanding of encrypted traffic from unlabeled real-world data. This paper summarizes the method into a detailed Algorithm 2.Algorithm 2: Pretraining algorithm for network traffic model.
Input: unlabeled dataset D1=x1,1,⋯,x1,N, D2.Output: pretrained model;1:Fori=1,⋯,Ndo2:  Extract the payload ofx1,i, x1,i=x1,i1,x1,i2,⋯,x1,in;3:Forj=1,⋯,ndo4:Hx1,i=0;5:Hx1,ij=−Px1,ij·log2Px1,ij;6:Hx1,i=Hx1,i+Hx1,ij;7:End for8:IfHx1,i<H0then9:labx1,i=0 (plaintext);10:Else11:labx1,i=1 (ciphertext);12:End if13:End for14:D1′=x1,1,labx1,1,⋯,x1,N,labx1,N;15:Model=ModelD1′;16: Randomly selectm consecutive 10-packet payload parts in D2 to form S+, S+=s1+,⋯,sm+;17:Fora=1,⋯,mdo18:sa+≜Fa=xa,1,xa,2,⋯,xa,10;19:S−=∅;20:Forb=1,⋯,10do21:IfP=0.3then22:fxa,b=xa′,b′, where a′,b′≠a,b;23:Else24:fxa,b=xa,b;25:End if26:sa−≜Fa′=fxa,1,fxa,2,⋯,fxa,10;27:S−=S−⋃sa−;28:End for29:End for30:D2′=S+⋃S−;31:Model=ModelD2′;32:Return pretrained model.Lines 1-15 are the packet-based methods. For an unlabeled packet, first extract the payload part to getx=x1,x1,⋯,xn, and then calculate the entropy of each packet,
(11)H=−∑i=1nPxilog2Pxi,1≤i≤n,Hx<H0,x∈Xplain,Hx≥H0,x∈Xcipher.Set the threshold of entropyH0=4. When H≥H0, the data packet label is a ciphertext data packet; when H<H0, the data packet label is a plaintext data packet. Train a CBD model with labeled packets.Lines 16-31 are the flow-based methods. First, construct positive and negative sample sets. The positive sample setS+ contains n positive samples, and the positive sample s+ is defined as a continuous flow F; each flow contains 10 consecutive packets,
(12)S+=s1+,s2+,⋯,sn+,si+≜Fi=x1i,x2i,⋯,x10i,1≤i≤n.The number of negative sample setS− is the same as the positive sample set S+, including n negative samples, and the negative sample s− is defined as a discontinuous flow F¯. It is obtained by transforming the positive samples. Each packet in the positive sample is replaced with other bags with a certain probability, and the replaced sample is called a negative sample.
(13)S−=s1−,s2−,⋯,sn−,si−≜F¯i=fx1i,fx2i,⋯,fx10i,1≤i≤n,fxji=xjiP=0.7xj′i′P=0.3,i′,j′≠i,j,1≤j≤10.Train a CBD model with a labeled set of positive and negative samples. After completing the pretraining, enter the three-stage training and testing process.


### 3.3. Closed-Set Individual Training

The first stage is closed-set individual training. After the data is preprocessed, thek class data is randomly selected to train the individual model, and the k classification is completed. Each individual model is a randomly selected k class with replacement. The specific structure diagram of the individual model is shown in Figure 3.Figure 3
Structure diagram of individual model.


#### 3.3.1. Encoder

The encoder and classifier are two important parts in the individual model. The encoder is mainly used for feature extraction, the input is a matrixX, and the output is a vector v,
(14)Encoder:X⟶v∈R240,whereX is a n×256-dimensional matrix on F28. The encoder mainly includes the CNN module and BERT module.The CNN module is inputted in sequence by a row vector; that is, for the inputX=x1T,⋯,xnT of the encoder, xi represents the i-th input of the CNN module, i=1,2,⋯,n. Define fC· as the CNN module function, fconj· is the convolution function of the j-th layer in the CNN module, fmpj· is the maximum pooling function of the j-th layer in CNN module, and Oij is the j-th layer output for the i-th input in the CNN module, j=1,2,3,4; then, the output of the CNN module is
(15)fCxi=Oi4∈R242,Oi4=max0,fmp4fcon4Oi3,Oi3=max0,fmp3fcon3Oi2,Oi2=max0,fmp2fcon2Oi1,Oi1=max0,fcon1xi.After the CNN module is a fully connected layer, which is used to change the dimension of the vector to facilitate the input of the subsequent BERT module. Defineffc· to be the fully connected function and zi∈R240 to be the output of the fully connected layer, then
(16)zi≜ffcOi4=W1Oi4+Bi=WifCxi+Bi,whereWi is the weight matrix and Bi is the bias matrix.This is followed by a Concat layer that stitches together the outputs aftern CNN modules and fully connected layers. Define ⊕ as the splicing symbol and Z as the output of the Concat layer, then
(17)Z=z1T,z2T,⋯,znT=z1⊕z2⊕⋯⊕zn=⊕i=1nzi,Z∈ℝn×240.The BERT module consists ofm transformer encoders, and Z is the input of the BERT module. Define fB· as the BERT module function, ftet· as the t-th layer transformer encoder function in the BERT module, and v as the output of the encoder, then
(18)v≜fBZ=ftem⋯ftet⋯fte1Z,t=1,2,⋯,m.


#### 3.3.2. Classifier

The classifier is mainly used to predict results, the input is a vectorv, and the output is a specific class y,
(19)Classifier:v⟶y∈1,2,⋯,k.The classifier consists of a dense module and Softmax layer. In the dense module, definefD· to be the function of the dense module and FX to be the output of the dense module, then
(20)FX≜fDv=max0,Wv+B,whereW represents the weight matrix and B represents the bias matrix.The last is the Softmax layer, definingfsm· as the Softmax function, then
(21)y=fsmfDv=argmaxlPy=lX,Py=lX=eFlX∑p=1keFpX,wherel,p∈1,2,⋯,k is the index of the class.So the output of the entire individual model is(22)y=fsmfDv=fsmfDfBZ=fsmfDfB⊕i=1nzi=fsmfDfB⊕i=1nWifCxi+Bi.


#### 3.3.3. Loss Function

Individual models are trained with a combination of cross-entropy loss and II-Loss [19].The cross-entropy loss function is often used in classification problems, which can measure the similarity between several classes. Given a batch of samplesX1,X2,⋯,XA, for a matrix Xa,a=1,2,⋯,A, define its label as labXa∈1,2,⋯,k, then the cross-entropy loss of this batch of samples is
(23)CE‐Loss=−1A∑a=1A∑l=1ksgnXa,llogPXa,l,sgnXa,l=1,labXa=l,0,labXa≠l,(24)PXa,l=PlabXa=l,wheresgnXa,l is a symbolic function, which determines whether the label of Xa is the class l. PXa,l is a probability function that calculates the probability that the label of Xa is the class l.The II-Loss function can make samples of different classes farther apart and samples of the same class closer by maximizing the distance between different classes and minimizing the distance between samples and their class mean. Given a batch of samplesX1,X2,⋯,XA, define the sample set of class l as Cl, the number of samples in Cl is Cl, the mean output of the dense module in Cl is μl, then
(25)X1,X2,⋯,XA=C1∪C2∪⋯∪Ck,μl=1Cl∑q=1ClFXq,q=1,2,⋯,Cl.The II-Loss of this batch of samples is(26)II‐Loss=Disintra_class−Disinter_class=1A∑l=1k∑i=1Clμl−FXl22−min1≤p,r≤k&p=rμp−μr22.The final loss function is(27)Loss=CE‐Loss+II‐Loss=−1A∑a=1A∑l=1ksgnXa,llogPXa,l+1A∑l=1k∑i=1Clμl−FXl22−min1≤p,r≤k&p=rμp−μr22.


### 3.4. Closed-Set Ensemble Training

The second stage is closed-set ensemble training. After the data is preprocessed, the ensemble model is trained with allN class known data to complete N-classification. The integrated strategy of the ensemble model adopts the learning method, which is a method of combining individual learners by training the learners. First, the encoder in the individual model is trained using a subset of the raw training dataset. Then, the ensemble model is trained with the raw training set using the output of the encoder as a feature. See Algorithm 3 for the ensemble strategy.Algorithm 3: Ensemble strategy.
Input: known classes data D=X1,L1,X2,L2,⋯,XR,LR; individual learning algorithm I1,I2,⋯,Iw; ensemble learning algorithm E.Output: ensemble model H;1: Randomly selectedk classes data from D to form D′, D′⊂D;2:Fors=1,2,⋯,wdo3:hs=ItD′;4:End for5:D∗=∅;6:Fori=1,2,⋯,Rdo7:Fors=1,2,⋯,wdo8:zrs=hsXr;9:End for10:D∗=D\D′∪zr1,zr2,⋯,zrw,yr;11:End for12:h∗=ED∗;13:HX=h∗h1X,h2X,⋯,hwX.The training of the ensemble model takest-fold cross-validation as an example and divides the known class dataset D and subset D′ into t datasets D1,⋯,Dt and D′1,⋯,D′t, respectively. Let D′j and D′−j=D′\D′j be the test set and training set corresponding to the jth individual model execution, respectively. Dj and D−j=DDj are the test set and training set corresponding to the jth ensemble model execution, respectively. Given w individual learning algorithms, use the sth learning algorithm to train on D′−j to obtain an individual learner hs−j. For each sample Xi in the test set D′j executed at the jth time, let zis be the learner hs−j on the Xi output result. Then, at the end of the entire cross-validation process, a new dataset can be generated by w individual learners D∗=zi1,⋯,ziw,yii=1R; the ensemble learner will use this dataset to train together with the dataset D\D′ that does not participate in the training of individual models.Lines 1-4 are the training of the individual model, which only uses a subsetD′ of the known class data D during training. Lines 5-12 are the training of the ensemble model, which uses all of the known class data D during training. The ensemble model consists of w encoders and 1 classifier. The encoder is obtained from the first stage of training, and the fixed structure and parameters remain unchanged. The classifier has the same structure as the first stage, but the specific parameters have changed. Define X as the input of the ensemble model, ypred as the output, fD∗· as the dense module function, GX as the dense module output, and fsm∗· as the Softmax function, then
(28)Ensemblemodel:X⟶ypred∈1,2,⋯,N,GX≜fD∗∑s=1wvs,ypred=fsm∗GX=fsm∗fD∗∑s=1wvs=argmaxl′Py=l′X,Py=l′X=eGl′X∑p′=1NeGp′X,wherew is the number of encoders in the ensemble model, each encoder comes from the corresponding individual model, and the coefficients remain unchanged. fD∗· and fsm∗· have the same structure as the dense module function fD· and Softmax function fsm· in the individual model but with different coefficients. l′,p′∈1,2,⋯,N is the index of the class.


### 3.5. Open-Set Testing

The third stage is open-set testing. The real-world data is inputted into the trained ensemble model, and the ensemble model producesM results Rec to be identified. Compute the maximum of the M results. If the maximum value is greater than or equal to the threshold, the test sample is classified as one of M classes; otherwise, it is classified as unknown.


#### 3.5.1. Threshold Estimation

The threshold for judging data classes in open-set testing is determined by outliers. Assume that 1% of the samples in the training set have noise, that is, they are abnormal outlier samples. When calculating the outlier distance for all training samples, sort them from small to large, and the largest 1% distance is the threshold. The detailed process of threshold estimation is shown in Algorithm4.Algorithm 4: Threshold estimation.
Input: training set X1,X2,⋯,XA; known classes N.Output: threshold τ;1:Fori=1,2,⋯,Ado2:Forj=1,2,⋯,Ndo3:Py=jXi=eFjXi/∑p=1keFpXi;4:End for5:outlierXi=maxjPy=jXi;6:End for7:sortoutlier;8: Threshold set = last 1%sortoutlier;9:Returnτ=minthresholdset.After the threshold is determined, the to-be-identified results of the ensemble model will determine the classes of data according to whether it is greater than or equal to the threshold.


## 3.1. Data Preprocessing

Data preprocessing is an essential step to achieve the goals of classification and identification. Data preprocessing mainly includes four parts: traffic split, traffic cleaning, traffic conversion, and time interval integration. Following the description in [12], we summarize the preprocessing process as Algorithm 1.Algorithm 1: Preprocessing algorithm for traffic data.
Input: the original network traffic dataset D, the number of classes of traffic c.Output: packet vector set X;1:Fori=1,⋯,cdo2:  Randomly selectni consecutive 10 packets to form ni flows;3:Forj=1,⋯,10nido4:  Read the payload part of the packetxi,j;5:Ifxi,j<2048then6:xi,j=xi,j,zeros1,2048−xi,j;7:Else8:xi,j=xi,j1:2048;9:End if10:xi,j=intxi,j′,10;11:couti,j=timexi,j+1−timexi,j;12:Ifcouti,j<1 second then13:   Continue;14:  Else15:   Addx0 between xi,j and xi,j+1;16:End if17:End for18: Generate packet vector setXi=xi,1,⋯,x0,⋯,xi,10ni;19:End for20:Return packet vector set X=X1⋃X2⋃⋯⋃Xc.Line 2 is the traffic split. For bidirectional flow, randomly intercept 10 consecutive data packets and define these ten packets as a flow. A total ofn flows are intercepted, that is, a total of 10n data packets.Lines 4-9 are the traffic cleaning. Read the payload part of each packet, then unify the length, truncate the first 256 bytes of each packet, and add 0 to the insufficient, to obtain the raw sequencex,
(9)x=b1,b2,⋯,b8×256,bi∈F2,i=1,⋯,2048.Line 10 is the traffic conversion. Convert the elements in the raw sequence into decimals according to bytes to obtain a 256-dimensional vectorx,
(10)x=x1,x2,⋯,x256,xi∈F28,i=1,⋯,256.Lines 11-16 are the time interval integration, which was first proposed in [38]. It means that according to the statistical results of the time interval of two adjacent data packets in different classes, for the interval of more than 1 s, insert a blank data packet and ignore it within 1 s. The blank data packet is represented as a 256-dimensional all-one vector x0=1,⋯,1,⋯,1256.The payload vectorx and the time interval vector x0 are formed into a set X in chronological order, and the model can be directly inputted in the next step.


## 3.2. Pretraining

Pretraining adopts a common pretraining method in the field of encrypted traffic analysis proposed in [12], which starts from the packet level and the flow level, and can directly deepen the model’s understanding of encrypted traffic from unlabeled real-world data. This paper summarizes the method into a detailed Algorithm 2.Algorithm 2: Pretraining algorithm for network traffic model.
Input: unlabeled dataset D1=x1,1,⋯,x1,N, D2.Output: pretrained model;1:Fori=1,⋯,Ndo2:  Extract the payload ofx1,i, x1,i=x1,i1,x1,i2,⋯,x1,in;3:Forj=1,⋯,ndo4:Hx1,i=0;5:Hx1,ij=−Px1,ij·log2Px1,ij;6:Hx1,i=Hx1,i+Hx1,ij;7:End for8:IfHx1,i<H0then9:labx1,i=0 (plaintext);10:Else11:labx1,i=1 (ciphertext);12:End if13:End for14:D1′=x1,1,labx1,1,⋯,x1,N,labx1,N;15:Model=ModelD1′;16: Randomly selectm consecutive 10-packet payload parts in D2 to form S+, S+=s1+,⋯,sm+;17:Fora=1,⋯,mdo18:sa+≜Fa=xa,1,xa,2,⋯,xa,10;19:S−=∅;20:Forb=1,⋯,10do21:IfP=0.3then22:fxa,b=xa′,b′, where a′,b′≠a,b;23:Else24:fxa,b=xa,b;25:End if26:sa−≜Fa′=fxa,1,fxa,2,⋯,fxa,10;27:S−=S−⋃sa−;28:End for29:End for30:D2′=S+⋃S−;31:Model=ModelD2′;32:Return pretrained model.Lines 1-15 are the packet-based methods. For an unlabeled packet, first extract the payload part to getx=x1,x1,⋯,xn, and then calculate the entropy of each packet,
(11)H=−∑i=1nPxilog2Pxi,1≤i≤n,Hx<H0,x∈Xplain,Hx≥H0,x∈Xcipher.Set the threshold of entropyH0=4. When H≥H0, the data packet label is a ciphertext data packet; when H<H0, the data packet label is a plaintext data packet. Train a CBD model with labeled packets.Lines 16-31 are the flow-based methods. First, construct positive and negative sample sets. The positive sample setS+ contains n positive samples, and the positive sample s+ is defined as a continuous flow F; each flow contains 10 consecutive packets,
(12)S+=s1+,s2+,⋯,sn+,si+≜Fi=x1i,x2i,⋯,x10i,1≤i≤n.The number of negative sample setS− is the same as the positive sample set S+, including n negative samples, and the negative sample s− is defined as a discontinuous flow F¯. It is obtained by transforming the positive samples. Each packet in the positive sample is replaced with other bags with a certain probability, and the replaced sample is called a negative sample.
(13)S−=s1−,s2−,⋯,sn−,si−≜F¯i=fx1i,fx2i,⋯,fx10i,1≤i≤n,fxji=xjiP=0.7xj′i′P=0.3,i′,j′≠i,j,1≤j≤10.Train a CBD model with a labeled set of positive and negative samples. After completing the pretraining, enter the three-stage training and testing process.


## 3.3. Closed-Set Individual Training

The first stage is closed-set individual training. After the data is preprocessed, thek class data is randomly selected to train the individual model, and the k classification is completed. Each individual model is a randomly selected k class with replacement. The specific structure diagram of the individual model is shown in Figure 3.Figure 3
Structure diagram of individual model.


### 3.3.1. Encoder

The encoder and classifier are two important parts in the individual model. The encoder is mainly used for feature extraction, the input is a matrixX, and the output is a vector v,
(14)Encoder:X⟶v∈R240,whereX is a n×256-dimensional matrix on F28. The encoder mainly includes the CNN module and BERT module.The CNN module is inputted in sequence by a row vector; that is, for the inputX=x1T,⋯,xnT of the encoder, xi represents the i-th input of the CNN module, i=1,2,⋯,n. Define fC· as the CNN module function, fconj· is the convolution function of the j-th layer in the CNN module, fmpj· is the maximum pooling function of the j-th layer in CNN module, and Oij is the j-th layer output for the i-th input in the CNN module, j=1,2,3,4; then, the output of the CNN module is
(15)fCxi=Oi4∈R242,Oi4=max0,fmp4fcon4Oi3,Oi3=max0,fmp3fcon3Oi2,Oi2=max0,fmp2fcon2Oi1,Oi1=max0,fcon1xi.After the CNN module is a fully connected layer, which is used to change the dimension of the vector to facilitate the input of the subsequent BERT module. Defineffc· to be the fully connected function and zi∈R240 to be the output of the fully connected layer, then
(16)zi≜ffcOi4=W1Oi4+Bi=WifCxi+Bi,whereWi is the weight matrix and Bi is the bias matrix.This is followed by a Concat layer that stitches together the outputs aftern CNN modules and fully connected layers. Define ⊕ as the splicing symbol and Z as the output of the Concat layer, then
(17)Z=z1T,z2T,⋯,znT=z1⊕z2⊕⋯⊕zn=⊕i=1nzi,Z∈ℝn×240.The BERT module consists ofm transformer encoders, and Z is the input of the BERT module. Define fB· as the BERT module function, ftet· as the t-th layer transformer encoder function in the BERT module, and v as the output of the encoder, then
(18)v≜fBZ=ftem⋯ftet⋯fte1Z,t=1,2,⋯,m.


### 3.3.2. Classifier

The classifier is mainly used to predict results, the input is a vectorv, and the output is a specific class y,
(19)Classifier:v⟶y∈1,2,⋯,k.The classifier consists of a dense module and Softmax layer. In the dense module, definefD· to be the function of the dense module and FX to be the output of the dense module, then
(20)FX≜fDv=max0,Wv+B,whereW represents the weight matrix and B represents the bias matrix.The last is the Softmax layer, definingfsm· as the Softmax function, then
(21)y=fsmfDv=argmaxlPy=lX,Py=lX=eFlX∑p=1keFpX,wherel,p∈1,2,⋯,k is the index of the class.So the output of the entire individual model is(22)y=fsmfDv=fsmfDfBZ=fsmfDfB⊕i=1nzi=fsmfDfB⊕i=1nWifCxi+Bi.


### 3.3.3. Loss Function

Individual models are trained with a combination of cross-entropy loss and II-Loss [19].The cross-entropy loss function is often used in classification problems, which can measure the similarity between several classes. Given a batch of samplesX1,X2,⋯,XA, for a matrix Xa,a=1,2,⋯,A, define its label as labXa∈1,2,⋯,k, then the cross-entropy loss of this batch of samples is
(23)CE‐Loss=−1A∑a=1A∑l=1ksgnXa,llogPXa,l,sgnXa,l=1,labXa=l,0,labXa≠l,(24)PXa,l=PlabXa=l,wheresgnXa,l is a symbolic function, which determines whether the label of Xa is the class l. PXa,l is a probability function that calculates the probability that the label of Xa is the class l.The II-Loss function can make samples of different classes farther apart and samples of the same class closer by maximizing the distance between different classes and minimizing the distance between samples and their class mean. Given a batch of samplesX1,X2,⋯,XA, define the sample set of class l as Cl, the number of samples in Cl is Cl, the mean output of the dense module in Cl is μl, then
(25)X1,X2,⋯,XA=C1∪C2∪⋯∪Ck,μl=1Cl∑q=1ClFXq,q=1,2,⋯,Cl.The II-Loss of this batch of samples is(26)II‐Loss=Disintra_class−Disinter_class=1A∑l=1k∑i=1Clμl−FXl22−min1≤p,r≤k&p=rμp−μr22.The final loss function is(27)Loss=CE‐Loss+II‐Loss=−1A∑a=1A∑l=1ksgnXa,llogPXa,l+1A∑l=1k∑i=1Clμl−FXl22−min1≤p,r≤k&p=rμp−μr22.


## 3.3.1. Encoder

The encoder and classifier are two important parts in the individual model. The encoder is mainly used for feature extraction, the input is a matrixX, and the output is a vector v,
(14)Encoder:X⟶v∈R240,whereX is a n×256-dimensional matrix on F28. The encoder mainly includes the CNN module and BERT module.The CNN module is inputted in sequence by a row vector; that is, for the inputX=x1T,⋯,xnT of the encoder, xi represents the i-th input of the CNN module, i=1,2,⋯,n. Define fC· as the CNN module function, fconj· is the convolution function of the j-th layer in the CNN module, fmpj· is the maximum pooling function of the j-th layer in CNN module, and Oij is the j-th layer output for the i-th input in the CNN module, j=1,2,3,4; then, the output of the CNN module is
(15)fCxi=Oi4∈R242,Oi4=max0,fmp4fcon4Oi3,Oi3=max0,fmp3fcon3Oi2,Oi2=max0,fmp2fcon2Oi1,Oi1=max0,fcon1xi.After the CNN module is a fully connected layer, which is used to change the dimension of the vector to facilitate the input of the subsequent BERT module. Defineffc· to be the fully connected function and zi∈R240 to be the output of the fully connected layer, then
(16)zi≜ffcOi4=W1Oi4+Bi=WifCxi+Bi,whereWi is the weight matrix and Bi is the bias matrix.This is followed by a Concat layer that stitches together the outputs aftern CNN modules and fully connected layers. Define ⊕ as the splicing symbol and Z as the output of the Concat layer, then
(17)Z=z1T,z2T,⋯,znT=z1⊕z2⊕⋯⊕zn=⊕i=1nzi,Z∈ℝn×240.The BERT module consists ofm transformer encoders, and Z is the input of the BERT module. Define fB· as the BERT module function, ftet· as the t-th layer transformer encoder function in the BERT module, and v as the output of the encoder, then
(18)v≜fBZ=ftem⋯ftet⋯fte1Z,t=1,2,⋯,m.


## 3.3.2. Classifier

The classifier is mainly used to predict results, the input is a vectorv, and the output is a specific class y,
(19)Classifier:v⟶y∈1,2,⋯,k.The classifier consists of a dense module and Softmax layer. In the dense module, definefD· to be the function of the dense module and FX to be the output of the dense module, then
(20)FX≜fDv=max0,Wv+B,whereW represents the weight matrix and B represents the bias matrix.The last is the Softmax layer, definingfsm· as the Softmax function, then
(21)y=fsmfDv=argmaxlPy=lX,Py=lX=eFlX∑p=1keFpX,wherel,p∈1,2,⋯,k is the index of the class.So the output of the entire individual model is(22)y=fsmfDv=fsmfDfBZ=fsmfDfB⊕i=1nzi=fsmfDfB⊕i=1nWifCxi+Bi.


## 3.3.3. Loss Function

Individual models are trained with a combination of cross-entropy loss and II-Loss [19].The cross-entropy loss function is often used in classification problems, which can measure the similarity between several classes. Given a batch of samplesX1,X2,⋯,XA, for a matrix Xa,a=1,2,⋯,A, define its label as labXa∈1,2,⋯,k, then the cross-entropy loss of this batch of samples is
(23)CE‐Loss=−1A∑a=1A∑l=1ksgnXa,llogPXa,l,sgnXa,l=1,labXa=l,0,labXa≠l,(24)PXa,l=PlabXa=l,wheresgnXa,l is a symbolic function, which determines whether the label of Xa is the class l. PXa,l is a probability function that calculates the probability that the label of Xa is the class l.The II-Loss function can make samples of different classes farther apart and samples of the same class closer by maximizing the distance between different classes and minimizing the distance between samples and their class mean. Given a batch of samplesX1,X2,⋯,XA, define the sample set of class l as Cl, the number of samples in Cl is Cl, the mean output of the dense module in Cl is μl, then
(25)X1,X2,⋯,XA=C1∪C2∪⋯∪Ck,μl=1Cl∑q=1ClFXq,q=1,2,⋯,Cl.The II-Loss of this batch of samples is(26)II‐Loss=Disintra_class−Disinter_class=1A∑l=1k∑i=1Clμl−FXl22−min1≤p,r≤k&p=rμp−μr22.The final loss function is(27)Loss=CE‐Loss+II‐Loss=−1A∑a=1A∑l=1ksgnXa,llogPXa,l+1A∑l=1k∑i=1Clμl−FXl22−min1≤p,r≤k&p=rμp−μr22.


## 3.4. Closed-Set Ensemble Training

The second stage is closed-set ensemble training. After the data is preprocessed, the ensemble model is trained with allN class known data to complete N-classification. The integrated strategy of the ensemble model adopts the learning method, which is a method of combining individual learners by training the learners. First, the encoder in the individual model is trained using a subset of the raw training dataset. Then, the ensemble model is trained with the raw training set using the output of the encoder as a feature. See Algorithm 3 for the ensemble strategy.Algorithm 3: Ensemble strategy.
Input: known classes data D=X1,L1,X2,L2,⋯,XR,LR; individual learning algorithm I1,I2,⋯,Iw; ensemble learning algorithm E.Output: ensemble model H;1: Randomly selectedk classes data from D to form D′, D′⊂D;2:Fors=1,2,⋯,wdo3:hs=ItD′;4:End for5:D∗=∅;6:Fori=1,2,⋯,Rdo7:Fors=1,2,⋯,wdo8:zrs=hsXr;9:End for10:D∗=D\D′∪zr1,zr2,⋯,zrw,yr;11:End for12:h∗=ED∗;13:HX=h∗h1X,h2X,⋯,hwX.The training of the ensemble model takest-fold cross-validation as an example and divides the known class dataset D and subset D′ into t datasets D1,⋯,Dt and D′1,⋯,D′t, respectively. Let D′j and D′−j=D′\D′j be the test set and training set corresponding to the jth individual model execution, respectively. Dj and D−j=DDj are the test set and training set corresponding to the jth ensemble model execution, respectively. Given w individual learning algorithms, use the sth learning algorithm to train on D′−j to obtain an individual learner hs−j. For each sample Xi in the test set D′j executed at the jth time, let zis be the learner hs−j on the Xi output result. Then, at the end of the entire cross-validation process, a new dataset can be generated by w individual learners D∗=zi1,⋯,ziw,yii=1R; the ensemble learner will use this dataset to train together with the dataset D\D′ that does not participate in the training of individual models.Lines 1-4 are the training of the individual model, which only uses a subsetD′ of the known class data D during training. Lines 5-12 are the training of the ensemble model, which uses all of the known class data D during training. The ensemble model consists of w encoders and 1 classifier. The encoder is obtained from the first stage of training, and the fixed structure and parameters remain unchanged. The classifier has the same structure as the first stage, but the specific parameters have changed. Define X as the input of the ensemble model, ypred as the output, fD∗· as the dense module function, GX as the dense module output, and fsm∗· as the Softmax function, then
(28)Ensemblemodel:X⟶ypred∈1,2,⋯,N,GX≜fD∗∑s=1wvs,ypred=fsm∗GX=fsm∗fD∗∑s=1wvs=argmaxl′Py=l′X,Py=l′X=eGl′X∑p′=1NeGp′X,wherew is the number of encoders in the ensemble model, each encoder comes from the corresponding individual model, and the coefficients remain unchanged. fD∗· and fsm∗· have the same structure as the dense module function fD· and Softmax function fsm· in the individual model but with different coefficients. l′,p′∈1,2,⋯,N is the index of the class.


## 3.5. Open-Set Testing

The third stage is open-set testing. The real-world data is inputted into the trained ensemble model, and the ensemble model producesM results Rec to be identified. Compute the maximum of the M results. If the maximum value is greater than or equal to the threshold, the test sample is classified as one of M classes; otherwise, it is classified as unknown.


### 3.5.1. Threshold Estimation

The threshold for judging data classes in open-set testing is determined by outliers. Assume that 1% of the samples in the training set have noise, that is, they are abnormal outlier samples. When calculating the outlier distance for all training samples, sort them from small to large, and the largest 1% distance is the threshold. The detailed process of threshold estimation is shown in Algorithm4.Algorithm 4: Threshold estimation.
Input: training set X1,X2,⋯,XA; known classes N.Output: threshold τ;1:Fori=1,2,⋯,Ado2:Forj=1,2,⋯,Ndo3:Py=jXi=eFjXi/∑p=1keFpXi;4:End for5:outlierXi=maxjPy=jXi;6:End for7:sortoutlier;8: Threshold set = last 1%sortoutlier;9:Returnτ=minthresholdset.After the threshold is determined, the to-be-identified results of the ensemble model will determine the classes of data according to whether it is greater than or equal to the threshold.


## 3.5.1. Threshold Estimation

The threshold for judging data classes in open-set testing is determined by outliers. Assume that 1% of the samples in the training set have noise, that is, they are abnormal outlier samples. When calculating the outlier distance for all training samples, sort them from small to large, and the largest 1% distance is the threshold. The detailed process of threshold estimation is shown in Algorithm4.Algorithm 4: Threshold estimation.
Input: training set X1,X2,⋯,XA; known classes N.Output: threshold τ;1:Fori=1,2,⋯,Ado2:Forj=1,2,⋯,Ndo3:Py=jXi=eFjXi/∑p=1keFpXi;4:End for5:outlierXi=maxjPy=jXi;6:End for7:sortoutlier;8: Threshold set = last 1%sortoutlier;9:Returnτ=minthresholdset.After the threshold is determined, the to-be-identified results of the ensemble model will determine the classes of data according to whether it is greater than or equal to the threshold.


## 4. Experiment and Evaluation

This section mainly introduces the specific experimental content designed for the proposed model OpenCBD, including experimental environment and settings, evaluation metrics, and specific results. And then, the results are discussed to verify the validity of the OpenCBD model.


### 4.1. Experiment Settings

The encrypted traffic data in this paper is selected from the public dataset ISCXVPN2016 [39]. 8 classes of data are selected as known classes, 5 classes of data are selected as unknown classes, and the intersection of known and unknown data is empty. The 13 classes of data include both Virtual Private Network (VPN) traffic that has been encapsulated by the VPN protocol and non-VPN traffic that has not been encapsulated by the VPN. 1000 samples are randomly selected for each class, and each sample contains 10 consecutive data packets. The specific data classes are shown in Table 3.Table 3
Experimental data classes.
Known/unknownClassesKnown classesVPN-Facebook-chatVPN-Facebook-audioVPN-Skype-chatVPN-Skype-fileFacebook-chatFacebook-audioSkype-chatSkype-fileUnknown classesVPN-Hangouts-audioHangouts-audioVPN-Hangouts-chatSkype-audioVPN-sftpDuring the experiment, 1000 samples of each class were randomly divided into the training set and test set, with training set samples : test set samples = 9 : 1; that is, 100 samples of each class were randomly selected as the test set, and the remaining 900 samples were the training set. The training set contains training data and validation data, with training data : validation data = 9 : 1.The experimental environment is a personal desktop, and the specific equipment information is shown in Table4.Table 4
Experimental environment.
HardwareSpecific informationGraphic processing unit (GPU)Nvidia RTX 1060TIMemory48 GBCentral processing unit (CPU)Intel Core i7 6-coreNumber of CPUsNumber of CPU cores: 6Number of CPU threads: 12Operating system (OS)Ubuntu 18.04All codes in the experiment are run in the environment of Python 3.6.5, and the specific hyperparameter settings are shown in Table5.Table 5
Experimental hyperparameters.
Hyperparameter nameHyperparameter valueEpoch80 in individual model, 60 in ensemble modelBatch64Plaintext packets in pretraining256 bytes of plaintextCiphertext packets in pretrainingRandomly selected from ISCXVPN2016, excluding the 13 classes mentioned above, and classes and sizes are not fixedSample numbers of positive and negative sample sets in pretraining5000, 5000k in individual model3 or 4 or 5 or 6 or 7Dimensions of CNN module input vectors256Number of convolution channels in CNN module1Convolution kernel size in CNN module3Convolution stride size in CNN module1Pooling size in CNN module3Dimensions of CNN module output vectors242Dimensions of fully connected layer outputs in the encoder240n in Concat layer15Dimensions of BERT module input matrix15×240m in BERT module8Head numbers of multihead attention in each transformer encoder layer12Dimensions of BERT module output vectors240Dimensions of fully connected layer outputs in classifierk in individual model, N in ensemble modelNumber of encodersw in ensemble model4 or 8 or 12N in ensemble model8M in open-set testingN+1=9The rest of the unmentioned hyperparameters are set according to the default values of the corresponding models in Python.


### 4.2. Evaluation Metrics

When evaluating model performance, the class of interest is usually the positive class and the other classes are the negative classes. Evaluation metrics are usually formulated from the following four basic situations:(1)
True positive (TP): predict the positive class as a positive class(2)
False positive (FP): predict the negative class as a positive class(3)
True negative (TN): predicts the negative class as a negative class(4)
False negative (FN): predicts the positive class as a negative classFrom the above four basic situations, four basic evaluation metrics can be obtained:(1)
The probability of classifying positive samples into positive classes is called TPR, also known as recall or sensitivity:(29)TPR=Recall=TPTP+FN.(2)
The probability of classifying a negative class sample into negative classes is called TNR, also known as specificity:(30)TNR=TNFP+TN.(3)
The probability of misclassifying negative samples into positive classes is called FPR, also known as false recognition rate:(31)FPR=FPFP+TN=1−TNR.(4)
The probability of misclassifying positive samples into negative classes is called FNR, also known as the rejection rate:(32)FNR=FNFN+TP=1−TPR.In addition, there are several other commonly used metrics:(1)
Precision refers to the proportion of true positive samples among all predicted positive samples:(33)Precision=TPTP+FP.(2)
F1-score considers precision and recall comprehensively and refers to the harmonic average of precision and recall:(34)2F1‐score=1Precision+1Recall,F1‐score=2×Precision×RecallPreciaion+Recall.(3)
Accuracy refers to the proportion of all predicted correct samples to the total samples:(35)Accuracy=TP+TNTP+TN+FP+FN=ncorrectn,wherencorrect represents the number of correctly predicted samples and n represents the total number of samples.
(4)
In the multiclassification problem, the F1-score, precision, and recall are all calculated by macro; that is, the index of each class is calculated first, and then, the unweighted average is taken to obtain the final index:(36)macro‐F1‐score=1C∑i=1CF1‐scorei,Macro‐precision=1C∑i=1Cprecisioni,Macro‐recall=1C∑i=1Crecalli,whereC represents the number of classes.The evaluation metrics of this experiment select accuracy, precision, recall, and F1-score of binary classification and multiclassification.


### 4.3. Results and Discussions

According to the detailed experimental setting in Section4.1 and the four evaluation metrics in Section 4.2, we present the specific experimental results in this section. The first is the experimental results of OpenCBD when doing 2-class classification tasks.As can be seen from Figure4 and Table 6, OpenCBD performs best when 7 known classes are randomly selected in the first stage of training. Because there are 8 known classes, each individual model randomly selects 7 classes for training, which enables the individual model to better understand the differences between different known classes and makes the model perform better. On the whole, the increase in the number of encoders helps to improve the performance of the model, but too many encoders will also lead to excessive time overhead of the model, and the performance improvement is no longer obvious. In Figure 4 and Table 6, when 7 known classes are randomly selected and the number of encoders is 12, there is no result. The reason is that 7 classes are randomly selected from the 8 known classes, and there are a total of C87=8 possibilities. If 12 encoders are integrated, there must be repetitions.Figure 4
When the number of encoders in the ensemble model of OpenCBD is 4, 8, and 12, accuracy of the 2-class classification tasks (identifying known and unknown classes) varies as the number of known classes is randomly selected in the first training stage.Table 6
When the number of encoders in the ensemble model of OpenCBD is 4, 8, and 12, accuracy, precision, recall, and F1-score of the 2-class classification tasks (identifying known and unknown classes) vary as the number of known classes is randomly selected in the first training stage.
AccuracyPrecisionRecallF1-scorekw481248124812481230.56070.6030.6130.7140.68530.69510.47750.65620.66120.57220.67040.677740.66150.68530.68690.73370.74590.74840.70620.74120.740.71970.74350.744150.62690.730.72380.72150.7650.7780.64120.810.77120.679020.78680.774660.6930.73070.72070.7750.79520.78410.70620.75750.75370.7390.77590.768670.72380.7484—0.77660.7832—0.77370.8175—0.77520.7999—While OpenCBD can distinguish known from unknown, it can also distinguish known classes.It can be seen from Figure5 and Table 7 that OpenCBD performs the best when 7 known classes are randomly selected in the first stage of training. Accuracy of 9 classes is over 72%, precision is over 76%, and recall and F1-score are over 75%.Figure 5
When the number of encoders in the ensemble model of OpenCBD is 4, 8, and 12, accuracy of the 9-class classification tasks (identifying 8 known classes and 1 unknown class) varies as the number of known classes is randomly selected in the first training stage.Table 7
When the number of encoders in the ensemble model of OpenCBD is 4, 8, and 12, accuracy, precision, recall, and F1-score of the 9-class classification tasks (identifying 8 known classes and 1 unknown class) vary as the number of known classes is randomly selected in the first training stage.
AccuracyPrecisionRecallF1-scorekw481248124812481230.53150.57150.58380.64620.67170.68930.45930.59530.60510.50190.61440.625840.59610.6530.6560.62410.71370.7230.59880.67840.68130.59940.68630.692950.60840.70070.69760.69910.74250.73550.61040.74460.71970.63430.73850.722760.66460.7130.7030.74190.77040.7720.66130.72420.71860.68450.73950.733970.70460.7223—0.76390.7616—0.73150.7597—0.74040.7533—As can be seen from Figure6, the probability of Facebook-audio being wrongly classified into an unknown class is higher than the other 7 classes. Since there are 5 classes of data that are regarded as an unknown class, the number of unknown classes predicted as the unknown class is the largest and the color is the warmest (red).Figure 6
Confusion matrix diagram for OpenCBD doing the 9-class classification tasks. Thex-axis represents the predicted classes of the output, and the y-axis represents the true classes. Warmer colors indicate larger values, and cooler colors indicate smaller values.In addition, we design different random selection methods in the first stage of training. A portion of each of the 8 classes of data is randomly selected, and the results are as follows.As can be seen from Figure7 and Table 8, the results of this training method are not much different from the original results, but the overall results of this method are slightly higher, all above 70%, and the highest accuracy can achieve more than 77%, and precision even exceeds 83%. Since all 8 known classes have data to participate in the training of the individual model in this method, the ensemble model learns more deeply for each known class.Figure 7
When the number of encoders in the ensemble model of OpenCBD is 4, 8, and 12, accuracy of the 2-class classification tasks (identifying known and unknown classes) varies with increasing percentage of known classes randomly selected.Table 8
When the number of encoders in the ensemble model of OpenCBD is 4, 8, and 12, accuracy, precision, recall, and F1-score of the 2-class classification tasks (identifying known and unknown classes) vary with increasing percentage of known classes randomly selected.
AccuracyPrecisionRecallF1-scorekw481248124812481260%0.70460.720.71840.7530.76780.77260.77370.78120.76870.76320.77440.770670%0.70380.74230.7730.7670.80150.83710.7450.77250.78370.75580.78670.809580%0.70460.73230.74230.76190.7860.80470.75630.77620.76750.7590.78110.785690%0.73840.73150.75380.78750.79090.82690.78750.76620.75870.78750.77840.7913As can be seen from Figure8 and Table 9, when OpenCBD is doing 9-class classification tasks, the values of the 4 evaluation metrics increase with the increase of the number of encoders, and the results are better when the percentage of known classes is 70%. The highest accuracy can reach 73.61%, the highest precision is over 80%, the highest recall is nearly 75%, and the highest F1-score is 65.85%.Figure 8
When the number of encoders in the ensemble model of OpenCBD is 4, 8, and 12, accuracy of the 9-class classification tasks (identifying 8 known classes and 1 unknown class) varies with increasing percentage of known classes randomly selected.Table 9
When the number of encoders in the ensemble model of OpenCBD is 4, 8, and 12, accuracy, precision, recall, and F1-score of the 9-class classification tasks (identifying 8 known classes and 1 unknown class) vary with increasing percentage of known classes randomly selected.
AccuracyPrecisionRecallF1-scorekw481248124812481260%0.67460.69530.69760.73420.75440.75320.71040.7280.72420.7140.73630.733470%0.68070.72070.75070.75230.78350.80310.69970.73260.74840.71590.75020.768580%0.68460.71230.72530.74690.77790.78360.71240.73460.73570.72260.7460.753390%0.71920.71460.73610.7710.77980.79390.74550.73170.73170.75190.74720.7526When the individual training selects 70% of the training set and the number of encoders is 12, the metrics of the two classes and nine classes are the highest. Therefore, we choose this ensemble method and compare the area under the ROC curve (receiver operating characteristic curve) of identifying only unknown classes and identifying both known and unknown classes at the same time. The results are shown in Figure9.Figure 9
Area under ROC of OpenCBD when only identifying unknown class samples and both known and unknown class samples.When verifying the performance of the OpenCBD model, we also selected several classic open-set recognition models for comparative experiments. We choose 3 models, namely, the threshold-based Softmax model [15], OpenMax model [15], and II-Loss-based model [19].It can be seen from Figure10 that the values of the four metrics in the 2-class classification of the OpenCBD model are basically around 80%, and the other three are around 70%. Our OpenCBD outperforms the other 3 by around 10% in the 2-class classification tasks. It can be seen from Figure 11 that in the 9-class classification, the metrics of the OpenCBD model are between 75% and 80%, and the other three are basically not more than 70%. Our OpenCBD outperforms the other 3 models by around 5%-10% in 9-class classification tasks. Figure 12 compares the area under ROC of the four models more clearly; the larger the area, the better the effect; and OpenCBD is significantly better than the other three.Figure 10
Performance comparison of four models on 2-class classification tasks.Figure 11
Performance comparison of four models on 9-class classification tasks.Figure 12
Area under ROC of four models.


## 4.1. Experiment Settings

The encrypted traffic data in this paper is selected from the public dataset ISCXVPN2016 [39]. 8 classes of data are selected as known classes, 5 classes of data are selected as unknown classes, and the intersection of known and unknown data is empty. The 13 classes of data include both Virtual Private Network (VPN) traffic that has been encapsulated by the VPN protocol and non-VPN traffic that has not been encapsulated by the VPN. 1000 samples are randomly selected for each class, and each sample contains 10 consecutive data packets. The specific data classes are shown in Table 3.Table 3
Experimental data classes.
Known/unknownClassesKnown classesVPN-Facebook-chatVPN-Facebook-audioVPN-Skype-chatVPN-Skype-fileFacebook-chatFacebook-audioSkype-chatSkype-fileUnknown classesVPN-Hangouts-audioHangouts-audioVPN-Hangouts-chatSkype-audioVPN-sftpDuring the experiment, 1000 samples of each class were randomly divided into the training set and test set, with training set samples : test set samples = 9 : 1; that is, 100 samples of each class were randomly selected as the test set, and the remaining 900 samples were the training set. The training set contains training data and validation data, with training data : validation data = 9 : 1.The experimental environment is a personal desktop, and the specific equipment information is shown in Table4.Table 4
Experimental environment.
HardwareSpecific informationGraphic processing unit (GPU)Nvidia RTX 1060TIMemory48 GBCentral processing unit (CPU)Intel Core i7 6-coreNumber of CPUsNumber of CPU cores: 6Number of CPU threads: 12Operating system (OS)Ubuntu 18.04All codes in the experiment are run in the environment of Python 3.6.5, and the specific hyperparameter settings are shown in Table5.Table 5
Experimental hyperparameters.
Hyperparameter nameHyperparameter valueEpoch80 in individual model, 60 in ensemble modelBatch64Plaintext packets in pretraining256 bytes of plaintextCiphertext packets in pretrainingRandomly selected from ISCXVPN2016, excluding the 13 classes mentioned above, and classes and sizes are not fixedSample numbers of positive and negative sample sets in pretraining5000, 5000k in individual model3 or 4 or 5 or 6 or 7Dimensions of CNN module input vectors256Number of convolution channels in CNN module1Convolution kernel size in CNN module3Convolution stride size in CNN module1Pooling size in CNN module3Dimensions of CNN module output vectors242Dimensions of fully connected layer outputs in the encoder240n in Concat layer15Dimensions of BERT module input matrix15×240m in BERT module8Head numbers of multihead attention in each transformer encoder layer12Dimensions of BERT module output vectors240Dimensions of fully connected layer outputs in classifierk in individual model, N in ensemble modelNumber of encodersw in ensemble model4 or 8 or 12N in ensemble model8M in open-set testingN+1=9The rest of the unmentioned hyperparameters are set according to the default values of the corresponding models in Python.


## 4.2. Evaluation Metrics

When evaluating model performance, the class of interest is usually the positive class and the other classes are the negative classes. Evaluation metrics are usually formulated from the following four basic situations:(1)
True positive (TP): predict the positive class as a positive class(2)
False positive (FP): predict the negative class as a positive class(3)
True negative (TN): predicts the negative class as a negative class(4)
False negative (FN): predicts the positive class as a negative classFrom the above four basic situations, four basic evaluation metrics can be obtained:(1)
The probability of classifying positive samples into positive classes is called TPR, also known as recall or sensitivity:(29)TPR=Recall=TPTP+FN.(2)
The probability of classifying a negative class sample into negative classes is called TNR, also known as specificity:(30)TNR=TNFP+TN.(3)
The probability of misclassifying negative samples into positive classes is called FPR, also known as false recognition rate:(31)FPR=FPFP+TN=1−TNR.(4)
The probability of misclassifying positive samples into negative classes is called FNR, also known as the rejection rate:(32)FNR=FNFN+TP=1−TPR.In addition, there are several other commonly used metrics:(1)
Precision refers to the proportion of true positive samples among all predicted positive samples:(33)Precision=TPTP+FP.(2)
F1-score considers precision and recall comprehensively and refers to the harmonic average of precision and recall:(34)2F1‐score=1Precision+1Recall,F1‐score=2×Precision×RecallPreciaion+Recall.(3)
Accuracy refers to the proportion of all predicted correct samples to the total samples:(35)Accuracy=TP+TNTP+TN+FP+FN=ncorrectn,wherencorrect represents the number of correctly predicted samples and n represents the total number of samples.
(4)
In the multiclassification problem, the F1-score, precision, and recall are all calculated by macro; that is, the index of each class is calculated first, and then, the unweighted average is taken to obtain the final index:(36)macro‐F1‐score=1C∑i=1CF1‐scorei,Macro‐precision=1C∑i=1Cprecisioni,Macro‐recall=1C∑i=1Crecalli,whereC represents the number of classes.The evaluation metrics of this experiment select accuracy, precision, recall, and F1-score of binary classification and multiclassification.


## 4.3. Results and Discussions

According to the detailed experimental setting in Section4.1 and the four evaluation metrics in Section 4.2, we present the specific experimental results in this section. The first is the experimental results of OpenCBD when doing 2-class classification tasks.As can be seen from Figure4 and Table 6, OpenCBD performs best when 7 known classes are randomly selected in the first stage of training. Because there are 8 known classes, each individual model randomly selects 7 classes for training, which enables the individual model to better understand the differences between different known classes and makes the model perform better. On the whole, the increase in the number of encoders helps to improve the performance of the model, but too many encoders will also lead to excessive time overhead of the model, and the performance improvement is no longer obvious. In Figure 4 and Table 6, when 7 known classes are randomly selected and the number of encoders is 12, there is no result. The reason is that 7 classes are randomly selected from the 8 known classes, and there are a total of C87=8 possibilities. If 12 encoders are integrated, there must be repetitions.Figure 4
When the number of encoders in the ensemble model of OpenCBD is 4, 8, and 12, accuracy of the 2-class classification tasks (identifying known and unknown classes) varies as the number of known classes is randomly selected in the first training stage.Table 6
When the number of encoders in the ensemble model of OpenCBD is 4, 8, and 12, accuracy, precision, recall, and F1-score of the 2-class classification tasks (identifying known and unknown classes) vary as the number of known classes is randomly selected in the first training stage.
AccuracyPrecisionRecallF1-scorekw481248124812481230.56070.6030.6130.7140.68530.69510.47750.65620.66120.57220.67040.677740.66150.68530.68690.73370.74590.74840.70620.74120.740.71970.74350.744150.62690.730.72380.72150.7650.7780.64120.810.77120.679020.78680.774660.6930.73070.72070.7750.79520.78410.70620.75750.75370.7390.77590.768670.72380.7484—0.77660.7832—0.77370.8175—0.77520.7999—While OpenCBD can distinguish known from unknown, it can also distinguish known classes.It can be seen from Figure5 and Table 7 that OpenCBD performs the best when 7 known classes are randomly selected in the first stage of training. Accuracy of 9 classes is over 72%, precision is over 76%, and recall and F1-score are over 75%.Figure 5
When the number of encoders in the ensemble model of OpenCBD is 4, 8, and 12, accuracy of the 9-class classification tasks (identifying 8 known classes and 1 unknown class) varies as the number of known classes is randomly selected in the first training stage.Table 7
When the number of encoders in the ensemble model of OpenCBD is 4, 8, and 12, accuracy, precision, recall, and F1-score of the 9-class classification tasks (identifying 8 known classes and 1 unknown class) vary as the number of known classes is randomly selected in the first training stage.
AccuracyPrecisionRecallF1-scorekw481248124812481230.53150.57150.58380.64620.67170.68930.45930.59530.60510.50190.61440.625840.59610.6530.6560.62410.71370.7230.59880.67840.68130.59940.68630.692950.60840.70070.69760.69910.74250.73550.61040.74460.71970.63430.73850.722760.66460.7130.7030.74190.77040.7720.66130.72420.71860.68450.73950.733970.70460.7223—0.76390.7616—0.73150.7597—0.74040.7533—As can be seen from Figure6, the probability of Facebook-audio being wrongly classified into an unknown class is higher than the other 7 classes. Since there are 5 classes of data that are regarded as an unknown class, the number of unknown classes predicted as the unknown class is the largest and the color is the warmest (red).Figure 6
Confusion matrix diagram for OpenCBD doing the 9-class classification tasks. Thex-axis represents the predicted classes of the output, and the y-axis represents the true classes. Warmer colors indicate larger values, and cooler colors indicate smaller values.In addition, we design different random selection methods in the first stage of training. A portion of each of the 8 classes of data is randomly selected, and the results are as follows.As can be seen from Figure7 and Table 8, the results of this training method are not much different from the original results, but the overall results of this method are slightly higher, all above 70%, and the highest accuracy can achieve more than 77%, and precision even exceeds 83%. Since all 8 known classes have data to participate in the training of the individual model in this method, the ensemble model learns more deeply for each known class.Figure 7
When the number of encoders in the ensemble model of OpenCBD is 4, 8, and 12, accuracy of the 2-class classification tasks (identifying known and unknown classes) varies with increasing percentage of known classes randomly selected.Table 8
When the number of encoders in the ensemble model of OpenCBD is 4, 8, and 12, accuracy, precision, recall, and F1-score of the 2-class classification tasks (identifying known and unknown classes) vary with increasing percentage of known classes randomly selected.
AccuracyPrecisionRecallF1-scorekw481248124812481260%0.70460.720.71840.7530.76780.77260.77370.78120.76870.76320.77440.770670%0.70380.74230.7730.7670.80150.83710.7450.77250.78370.75580.78670.809580%0.70460.73230.74230.76190.7860.80470.75630.77620.76750.7590.78110.785690%0.73840.73150.75380.78750.79090.82690.78750.76620.75870.78750.77840.7913As can be seen from Figure8 and Table 9, when OpenCBD is doing 9-class classification tasks, the values of the 4 evaluation metrics increase with the increase of the number of encoders, and the results are better when the percentage of known classes is 70%. The highest accuracy can reach 73.61%, the highest precision is over 80%, the highest recall is nearly 75%, and the highest F1-score is 65.85%.Figure 8
When the number of encoders in the ensemble model of OpenCBD is 4, 8, and 12, accuracy of the 9-class classification tasks (identifying 8 known classes and 1 unknown class) varies with increasing percentage of known classes randomly selected.Table 9
When the number of encoders in the ensemble model of OpenCBD is 4, 8, and 12, accuracy, precision, recall, and F1-score of the 9-class classification tasks (identifying 8 known classes and 1 unknown class) vary with increasing percentage of known classes randomly selected.
AccuracyPrecisionRecallF1-scorekw481248124812481260%0.67460.69530.69760.73420.75440.75320.71040.7280.72420.7140.73630.733470%0.68070.72070.75070.75230.78350.80310.69970.73260.74840.71590.75020.768580%0.68460.71230.72530.74690.77790.78360.71240.73460.73570.72260.7460.753390%0.71920.71460.73610.7710.77980.79390.74550.73170.73170.75190.74720.7526When the individual training selects 70% of the training set and the number of encoders is 12, the metrics of the two classes and nine classes are the highest. Therefore, we choose this ensemble method and compare the area under the ROC curve (receiver operating characteristic curve) of identifying only unknown classes and identifying both known and unknown classes at the same time. The results are shown in Figure9.Figure 9
Area under ROC of OpenCBD when only identifying unknown class samples and both known and unknown class samples.When verifying the performance of the OpenCBD model, we also selected several classic open-set recognition models for comparative experiments. We choose 3 models, namely, the threshold-based Softmax model [15], OpenMax model [15], and II-Loss-based model [19].It can be seen from Figure10 that the values of the four metrics in the 2-class classification of the OpenCBD model are basically around 80%, and the other three are around 70%. Our OpenCBD outperforms the other 3 by around 10% in the 2-class classification tasks. It can be seen from Figure 11 that in the 9-class classification, the metrics of the OpenCBD model are between 75% and 80%, and the other three are basically not more than 70%. Our OpenCBD outperforms the other 3 models by around 5%-10% in 9-class classification tasks. Figure 12 compares the area under ROC of the four models more clearly; the larger the area, the better the effect; and OpenCBD is significantly better than the other three.Figure 10
Performance comparison of four models on 2-class classification tasks.Figure 11
Performance comparison of four models on 9-class classification tasks.Figure 12
Area under ROC of four models.


## 5. Conclusion

In this paper, we proposed a novel model that can simultaneously identify unknown traffic and classify known traffic. The model could be trained on the known traffic of the closed set and tested on the network traffic of the open set. The model first combined the convolutional neural network and the transformer encoder to construct a deep learning-based model. Then, use the general pretraining method in the field of encrypted traffic analysis to pretrain from unlabeled traffic data, so that the model could learn the basic characteristics of encrypted traffic. Then, according to the characteristics of the open set, a three-stage training and testing process was designed. During training, choose the cross-entropy loss function suitable for classification and the II-Loss function suitable for clustering. At the same time, using the idea of ensemble learning, a traffic identification model OpenCBD based on open-set recognition was constructed. For real-world traffic data, if it belonged to a known class, the class to which it belonged can be identified, and if it belonged to an unknown class, it can be identified to belong to an unknown class. Experiments were carried out in public datasets, 8 classes of data were selected as known classes and 5 classes of data were selected as unknown classes, and a class-balanced dataset was constructed in the experiment to eliminate possible human influence to the greatest extent.In the future work, we can consider the clustering of unknown classes of encrypted traffic and further separate unknown traffic according to its characteristics, which is convenient for further discovery and research of new classes of traffic.



---

*Source: 1746373-2022-05-12.xml*