# A VPN-Encrypted Traffic Identification Method Based on Ensemble Learning
## 2 Methodology
### Time-Related mRMR Feature Selection  
The mRMR-based feature selection method for VPN-encrypted traffic belongs to the class of filtered feature selection methods. The method uses many trade-offs between relevance and redundancy. Additionally, it uses mutual information as the calculation criterion to measure the redundancy between features and the relevance between features and classes. Thus, the relevance between features and classes is maximized and the redundancy between features and features is minimized and achieves feature selection. The maximum correlation means that the features with the highest correlation to the model are selected, and the higher the correlation is, the better the encrypted traffic can be identified. At the same time, the method can simplify the model and shorten the training time of the model. The mRMR feature selection method uses mutual information as a measure of the correlation between features and features as well as between features and classes. In the obtained feature set, there are large differences between features and correlations with the target variables. Using the mRMR algorithm for feature selection, a streamlined subset of features with good identification can be obtained, and the optimal number of features can also be determined. The mRMR feature selection method has the framework shown in Figure 2.  
The mRMR algorithm is a filtered feature selection method. The method weighs relevance and redundancy differently. Additionally, uses mutual information as a computational criterion to measure the redundancy between features and the relevance between features and class variables. Performs feature selection by maximizing the relevance of features to class variables and minimizing the redundancy between features. The maximum relevance principle refers to the selection of features that have the greatest relevance to the mode. The greater the relevance, the greater the problem-solving capability of the trained model. The specific steps of the mRMR algorithm for feature selection of VPN-encrypted traffic are as follows. First, the feature set \(S\) is initialized to be empty, and the input training dataset contains the feature set \(F\) and the class set \(C\). For data containing N-dimensional features, the sorted feature list is obtained by a cyclic process of N iterations. In each iteration, one feature is selected. This feature has the maximum relevance to the target variable compared to other features while having the minimum redundancy with other features that have been selected. Figure 1: FL_XGB-VPN-encrypted traffic identification model general framework. Mutual information [34] is a measure of information in information theory. It reflects the amount of information contained in a random variable with respect to other variables \(Y\). Assuming that the marginal probability density function of variable \(X\) is \(P(x)\), the marginal probability density function of variable \(Y\) is \(P(y)\), and their joint probability density function is calculated to be \(P(x,y)\), the value of mutual information is calculated as shown in Equation (1). \[I(x,y)=\sum_{x\in X}\sum_{y\in Y}P(x,y)log\frac{P(x,y)}{P(x)P(y)} \tag{1}\]  
Assume that \(F\) is the set containing N-dimensional features and \(S\) is the set containing the selected features. For each feature \(F_{i}\) in the feature set to be selected, the correlation between the feature set \(F\) composed of all features and the sample class set \(C\) is calculated. The correlation between the feature \(F_{i}\) to be selected and the class set \(C\) is \(I(F_{i},c)\). \(D\) indicates the relevance of the feature to be selected to the class, and the larger the \(D\) the higher the relevance of the feature to the class. The correlation \(D\) is calculated as shown in Equation (2).