# A VPN-Encrypted Traffic Identification Method Based on Ensemble Learning
## 2 Methodology
### VPN-Encrypted Traffic FL-XGB Identification Model
#### 2.2.1 Overview of the Basic Xgboost Model  
Xgboost model has high operational efficiency and prediction accuracy compared with other methods in the field of machine learning and data mining. Xgboost is an improved gradient boosting learning (GDBT) framework, which is a boosting method. Xgboost adds regularizations such as leaf node weights and depth of the tree to the cost function. The regularization term controls the complexity of the model and prevents overfitting. Xgboost has the advantages of being fast, effective, capable of handling large-scale data, and supporting custom loss functions. Classification and regression tree (CART) is a widely used decision tree learning method [35]. Xgboost model consists of CART trees, and its parameters are present in each CART. The determination of the tree structure is to select the optimal split node with the information gain rate as the split criterion. The greedy algorithm is used to enumerate all nodes, the information gain of each node before and after the split is calculated, and the node with the greatest information gain is selected. Essentially it is to do two loops. The first loop is implemented for each feature's split point, and then the gain is calculated, whereas the best split point for the feature is selected. Among them, the split gain uses the difference in the change of the objective function after the split. The second loop object is to select the feature with the highest gain for all features.  
The idea of using iterative operations to set some weak learners into a strong learner to achieve accurate identification. The results of the Xgboost model after t iterations is shown in Equation (5).  
\[\hat{y}_{i}^{(t)}=\hat{y}_{i}^{(t-1)}+f_{t}(x_{i}) \tag{5}\]  
In Equation (5), \(\hat{y}_{i}^{(t)}\) is the identification result of sample \(i\) after the \(t\)th iteration, \(\hat{y}_{i}^{(t-1)}\) is the identification result of sample \(i\) after the \((t-1)\)th iteration, and \(f_{t}(x_{i})\) is the model identification result of the tth tree. At this point, the objective function \(\lambda^{(t)}\) of Xgboost is shown in Equation (6).  
\[\lambda^{(t)}=\sum_{i=1}^{n}l\left(y_{i},\hat{y}_{i}^{(t-1)}+f_{t}(x_{i})\right) +\Omega(f_{t}) \tag{6}\]  
In Equation (6), \(y_{i}\) is the true value of the ith sample, \(\Omega(f_{t})\) is the regularization term of the function, and \(\sum_{i=1}^{n}l\left(y_{i},\hat{y}_{i}^{(t-1)}+f_{t}(x_{i})\right)\) is the loss function of the model. \(\hat{y}_{i}^{(t-1)}\) is known, the model has to learn only the tth tree \(f_{t}\), and then the error function is subjected to a second-order [PERSON] expansion at \(\hat{y}_{i}^{(t-1)}\) as shown in Equation (7).  
\[\lambda^{(t)}\cong\sum_{i=1}^{n}\left[l\left(y_{i},\hat{y}_{i}^{(t-1)}\right) +g_{i}f_{t}(x_{i})+\frac{1}{2}h_{i}f^{2}(x_{i})\right]+\Omega(f_{t}). \tag{7}\]  
In Equation (7), \(g_{i}=\partial_{y^{(t-1)}}l(y_{i},\hat{y}_{i}^{(t-1)})\) is the first derivative of the loss function, and \(h_{i}=\partial_{y^{(t-1)}}^{2}l(y_{i},\hat{y}_{i}^{(t-1)})\) is the second derivative of the loss function. Then, after removing the constant term from the formula, the result is obtained as shown in Equation (8).  
\[\widetilde{\lambda}^{(t)}=\sum_{i=1}^{n}\left[g_{i}f_{t}(x_{i})+\frac{1}{2}h_ {i}f_{t}{}^{2}(x_{i})\right]+\Omega(f_{t}) \tag{8}\]  
Xgboost ensemble learning model, when dealing with the problem of data imbalance, usually requires the use of operations such as active acquisition or sampling to balance the data. However, the operation changes the distribution of the original dataset. In this paper, we take advantage of Xgboost's support for custom loss functions and modify the loss function based on the cross-entropy loss function. On this basis, we solve the problem of VPN-encrypted traffic data imbalance without changing the distribution of the original dataset.