# A VPN-Encrypted Traffic Identification Method Based on Ensemble Learning
## 2 Methodology
### Optimization of FL-XGB-VPN-Encrypted Traffic Identification Model  
The problem of low identification rate still exists when using the FL-XGB model for VPN-encrypted traffic. The accuracy of the FL-XGB identification model for VPN-encrypted traffic is affected by parameters, and parameter optimization is needed in combination with parameter search strategies. Bayesian optimization is used to update the posterior distribution of the objective function by continuously adding sample points. The Bayesian optimization algorithm is based on [PERSON] theorem, which is expressed as shown in Equation (13). \[p(f|D_{1:t})=\frac{p(D_{1:t}|f)p(f)}{p(D_{1:t})} \tag{13}\]  
In Equation (13), \(f\) is the unknown objective function, \(D_{1:t}=\{(x_{1},y_{1}),(x_{2},y_{2}),\ldots\,(x_{t},y_{t})\}\) is the set of evaluated points, \(x_{t}\) is the decision vector, \(y_{t}=f(x_{t})+\varepsilon\) is the observed value, \(\varepsilon\)is the observed error, \(p(D_{1:t}|f)\) is the likelihood distribution of \(y\), \(p(D_{1:t})\) is the marginalized \(f\) marginal likelihood distribution, \(p(f)\) is the prior probability of \(f\), and \(p(f|D_{1:t})\) is the posterior probability of \(f\). After correcting the prior probability by the set of evaluated points, the posterior probability distribution is the confidence level of the parameters in the unknown objective function or parametric model. The Bayesian optimization algorithm uses a probabilistic agent model to fit the true objective function, and selects the next evaluation point based on the acquisition function. The commonly used probabilistic agent models include the Beta-Bernoulli model, linear model, Gaussian process, random forest, etc. Among them, Gaussian process is highly flexible, scalable and analyzable, it is the most widely used probabilistic agent model in Bayesian optimization. The Gaussian process is a paradigmization of the multivariate Gaussian probability distribution, consisting of a mean function and a semi-positive definite covariance function, as shown in Equation (14). \[y\sim gp\big{(}\mu_{t}(x),k(x,x^{\prime})\big{)} \tag{14}\]  
In Equation (14), \(\mu_{t}(x)\) is the mean function and \(k(x,x^{\prime})\) is the covariance function. When fitting discrete data \((x_{t},y_{t})\) using a Gaussian process, \(\mu_{t}(x)\) is usually set to 0 and \(k(x,x^{\prime})\) is usually used as a Matern covariance function, as shown in Equation (15). \[k\big{(}x,x^{\prime}\big{)}=\sigma_{f}{}^{2}[1+\sqrt{5}\frac{r}{\sigma_{l}}+ \frac{5}{3}\bigg{(}\frac{r}{\sigma_{l}}\bigg{)}^{2}]e^{-\sqrt{5}\frac{r}{ \sigma_{l}}} \tag{15}\]  
In Equation (15), \(r\) is the Euler distance of \(x\) and \(x^{\prime}\), \(\sigma_{f}\) is the characteristic deviation, \(\sigma_{l}\) is the characteristic length. \(\sigma_{f}\) and \(\sigma_{l}\) will change automatically as the Gaussian process is fitted, the initial value of \(\sigma_{l}\) is the standard deviation of \(x_{i}\), and the initial value of \(\sigma_{f}\) is the standard deviation of \(y_{i}\) divided by \(\sqrt{2}\). The Bayesian optimization method is an acquisition function based on a strategy of lifting probability and lifting amount, as shown in Equation (16). \[\alpha_{t}(x;D_{1:t})= \begin{cases}(v^{*}-\mu_{t}(x))\varnothing\Big{(}\frac{ v^{*}-\mu_{t}(x)}{\sigma_{t}(x)}\Big{)}+\sigma_{l}(x)\varnothing\Big{(}\frac{v^{*}- \mu_{t}(x)}{\sigma_{t}(x)}\Big{)},&\sigma_{l}(x)>0\\ 0,&\sigma_{l}(x)=0\end{cases}  \tag{16}\]  
In Equation (16), \(\alpha_{t}(x;D_{1:t})\) is the acquisition function, \(v^{*}\) is the current optimal function value, \(\varnothing(x)\) is the standard normal distribution cumulative density function. \(\mu_{t}(x)\) and \(\sigma_{l}(x)\) are the mean and variance, respectively. In this paper, the acquisition function is selected based on the confidence interval strategy, and comparison of the maximum of the confidence interval. The location of the next extreme value point of the confidence interval is shown in Equation (17).