## 3 Data Enhancement Methods
### WGAN-GP Data Enhancement  
Another data enhancement method used in this study was _WGAN-GP_. _WGAN-GP_ is a variant of a generative adversarial network (_GAN_), an improved version of _WGAN_. In _GAN_, the core objective is to make the generated data distribution close to the real data distribution, that is, to make the generated data more like the real data. To compare the real data distribution with the generated data distribution, traditional generative adversarial networks use a metric called Jensen-Shannon divergence (_JS_ divergence) [15], which is based on Kullback-Leibler divergence (_KL_ divergence) [16], as shown in Equation (1), where \(P_{r}(x)\) represents the real data distribution and \(P_{g}(x)\) represents the generated data distribution. The KL divergence calculates the asymmetric distance between the two probability distributions \(P_{r}(x)\) and \(P_{g}(x)\). In contrast, the _JS_ divergence mainly solves the asymmetric problem of the _KL_ divergence, as shown in Equation (2). \[KL(P_{r}||P_{g})=\int\log{(\frac{P_{r}(x)}{P_{g}(x)})P_{r}(x)du(x)} \tag{1}\]  
\[JS(P_{r}||P_{g})=\frac{1}{2}KL(P_{r}||\frac{P_{r}+P_{g}}{2})+\frac{1}{2}KL(P_{ g}||\frac{P_{r}+P_{g}}{2}) \tag{2}\]  
However, there is a drawback to this measurement method. When there is no overlap between the real data distribution \(P_{r}(x)\) and the generated data distribution \(P_{g}(x)\), or when the overlap is too small, this phenomenon will cause a gradient of zero or a very small value when the generated adversarial network updates the weights. That is, the weights cannot be updated. This problem occurs when the discriminator is considerably more robust than the generator. For this purpose, _WGAN_ proposes Wasserstein's distance, also known as the EM (earth mover) distance, to improve the data distribution measure of the _JS_ divergence in traditional generative adversarial networks (_GAN_s). The Wasserstein distance formula is shown in Equation (3), where the \(E\) function denotes the expectation value function. \[W(P_{r},P_{g})=inf_{\gamma\sim\prod{(P,P_{g})}}E_{(x,y)\sim\gamma}[||x-y||] \tag{3}\]  
The advantage of the Wasserstein distance over the \(JS\) divergence is that it can still reflect the proximity of the two data distributions \(P_{r}(x)\) and \(P_{g}(x)\) when the two distributions do not overlap. The objective of the _WGAN_ discriminator is to maximize the EM distance between the real data and the generated data, while the objective of the generator is to minimize the EM distance between the real data and the generated data. In addition, _WGAN_ uses a weight clipping method to constrain the gradient of the weight updating when the weight is updated. This is used to limit the weights of the discriminator to conform to the 1-Lipschitz continuity [17], that is, Equation (4), where \(K\) denotes a constant. It is conceptualized as imposing a restriction on the continuous function \(f\) by requiring that the elements \(x_{1}\) and \(x_{2}\) in the domain of definition satisfy this condition when \(K\geq 0\). \[|f(x_{1})-f(x_{2})|\leq K|x_{1}-x_{2}| \tag{4}\]  
However, the weight clipping method may lead to the problem of gradient explosion or gradient disappearance. Therefore, _WGAN-GP_[13] uses the gradient penalty method to improve these problems caused by the weight clipping method. Equation (5) shows the loss function \(L_{WGAN}\) of the original _WGAN_. Equation (6) illustrates the weight penalty term \(GP\) introduced by the _WGAN-GP_. Equation (7) expresses the main improvement of the _WGAN-GP_; that is, a new penalty term \(GP\) is added to the original loss function of _WGAN_ to replace the original weight clipping method used by the _WGAN_. This leads to the derivation of the new loss function \(L_{WGAN-GP}\) for _WGAN-GP_ in Equation (7). It is found that the use of the gradient penalty makes the network model training more stable and easier to converge. Therefore, our study used _WGAN-GP_ to augment the sparse attack classes in the NSL-KDD dataset.