## 2 Methods
### Generative Adversarial Networks
#### 2.1.2 The Principle of Generative Adversarial Networks (GANs)  
Generative models can be broadly categorized into two types: those that represent the exact distribution function of the data and those that involve a fuzzy distribution function for generating new data. The GAN belongs to the latter category. The GAN, an unsupervised learning model, consists of a discriminator and a generator and is employed for generating images, audio, video, or text. Both the discriminator and the generator can be constructed using any neural network architecture, such as Artificial Neural Networks (ANNs), Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Long Short-Term Memory Networks (LSTMs), etc. GAN directly models the probability distribution of data samples implicitly through neural networks. The generator takes a random vector from a distribution, often a normal or uniform distribution denoted as \(P(z)\), and transforms it into synthetic data \(G(z)\). The discriminator essentially functions as a binary classifier. Given a real image \(x\), it outputs the probability that the input data are real by comparing them with the data generated by the generator, \(G(z)\), where 0 represents fake and 1 represents true. The training process of GANs, as illustrated in Figure 1, involves the generator learning to produce a distribution similar to the real data distribution, while the discriminator learns to differentiate between inputs. The generator and discriminator engage in continuous adversarial training until the model reaches Nash equilibrium. At this equilibrium, the generator produces data that closely resemble the target distribution, and the discriminator cannot distinguish between real and generated data, assigning both probabilities of 1/2. In other words, the discriminator is unable to discern the difference between real and generated images, and both the generator and discriminator achieve optimal performance. The training process of GANs involves a minimax game between the generator \(G\) and the discriminator \(D\). The generator \(G\) takes a random vector \(z\) from a simple distribution \(P_{z}\) and generates synthetic data \(G(z)\). The discriminator \(D\) evaluates the probability that the input data are real. The objective function is defined as:  
\[\min_{G}max_{D}\big{\{}V(D,G)=E_{x-P_{data}}[\log D(x)]+E_{z-P_{z}}[\log(1-D(G(z )))]\big{\}}. \tag{1}\]  
Here, \(P_{data}\) represents the distribution of real data, and \(z\) is the input random vector to the generator. The first term represents the expected log probability that the discriminator correctly identifies real data, and the second term represents the expected log probability that the discriminator correctly identifies generated data as fake. The generator aims to minimize this objective function, while the discriminator aims to maximize it. KS (KL divergence), also known as Kullback-Leibler divergence, is a critical concept in statistics used to measure the proximity between two probability distributions. The larger the KS divergence, the greater the difference between the probability distributions; conversely, smaller KS divergence indicates closer proximity between distributions. In GAN, KS divergence is effective in determining whether the generator and discriminator have reached Nash equilibrium. Assuming two continuous distributions \(P\) and \(Q\), their KL divergence is defined as follows:  
\[D_{KZ}(P||Q)=\int_{-\infty}^{+\infty}P(x)\log\frac{P(x)}{Q(x)}d(x). \tag{2}\]  
Figure 1: Schematic diagram of GAN structure. The objective of the generator is to minimize the gap between the distribution \(P_{data}(x)\) and the distribution \(P_{G}(x;\ \theta)\) generated by feeding the random vector z into the generator, eventually approaching the same distribution. The generation principle is illustrated in Figure 2. Here, \(\theta\) is determined by the network parameters, and the ultimate task of the network is to find \(\theta\) to make \(P_{data}(x)\) close to \(P_{G}(x;\ \theta)\).