# Improvement of Learning Stability of Generative Adversarial Network Using Variational Learning
## 2 Preliminary Works  
### Generative Adversarial Networks  
GAN consists of two types of artificial neural network models, that is, the discriminator and the generator. The generator learns to be able to create fake data to the degree that the discriminator can regard the generated data as real data, and the discriminator learns to be able to discriminate between real data and fake data that are generated by the generator. In this way, the generator and the discriminator achieve their objective by competing with each other based on opposing goals. This competition structure can be expressed as a minimax game, as follows:  
\[\min_{G}\max_{D}V(D,G)=E_{x\sim P_{data}}[\log{(P_{\psi}(y|x))}]+E_{z\sim P(z)} [\log{(1-P_{\psi}(y|P_{\theta}(x|z)))}]. \tag{1}\]  
Herein, \(G\) is the generator and \(D\) is the discriminator. The generator is a neural network model \(P_{\theta}(x|z)\) that generates fake data \(x_{fake}\) with a latent variable \(z\) as the input sampled from distribution \(P(z)\), which was assumed as the prior. Additionally, \(\theta\) is a parameter of the generator network. The discriminator is a neural network model which serves as a classifier that discriminates between real data \(x_{real}\sim P_{data}\) and fake data \(x_{fake}\sim P_{\theta}(x|z)\) generated from the generator, and it can be expressed as \(P_{\psi}(y|x)\). In this instance, \(\psi\) is a parameter of the discriminator's network. Additionally, \(y\) is the output of the discriminator and outputs zero if the discriminator regards the sample as fake data, or outputs unity values if the discriminator regards the sample as real data. The minimax game of Equation (1) is a loss for the discriminator and a loss for the generator and can be decomposed, as follows.  
\[E_{x\sim P_{data}}[-log{(P_{\psi}(y|x))}]+E_{z\sim P(z)}[-log{(1-P_{\psi}(y|P_ {\theta}(x|z)))}] \tag{2}\]  
\[E_{z\sim P(z)}[log{(1-P_{\psi}(y|P_{\theta}(x|z)))}] \tag{3}\]By using Equation (2), the discriminator \(P_{\theta}(y|x)\) can update \(\psi\) to minimize the binary cross entropy for data \(x_{real}\sim P_{data}\) and \(x_{fake}\sim P_{\theta}(x|z)\). By using Equation (3), the generator \(P_{\theta}(x|z)\) can update \(\theta\) to increase the probability that the discriminator regards fake data \(x_{fake}\sim P_{\theta}(x|z)\) as real data, that is, to generate fake data that are very similar to the actual data.  
As the learning progresses, if \(P_{\theta}(x|z)\) becomes equal to \(P_{data}\) and reaches the optimum, then the discriminator can no longer discriminate between real data and fake data that are generated by the generator. At this point, the minimax game of Equation (1) reaches the Nash equilibrium and learning is complete. If the discriminator is always optimum for the generator during the learning, that is, if the discriminator can always perfectly discriminate between real and fake data, then the result of the minimax game is equivalent to the minimization of the Jensen Shannon divergence (JSD) [34] of \(P_{data}\) and \(P_{\theta}(x|z)\). Correspondingly, there is always a unique solution to this [1]. However, in the actual learning of GAN, it is practically difficult to find the optimum discriminator for the generator at every learning step. Therefore, learning should be done by alternating the generator and the discriminator.