# Survey on Implementations of Generative Adversarial Networks for Semi-Supervised Learning
## 3 Literature Review of GANS for SSL
### Extensions Using Pseudo-Labeling and Classifiers  
GANs were introduced by [PERSON] et al. [49] as an architecture involving a generator and a discriminator competing against each other, with the generator generating fake images, and the discriminator identifying them as fake. As [PERSON] et al. [1] notes, GANs are good candidates for SSL because the generator is trained on unlabeled images and the discriminator's primary function is to assess the quality of the generator. While the original implementations focused on using the GAN framework for image generation, it was not long before CatGAN [30] was proposed in 2015 that added an unsupervised classifier to the proposed model in order to enable categorical classification using a cross-entropy loss. This paper also added a cross-entropy loss term for the labeled samples that penalized misclassifications of real data. This approach was also used in SGAN [31], that leveraged a single discriminator/classifier network by having N + 1 classifying neurons, where N is the number of classes and one neuron is added to identify fake samples. Improved GAN [32] introduced feature matching, which involved training the generator to produce images that match the expected value of features at an intermediate layer of the discriminator instead of for the final layer. This approach prevented the generator from overtraining to the specific discriminator. Mini-batch discrimination was also proposed where the discriminator predicted whether a mini-batch of images were real or fake instead of individually evaluating single images. This helped in making the generator produce more varied samples since the generator traced to the one point that the discriminator believed was realistic. Mini-batch discrimination generated better images. However, feature matching worked much better for the SSL component. In addition to the proposed techniques, the authors also argued that training GANs using gradient descent techniques was counterintuitive as they were designed to minimize the cost function instead of finding the Nash equilibrium. This argument is an important precursor to subsequent work that tried to reach a balance between generators and discriminators. For example, GoodBadGAN [29] was based on the premise that obtaining good classifier performance, and an effective generator at the same time was difficult, and therefore the focus should be on achieving one outcome only. They based their argument on [PERSON] et al. [32] and noted that while mini-batch discrimination produced better images, it was feature matching that showed an improved performance for SSL. They also questioned training the discriminator and generator jointly, and demonstrated that a good discriminator could be produced by using a bad generator. This was first carried out by increasing the generator entropy by adding \begin{table}
\begin{tabular}{c c} \hline \hline
**Term** & **Definition** \\ \hline x & Original labeled data points \\ \hline y & Original labels \\ \hline x\({}_{\text{u}}\) & Unlabeled data points \\ \hline y\({}^{\prime}\) & Labels of generated data \\ \hline z & Randomly generated latent space \\ \hline G(z) & Generator \\ \hline E(z) & Encoder \\ \hline D & Discriminator \\ \hline C & Classifier \\ \hline P(y) & Probability—Discriminator output \\ \hline P(c) & Probability—Classifier output \\ \hline H(x) & Entropy of a given distribution over data x \\ \hline \(\text{E}_{\text{x}\sim\text{P}_{\text{data}(x)}}\) & Expected value given x distributed as \(\text{P}_{\text{data}(x)}\) \\ \hline \(\parallel\text{f}\parallel_{\text{L}}^{2}\) & Laplacian Norm \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Notations used. an auxiliary cost in addition to forcing the generator to produce samples closer to the decision boundary, which was achieved by adding a term to the generator's objective function that penalized high density samples. This pushed the generated samples to move towards low-density areas. The final generator objective function was defined as shown in Equation (1).