# Survey on Implementations of Generative Adversarial Networks for Semi-Supervised Learning
## 3 Literature Review of GANS for SSL
### The TripleGAN Approach  
Another class of techniques for the implementation of semi-supervised GANs is based on the TripleGAN architecture proposed by [PERSON] et al. [26]. Addressing the issue that the generator and discriminator cannot be optimal at the same time, this paper proposed a different approach to [PERSON] et al. [29]. TripleGAN consisted of injecting an additional classifier, which along with the generator characterized the conditional distributions between images, while the discriminator was limited to identifying fake image-label pairs. Figure 4 shows the architecture used for the TripleGAN where the discriminator either outputs an accept (A) or a reject (R), which serve as the adversarial losses, while the classifier produces the cross-entropy losses (CE) for the supervised part of the learning. The discriminator in TripleGAN takes image-label pairs of which there are 3 kinds; a true data-label pair from the labeled data (\(x\),\(y\)), a generated data-label pair (\(G(z)\),\(y^{\prime}\)), and an unlabeled data sample assigned a pair by passing it through the classifier (\(x_{u}\),\(P(c)\)) using pseudo-labeling. The resulting objective function is shown in Equation (4). \[ \begin{split}\min_{C,G}\max_{D}& E_{(x,y)\sim p(x,y)}[ \log D(x,y)]+\alpha E_{(x,y)\sim p_{z}(x,y)}[\log(1-D(x,y))]\\ &+(1-\alpha)E_{(x,y)\sim p_{\bar{z}}(x,y)}[\log(1-D(G(y,z),y))] \end{split}  \tag{4}\]  
EnhancedTGAN [34] was an extension of TripleGAN that redesigned the training targets of the generator and classifier. They designed the generator to produce images on the basis of a class distribution that was regulated by a feature-semantics matching term in the loss function. Furthermore, they added another classifier, which worked in collaboration to provide additional categorical information for the generator to train on. Another notable extension of TripleGAN was MarginGAN [27] where the classifier increased the margin for real samples, and decreased the margin for fake samples. The generator tried to increase the margin for the fake samples only. This approach further helped prevent the drop in performance that typically happens due to the misclassification of a pseudo-label. They based their theory on [29] and aimed to implement the theory within the TripleGAN framework. \(\Delta\)-GAN [35] combined ideas from BiGAN and TripleGAN. The model consisted of two generators and two discriminators, with the generators providing bidirectional mapping between domains and the discriminator classifying the real data pairs from the two kinds of fake data pairs. Structured GANs [36] were similar to \(\Delta\)-GAN but assumed that generated data were conditioned on two independent latent variables, one of which encoded the designated semantics (\(y\)), while the other accounted for other factors of variation (\(z\)). Under the assumption that these latent variables were independent of each other, the authors proposed a set of two inference networks, one to map an input data point (\(x\)) to the designated semantics (\(y\)), and the other to map an input point to \(z\). These networks were trained using two different adversarial games, one for each mapping of the input data. R\({}^{3}\)-CGAN [28] was another GAN architecture based on \(\Delta\)-GAN. This architecture was based on the observation that the classification network often gives incorrect yet confident predictions on unlabeled data while generating pseudo-labels. Furthermore, due to the imbalance between real and fake samples, the discriminator learns the real samples and rejects any unseen data even if they are real. The authors proposed using a regularization approach based on Random Regional Replacement in the learning process of the classification and discriminative networks. They implemented two discriminative networks in addition to the classifier and the generator. Fake sample pairs of two types were used, one consisting of synthesized data paired with the target label, and the other consisting of an unlabeled sample paired with its pseudo-label.