# Generative Adversarial Network for Class-Conditional Data Augmentation
## 3 The Proposed Method
### Class-Conditional GAN-Based DA  
As shown in the experiments in Figure 3, autoencoder initialization can broaden the inter-class gap in the latent space and shrunken the intra-class gap compared with original autoencoders. Figure 4: Difficulty in high-resolution data generation. Thus, autoencoder initialization helps GANs to stably train imbalanced datasets. However, if samples are generated only with class-conditional noises, the training becomes unstable due to the possible overlaps between classes in the latent space. To settle this problem, we propose a variant of the GAN architecture called GANDA. In GANDA, minority classes are effectively generated using data from both majority and minority classes in imbalanced datasets to restore the balance of datasets. To generate a specific class sample, the proposed method explicitly conditions the class label to our generator. At the same time, class-conditional noises, which contain relative distances in the latent space with other classes, is used to leverage other class information. As the training proceeds, our method generates realistic minority classes and simultaneously improves the classification accuracy of both majority and minority classes. The proposed method consists of two main steps: conditional denoising autoencoder initialization and adversarial learning. We describe each step in detail, as follows. \(\bullet\) Conditional denoising autoencoder initialization: We first add noise \(\epsilon\) to an input image \(\mathbf{x}\in\mathbb{R}^{d}\), which can diversify output images. Subsequently, we concatenate the noisy input image with a one-hot label \(\mathbf{y}=\{c_{1},c_{2},\dots,c_{n}\}\) and feed it into the encoder \(\mathbf{\phi}\in\mathbb{R}^{d\times p}\), in which class-aware latent features \(\mathbf{z}\) are extracted as follows,  
\[\mathbf{z}=\mathbf{\phi}(\mathbf{x}+\mathbf{e},\mathbf{y}). \tag{1}\]  
The estimated latent feature is concatenated with the same one-hot label \(\mathbf{y}\) once more to explicitly guide the generation process. The concatenated feature is then fed into the decoder \(\mathbf{\psi}\in\mathbb{R}^{p\times d}\), which results in \(\mathbf{x^{\prime}}\) as follows,  
\[\mathbf{x^{\prime}}=\mathbf{\psi}(\mathbf{z},\mathbf{y}). \tag{2}\]  
We train the proposed conditional denoising autoencoder using the reconstruction loss \(\mathcal{L}_{recon}\):  
\[\mathcal{L}_{recon}(\mathbf{x},\mathbf{x^{\prime}})=||\mathbf{x}-\mathbf{x^{\prime}}||_{2}. \tag{3}\]  
To optimize the objective function in (3), we adopt the \(l_{2}\) norm. Please note that the trained autoencoder describes the distribution of the latent codes of the classes. Thus, we initialize a part of the discriminator, \(D_{\epsilon}\), and generator, \(G\), using the parameters of encoder \(\mathbf{\phi}\) and decoder \(\mathbf{\psi}\), respectively. \(\bullet\) Adversarial learning: After conditional denoising autoencoder initialization, we train generator \(G\) and discriminator \(D\) via adversarial training. Before the adversarial learning proceeds, we use the encoder \(\mathbf{\phi}\) to determine the multinomial distribution of the class latent code in the training data. Then, the generator is trained to produce a fake class sample \(\hat{x}_{y}\) by selecting the latent code \(\mathbf{z_{y}}\), which is extracted from the aforementioned multinomial distribution determined by \(\mathbf{\phi}\) and is concatenated with the corresponding one-hot label \(\mathbf{y}\). \[\hat{x}_{y}=G(\mathbf{z_{y}}|\mathbf{y}). \tag{4}\]  
The discriminator classifies the generated sample as belonging to one of \(n\) classes or as being fake. \[D(\mathbf{x}|\mathbf{y})=P_{D}(y=i|\mathbf{x},\mathbf{y})=\frac{\exp(l_{i})}{\sum_{j=1}^{k+1} \exp(l_{j})}, \tag{5}\]  
where \(l_{j}\) denotes the \(j\)-th output of the discriminator. Please note that the discriminator can have \((n+1)\) outputs in total, where \(n\) is the number of class labels, and \(1\) is for the fake label. The objective function for adversarial learning is as follows,  
\[