# Generative Enhancement of 3D Image Classifiers
## 5 Models and Methods
### 3D Conditional Generative Adversarial Network  
The last model is a 3D Conditional Generative Adversarial Network (3D CGAN), which consists of a generator-discriminator pair connected in sequence and trained in opposition to each other. The role of the generator is to transform a latent representation (vector) of an object to its visual representation (voxel grid), while the discriminator tries to distinguish real samples (taken from the training set) from the fake ones (created by the generator). The objective of the adversarial training is to find the equilibrium, where the generator produces samples that are realistic enough to confuse the discriminator [11]. A conditional GAN introduces a condition for the generator, specifying the desired output by providing a one-hot encoded class vector or another form of specification (e.g., keywords or description text), as originally shown in [45]. In our model, the conditioning is achieved by feeding a one-hot encoded class vector to the generator input. However, the discriminator does not receive this information, as it would trivialize its classification objective and make it unusable for classifying unknown samples (as opposed to [45]). The starting point for this architecture has been the deep convolutional 3D GAN model that was published in [10]. The main difference from the standard GAN architecture in our model is the discriminator, which is mostly identical to the plain and augmented CNN models mentioned above. The only difference between the discriminator and two previously described convolutional classifiers is in the output layer, which contains one extra output in addition to the class label, which indicates whether the input is considered to be fake or not. Essentially, it distinguishes one additional \"fake\" class. The layers of discriminator use the Leaky ReLU activation function, along with batch normalization. The generative part of the CGAN network (generator), as shown in Figure 3, consists of three deconvolutional (or transposed convolutional) layers with a descending amount of volumetric filters. The upsampling of the activation volumes is achieved by kernel stride, which is equal to 2 in every layer. ReLU activation function is used after each layer, followed by a batch normalization function. The output layer uses _tanh_ activation function instead to fit the output into the dataset value range of \([-1,1]\). The generator is conditioned by appending the desired one-hot encoded class vector to the latent feature vector, containing 200 feature values that were sampled from a Gaussian distribution \(\mathcal{N}(0,1)\) forming an input vector \(\mathbf{z}\). The input is first fed through a fully connected layer expanding it to the shape required by the first deconvolutional layer. The dense layer is also followed by ReLU and batch normalization functions. The whole 3D CGAN classifier network, as shown in Figure 4, is constructed by connecting the generator and discriminator in sequence. During training, the discriminator receives separate batches of real samples from the training dataset and batches of fake models that were generated by the generator network. During another step of the same training loop, the generator is trained to produce realistic examples of the desired class. The generator loss is calculated at the output of the discriminator, whose parameters are fixed. The loss function chosen to train the discriminator, as shown in (2), is a standard categorical cross-entropy. \[L_{D}=-\sum_{i=1}^{M}\sum_{j=1}^{C+1}\mathbf{y}_{j}^{*(i)}\log(\mathfrak{g}_{j}^{*( i)}) \tag{2}\]  
An extended target (ground truth) vector \(\mathbf{y}^{*}\) is constructed in order to include both fake-real discrimination error and classification error, as shown in (3). This is necessary for the adversarial training of both the generator and the discriminator so that the generator is taught to produce realistic samples. Without this extension, the generator is only trained to generate noise that will be categorized into the desired class.