# Generative Enhancement of 3D Image Classifiers
## 1 Introduction  
Generative modeling has been successfully applied in order to solve a wide range of tasks, including realistic data generation (numeric time-series [1], text [2], image [3], audio [4], video [5], etc. ), extrapolation [6], image super-resolution [7], inpainting [8], and many others. Data classification is also one of those, even though it is usually not the main goal of designing a generative network. Deep generative models have been proven to learn a salient representation of the data that they had been trained on, which is ideal for the classification task. This latent representation is perceptually significant on the generated output, which has been shown by various latent vector arithmetic experiments [9; 10]. Lately, the Generative Adversarial Network (GAN) is one of the most popular varieties of generative models [11]. From the classification perspective, a GAN network's ability to generate highly realistic data indicates a potential for high performance in classification tasks, especially where large training datasets are not available. Limited datasets are not uncommon in the domain of 3D imaging. While GAN was not originally designed for classification, simple modifications exist to enable this functionality, such as 3D-GAN [10]. One such modification is to use the discriminator itself as a classifier. This can be achieved by extending the existing discriminator output with additional units, one per class, and training the model (semi)supervisedly. [12] While mostly used with 2D image data, a straightforward adjustment of the network (using 3D (de)convolution instead of 2D) enables the use of 3D data in the form of voxel grids or point clouds. State of the art generative [13; 14; 15] and non-generative [16; 17; 18] classifiers compete very closely in terms of 3D image classification accuracy, as can be seen in the results of benchmarks, such as ModelNet [19]. While models that utilize the two approaches differ significantly in structure, we seethe potential for their combination by enhancing the non-generative classifiers with elements of generative modeling. Aside from improving the classification accuracy, this could likely help with the 3D domain-specific problems, such as small datasets, too generic and synthetic training examples, etc. This idea is lightly supported in [20], where the authors observe an occurrence of beneficial interaction within the model when combining supervised and unsupervised loss while training a feature matching GAN classifier. This hints towards a potential for such interactions within a generative classifier, which allows further improvement of non-generative state-of-the-art models by embedding them within a generative network architecture. For these reasons, we propose a general methodology of generative enhancement, which aims to enable any compatible classifier in order to utilize a generative modeling approach to improve its classification accuracy. Although it is not theoretically limited to a particular type of data, this methodology requires a fully differentiable trainable generator. We propose and evaluate this methodology for voxel grid 3D format by applying it to a simple 3D convolutional classifier. We investigate the reproducibility and measurability of the benefits of this transformation by comparing the classification performance of the resulting conditional GAN network to the original 3D Convolutional Neural Network (CNN) classifier, identical to the GAN discriminator. Experiments are also performed in order to investigate possible explanations, such as the introduction of noise or data augmentation by the generator. The sections of this paper are structured in the following manner: following the introduction in Section 1, Section 2 discusses related work and publications in the field relevant to this paper. Section 3 describes the dataset that was used in the following experiments, as well as the details of its preprocessing and transformation. Section 4 presents the proposed Generative Enhancement methodology.