# Generative Enhancement of 3D Image Classifiers
## 6 Results and Discussion
### Classification  
All three models have been evaluated while using the test data from the ModelNet10 dataset. Classification accuracy, as well as mean average precision, have been calculated. The results are shown in Table 1. Several other modern models utilizing generative modeling with published ModelNet10 results are also included in the table. The results show that generative enhancement of a very simple convolutional classifier (plain CNN) brings its average precision close to the state-of-the-art classifiers. The augmented CNN classifier also shows improvement over the plain CNN, but the difference is not as significant as between the plain CNN and GAN. This suggests that the benefits of generative enhancement do not only stem from its augmentative effect on the dataset. Some of the features learned to discriminate between fake and real examples are also likely utilized during the classification, which improves accuracy. An experiment (labeled as 'no-fake') shows a slightly lesser improvement of accuracy when training the model without using the extended ground truth vector shown in (3), therefore non-adversarially. The ROC plot in Figure 5 shows another comparison of classification performance of our three models and the confusion matrix in Figure 6 provides some additional insight into classification performance. \begin{table}
\begin{tabular}{l c c} \hline \hline  & **Accuracy [\%]** & **Mean AP [\%]** \\ \hline VRN Ensemble [13] & 97.14 & \\ \hline Achlioptas et al. [14] & 95.40 & \\ \hline VIPGAN [48] & 94.05 & 90.69 \\ \hline Primitive-GAN [15] & 92.20 & \\ \hline
3D-GAN [10] & 91.00 & \\ \hline Plain CNN (ours) & 83.04 & 83.75 \\ \hline Augmented CNN (ours) & 84.58 & 86.31 \\ \hline GAN Classifier (ours) & 89.21 & 89.18 \\ \hline GAN Classifier no-fake (ours) & 87.11 & 90.04 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Comparison of classification accuracy and mean average precision (Mean AP) score of generative modelling-based classifier architectures validated on the ModelNet10 dataset [19]. These are the results from the public ModelNet leaderboard. Some of the models do not publish their Mean AP. Figure 5: Linear and semilogarithmic ROC curves for each of the three designed models. These curves are calculated by micro-averaging the curves for each class. Although the accuracy of the designed models is not superior to the state of the art, the results show that training a classifier inside a GAN model can produce a significant accuracy improvement when compared to conventional training. In our case, the relative improvement from plain CNN to the GAN-trained classifier was 7.43%. This is likely one of the simplest types of classifiers that is compatible with this methodology, clearly demonstrating its principle and benefits. However, the advantage of this methodology is that it is general enough to be applicable to many of the existing compatible classifiers that already achieve significantly better performance than the models used in our demonstration. The improvement of 7.43% that is shown on the simple 3D CNN is obviously not guaranteed in every case; however, it is likely that some of the more sophisticated classifiers can also benefit from this enhancement, albeit with a lower relative performance increase. Comparing the augmented CNN to the GAN classifier, the difference is smaller (5.47% relative improvement), however still significant. This suggests that the improvement is caused not only by introducing noise to the dataset. It also means that the generator is not just a data augmentation mechanism, because, in that case, the augmented CNN and GAN classifier should achieve similar performance. The results suggest that there exists some form of interaction between the generative and classifying part of a conditional GAN network that improves the classification accuracy of the model. The interactions also likely improve the generative performance, as other researchers recommend using class labels, if available when training a GAN network in general [21].