# Appearance-Based Gaze Estimation as a Benchmark for Eye Image Data Generation Methods
## 6 Results
### Visual Inspection  
The EyeGAN-RITNet generated images can be seen in Figure 14. They tend to have good perceptual quality, gaze and camera position consistency with the input segmentation mask of the depicted Blender render. The examples of image quality issues are provided in Figure 15. We can see that the images with certain edge gaze angles exhibit distorted iris shapes, the imperfectly segmented pupil leads to image artifacts, and the case of an almost closed eyelid is challenging. Moreover, we found that good consistency with the input segmentation mask does not always result in good gaze alignment between the synthetic render that corresponds to this mask and the generated image. This finding suggests the exploration of entities other than a segmentation mask that would be a better compact representation of ET image content. The major conceptual issue with the EyeGAN method is the complete lack of control over the output subject appearance, which makes it unusable as a subject-specific data generator. Note that this could be fixed by training multiple EyeGAN models--one per subject. Nevertheless, this solution carries a significant computational burden and has poor scalability.  
The StarGANv2-based methods are the only ones that provide control over the visual subject appearance in their output. The StarGANv2-SPADE+AdaIN generated images shown in Figure 16 have improved perceptual quality, while it is visually noticeable that the overall content consistency is degraded when compared to EyeGAN-RITNet. The poor consistency is the most notable for the pupil, an observation that is supported by the lowest per-class mIoU in Table 1. The overall gaze consistency is manifested the most by the correctness of the iris placement, which is indeed mediocre, according to the corresponding per-class mIoU of 0.56. There is another major issue with the StarGANv2-SPADE+AdaIN results that can be seen in Figure 17. The resulting appearance can be controlled via a given VOG real image only to a certain extent, as the network suffers from mode collapse for certain subject identities. We manually went over all 40 subjects in the training set to assess the severity of the mode collapse issue of the StarGANv2 method. The 24 subjects had good visual consistency in the output images, and 5 of them can be seen in Figure 16. The remaining 16 were deemed to have bad consistency (with 5 of them shown in Figure 17). The common theme across all StarGANv2 issues seems to be some level of network overfitting. The observed results may suggest that the network had mostly learned to reproduce an image from the training set with the segmentation mask that is the closest to the input one. Therefore, we propose adding two following steps for future work on improving the StarGANv2 results. First, the overfitting should be addressed with the appropriate network regularization techniques. Second, other methods in addition to the AdaIN should be explored to improve the visual style transfer of the subject appearance. For instance, the features that are learned by the style encoder may not be as relevant to the subject appearance, which can be improved by training the encoder in a multi-task scenario with the added goal of the subject identity classification.