# Appearance-Based Gaze Estimation as a Benchmark for Eye Image Data Generation Methods
## 2 Prior Work
### Learning-Based Data Generation Methods  
While synthetic data sets generated using 3D creation software have been utilized for multiple applications in the ET domain, many modern solutions for data generation rely on a framework of generative adversarial networks (GANs) introduced by [PERSON] et al. in [30]. Since its introduction, GAN-based methods have enabled breakthroughs in the generation of high-resolution photorealistic images [31] (BigGAN), [32] (StyleGAN3); high-fidelity speech audio synthesis [33] (Hifi-GAN); unsupervised image [34] (pix2 pix), [35] (CycleGAN), [24] (StarGAN-v2); and audio [36] (Mic2 Mic) domain translation. The original GAN framework training process is notoriously unstable. This issue inspired many follow-up works proposing alternative adversarial loss functions and optimization problem statements [37] (LS-GAN), [38] (WGAN-GP), [39] (DRAGAN). Unfortunately, none has been shown to be superior to others, and adjusting a GAN to a specific task usually requires an extensive hyperparameter search [40]. Another common challenge is to pick a set of metrics that allow for a fair and thorough evaluation of the generated output. One of the most comprehensive surveys on the pros and cons of various metrics available in the literature can be found in [41]. We are interested in applying the GAN framework in an attempt to combine the desirable characteristics of synthetic and real eye images, namely, to benefit from the efficiency of the synthetic rendering process (i.e., ability to rapidly vary a camera pose, a gaze vector, environmental lighting conditions, etc.) while capturing the inherent realism encountered in real-world data capture.  
In the more recent literature, diffusion probabilistic models [42] were shown to outperform GANs in generating realistic image data [43]. They represent a class of likelihood-based models that learn the underlying parametric distribution of given data samples. The training of a diffusion model typically involves a two-stage process with the forward and backward parts. During the forward diffusion an input sample is corrupted with a varying level of Gaussian noise. This effect is reversed in the backward diffusion with the denoising neural network, which learns to predict the noise added to the sample during the whole forward process. At inference time, the denoising network is utilized to progressively transform the initial random noise into an output sample from the learned domain [42]. The diffusion-based generative models achieve state-of-the-art results of increasingly impressive visual fidelity, but require thousands of hours of GPU train time and hundreds of millions of training samples to achieve that [44; 45; 46]. While there is an active work in making the various parts of the diffusion method more efficient [47; 48], the computational and data requirements still prohibit their use by smaller research groups with limited resources. We suggest these methods be considered in future work and decided to limit our scope to GAN-based approaches for image generation as a more attainable solution for the broader research community.  
GANs have been utilized to augment image-based training data in the literature throughout various studies. Examples include the generation of new images from existing training sets to improve classification accuracy [49], along with the transformation of contrast computer tomography (CT) images to improve segmentation in non-contrast CT images [50]. Within the eye image domain, the most notable application of data augmentation using GANs was an attempt to address the aforementioned lack of realism within synthetically generated images. In this section, we describe three of these networks--SimGAN [26], GazeGAN [51], and EyeGAN [25]--and discuss their applicability in our problem setting.