# Appearance-Based Gaze Estimation as a Benchmark for Eye Image Data Generation Methods
## 5 Experiments  
### Data Augmentation for Gaze Estimation  
The data augmentation study was designed to test the real-world validity of the generated images. We chose the task of appearance-based gaze estimation to be our performance benchmark based on the following reasons. First, estimating gaze is the most fundamental task in image-based ET research. Second, modern appearance-based approaches learn a wide range of features that the underlying network finds to be useful for ET purposes. Making sure that the distribution of such features ends up being similar between the real and generated samples is an essential step in evaluating a synthetic data source. A severe distribution mismatch would cause the domain shift problem when the end-to-end solution is trained on some mix of real and fake data and then evaluated purely on real samples. Overall, a good generation method would be a tool for improving theperformance of the end task when the data produced by it are mixed in with the real training set, which should be especially useful when the availability of real data is scarce. This exact use case was assessed with the benchmark we define as follows.  
A subject-specific neural network was trained to regress eye images to gaze. The same reduced VOG real data set was used as in training all data generation methods. For every subject, the samples were randomly split into train and test sets using an 80%/20% ratio, and the following experiment was repeated 5 times in a cross-validation manner, each time with a different fold picked as a test set. The test set was set aside and used only at the evaluation step. The training set was further subdivided with the goal of simulating the scarce real data availability. There were 3 evaluation scenarios that each subject went through:  
1. \"The sparse real training set\": the baseline where only 5% of the original training set was retained for training by random subsampling.
2. \"Data generation method as a data augmentation tool\": for that same 5% of real samples, the remaining 95% were added back from a data generation method that was being evaluated. The second Blender data set defined earlier in the \"Synthetic Blender Data\" subsection was used as input to all data generation methods. Note that as explained there, it was generated in such a way that the resulting fake images should follow the real ones they replace in the gaze position and camera location as closely as possible.
3. \"The full real training set\": the gaze estimation error on the full 100% training set was used to set the highest bar for a data generation method the output of which would be indistinguishable from real data.  
The results were normalized to show the change in the gaze estimation error once the generated data were mixed in with the sparse training set of real images. The data point of 0% would represent no change in performance, and reaching 100% would mean that the performance on the full real training set was matched. This was done on a per-subject basis using the following formula (where \(GE_{rel}^{\%}\) is the gaze error change as a percentage relative to the baseline performance on the sparse training set, and \(GE_{i}^{*}\) is the gaze error in \({}^{\circ}\) achieved in the \(i\)-th evaluation scenario from the list above):  
\[GE_{rel}^{\%}=\frac{GE_{1}^{*}-GE_{2}^{*}}{GE_{1}^{*}-GE_{3}^{*}}\approx 100.0 \ (\%) \tag{2}\]