# Advancements in Gaze Coordinate Prediction Using Deep Learning: A Novel Ensemble Loss Approach
###### Abstract  
Recent advancements in deep learning have enabled gaze estimation from images of the face and eye areas without the need for precise geometric locations of the eyes and face. This approach eliminates the need for complex user-dependent calibration and the issues associated with extracting and tracking geometric positions, making further exploration of gaze position performance enhancements challenging. Motivated by this, our study focuses on an ensemble loss function that can enhance the performance of existing 2D-based deep learning models for gaze coordinate (x, y) prediction. We propose a new function and demonstrate its effectiveness by applying it to models from prior studies. The results show significant performance improvements across all cases. When applied to ResNet and iTracker models, the average absolute error reduced significantly from 7.5 cm to 1.2 cm and from 7.67 cm to 1.3 cm, respectively. Notably, when implemented on the AFF-Net, which boasts state-of-the-art performance, the average absolute error was reduced from 4.21 cm to 0.81 cm, based on our MPIIFaceGaze dataset. Additionally, predictions for ranges never encountered during the training phase also displayed a very low error of 0.77 cm in terms of MAE without any personalization process. These findings suggest significant potential for accuracy improvements while maintaining computational complexity similar to the existing models without the need for creating additional or more complex models. appearance-based model; ensemble loss function; Gaze estimation; human computer interaction; multi loss function +
Footnote †: journal: Medical  
gaze direction. Additionally, a ResNet-trained model that uses images of faces and pairs of eyes has outperformed iTracker [7] in gaze accuracy. Another notable gaze estimation model, SAGE (fast accurate gaze tracker) [10], combines eye images and landmarks, demonstrating high effectiveness. These and many other types of research have been done on gaze estimation from facial images [11; 12; 13]. However, a significant obstacle to furthering these models is the limited availability of resources. High-performance models often require sophisticated deep architectures and significant computational resources, along with extensive datasets [14]. This creates a formidable impediment, hindering developers from deploying these models in everyday services and applications. This paper presents a computational framework for gaze estimation through regression analysis, utilizing ensemble loss functions to enhance efficiency. Our aim is to comprehensively estimate gaze coordinates while preserving computational resources. Our methodology enhances gaze tracking precision by integrating an ensemble loss approach within the deep learning framework. Specifically, we employ an ensemble of loss functions that not only evaluate the regression accuracy of the predicted gaze coordinates but also incorporate a multi-class classification scheme. By combining these distinct aspects of loss functions, our model effectively addresses both the precision of gaze coordinates and their categorical accuracy, leading to improved performance in gaze tracking applications. This innovative approach enables the achievement of highly accurate gaze coordinates by adjusting loss functions, eliminating the need for extensive network rebuilding, complex neural network training, or the accumulation of immense datasets tailored to a specific deep neural network. Our proposed framework reduces computational and resource expenses and enhances resource efficiency significantly. Additionally, there have been advancements in appearance-based models for predicting gaze direction in 3D angles and forecasting standardized X and Y coordinates on a 2D plane [15; 16]. These advancements are applicable to various devices, including laptops, smartphones, and tablet PCs. As the field progresses, this research contributes to the enhancement of gaze estimation techniques, providing a practical and resource-efficient solution with broad applicability.