# 2D Gaze Estimation Based on Pupil-Glint Vector Using an Artificial Neural Network
## 2 Proposed Methods for Gaze Estimation  
According to the respective characteristics of linear and nonlinear regression, a novel 2D gaze estimation method based on pupil-glint vector is proposed in this paper. An improved artificial neural network (DLSR-ANN) based on direct least squares regression is developed to solve the mapping function between pupil-glint vector and gaze point and then calculate gaze direction. The flow-process of gaze direction estimation is shown in Figure 1. First, when gazing at the calibration markers on the screen, corresponding eye images of subjects are acquired through a camera fixed on the head-mounted gaze tracking system. Second, through preprocessing such as Otsu optimal threshold binarization and opening-and-closing operation, pupil and glint centers are detected by utilizing circular ring rays location (CRRL) method. As inputs of the proposed DLSR-ANN, pupil-glint vector is calculated through the subtraction of pupil and glint center coordinates. Third, a three-layer DLSR-ANN (input layer, hidden layer, and output layer) is developed to calculate the mapping function between pupil-glint vectors and corresponding gaze points. Finally, gaze points on the screen can be estimated according to the mapping function determined. Pupil-glint vector is calculated through the subtraction of pupil and glint center coordinate. 2 nd linear gaze mapping function based on pupil-glint vector is expressed as Equation (1). \[\left\{ \begin{array}{l}x_{ci}=a_{1}+a_{2}x_{ei}+a_{3}y_{ei}+a_{4}x_{ci}y_{ ei}+a_{5}x_{ei}^{2}+a_{6}y_{ei}^{2}\\ y_{ci}=b_{1}+b_{2}x_{ei}+b_{3}y_{ei}+b_{4}x_{ci}y_{ei}+b_{5}x_{ei}^{2}+b_{6}y_{ ei}^{2}\end{array} \right., \tag{1}\]  
where \(i=1,2,\cdots,N\). \(N\) is the number of calibration markers. \((x_{ci},y_{ci})\) is the coordinate of gaze calibration markers on screen coordinate system. \((x_{ei},y_{ei})\) is the coordinate of pupil-glint vector on image coordinate system. Least squares, as conventional linear methods, is utilized to solve the gaze mapping function shown in Equation (1). Residual error is defined as:  
\[R^{2}=\sum\
olimits_{i=1}^{N}\left[x_{ci}-\left(a_{1}+a_{2}x_{ei}+a_{3}y_{ei}+ a_{4}x_{ei}y_{ei}+a_{5}x_{ei}^{2}+a_{6}y_{ei}^{2}\right)\right]^{2}. \tag{2}\]  
By calculating a partial derivative of \(a_{j}\)\((j=1,2,3,4,5,6)\) in Equation (2), the constraint condition can be obtained as in Equation (3). \[\frac{\partial R^{2}}{\partial a_{j}}=-2\sum\
olimits_{i=1}^{N}\sigma_{j} \left[x_{ci}-\left(a_{1}+a_{2}x_{ei}+a_{3}y_{ei}+a_{4}x_{ei}y_{ei}+a_{5}x_{ ei}^{2}+a_{6}y_{ei}^{2}\right)\right]=0, \tag{3}\]  
Figure 1: Flow-process of gaze direction estimation. where \(\sigma_{1}=1\), \(\sigma_{2}=x_{ei}\), \(\sigma_{3}=y_{ei}\), \(\sigma_{4}=x_{ei}y_{ei}\), \(\sigma_{5}=x_{ei}^{2}\), \(\sigma_{6}=y_{ei}^{2}\). The value of \(a_{j}\) can be calculated according to Equation (4). \[\left[ \begin{array}{c}\sum\limits_{i=1}^{N}x_{ci}\\ \sum\limits_{i=1}^{N}x_{ci}x_{ei}\\ \sum\limits_{i=1}^{N}x_{ci}y_{ei}\\ \sum\limits_{i=1}^{N}x_{ci}y_{ei}\\ \sum\limits_{i=1}^{N}x_{ci}x_{ei}^{2}\\ \sum\limits_{i=1}^{N}x_{ci}y_{ei}^{2}\\ \sum\limits_{i=1}^{N}x_{ci}y_{ei}^{2}\\ \sum\limits_{i=1}^{N}x_{ci}y_{ei}^{2}\\ \end{array} \right]=\left[