# A Method of Free-Space Point-of-Regard Estimation Based on 3D Eye Model and Stereo Vision
## 3 Methods
### 3D Point-of-Regard Estimation
#### 3.3.2 Eyeball Features Coordinate Alignment  
The inner eye corner is a stable facial feature and can be used as a reference for facial posture. We firstly establish an eye coordinate system, the structure and the definition are shown in Figure 8, the origin of the eye coordinate system is established with the \(P_{AL}\), and its coordinate axis direction and the scale factor are the same as the device coordinate system. Meanwhile, we form the inner eye corner vector from \(P_{AL}\) to \(P_{AR}\), and it should be used as a reference to describe the position of the eyeball center and the pupil center relative to the gaze tracking system. Firstly, we get a pair of the coordinates of the eyeball centers and a pair of the coordinates of the inner eye corners in the calibration. Then we describe how to use the calibrated eyeball center when we only know the coordinates of the inner eye corners during utilizing the proposed system. Take the coordinate alignment of the eyeball centers \(P_{AL}\) and \(P_{AR}\) as an example. The positions of the eyeball center \(e_{l}\) and \(e_{r}\) are calculated in the calibration process, but these two coordinates could be changed when the head posture changes. The left and right inner eye corners are obtained at the same time when calibrating, and they can also be calculated when using. Therefore, the coordinates of the eyeball center can be recovered by calculating the relative position between the inner eye corner and the eyeball center. In this paper, we choose the inner corner vector as the reference for alignment. The vector \(\overrightarrow{v_{o}}\) obtained during calibration is the initial inner corner vector, and the eye coordinate system is established at the left eye corner \(P_{AL}\). As shown in Figure 8, the vector \(\overrightarrow{v_{er}}\) and the vector \(\overrightarrow{v_{er}}\) can be represented by \(\overrightarrow{v_{o}}\) and rotation matrix. Get:  
\[\overrightarrow{v_{el}}=R_{el}\overrightarrow{v_{0}}=P_{AL}\overrightarrow{e _{l}},\overrightarrow{v_{er}}=R_{er}\overrightarrow{v_{0}}=P_{AL} \overrightarrow{e_{r}} \tag{12}\]  
When in use, we first get the \(\overrightarrow{v_{i}}\) in the eye coordinate system, then calculate the rotation matrix \(R\)' from \(\overrightarrow{v_{o}}\) to \(\overrightarrow{v_{i}}\):  
\[\overrightarrow{v_{i}}=R^{\prime}\overrightarrow{v_{0}} \tag{13}\]  
Then align the left corner coordinate of the \(i\) time with the initial left corner coordinate, and at the same time get the translation vector \(T\)'. The left and right eyeball vectors where \(\overrightarrow{v_{i}}\) is located are:  
\[\overrightarrow{v_{er}^{\prime}}=R_{er}\overrightarrow{v_{i}}=R_{er}R^{ \prime}\overrightarrow{v_{0}},\overrightarrow{v_{el}^{\prime}}=R_{el} \overrightarrow{v_{i}}=R_{el}R^{\prime}\overrightarrow{v_{0}} \tag{14}\]  
Therefore, when in use, the \(e_{l}\) and \(e_{r}\) coordinates can be written as follows:  
\[ \begin{split} e_{r}&=R_{er}R^{\prime}\overleftarrow{ v_{0}}\frac{\left\|\overrightarrow{v_{er}}\right\|}{\left\|\overrightarrow{v_{er}} \right\|}+T^{\prime}\\ e_{l}&=R_{el}R^{\prime}\overleftarrow{v_{0}}\frac{ \left\|\overrightarrow{v_{el}}\right\|}{\left\|\overrightarrow{v_{er}} \right\|}+T^{\prime}\end{split}  \tag{15}\]  
Similarly, the pupil center coordinates are also calculated using the above derivation process. When using the proposed gaze tracking system, the pupil center coordinates solved in real-time do not needed to be aligned. However, in the experiments described below, the pupil centers acquired at different times need to be aligned. If necessary, the \(p_{l}\) and \(p_{r}\) coordinates after alignment can be written as follows:  
\[ \begin{split} p_{r}&=R_{pr}R^{\prime}\overleftarrow{ v_{0}}\frac{\left\|\overrightarrow{v_{pr}}\right\|}{\left\|\overrightarrow{v_{ pr}}\right\|}+T^{\prime}\\ p_{l}&=R_{pl}R^{\prime}\overleftarrow{v_{0}} \frac{\left\|\overrightarrow{v_{pl}}\right\|}{\left\|\overrightarrow{v_{pl}} \right\|}+T^{\prime}\end{split}