# A Method of Free-Space Point-of-Regard Estimation Based on 3D Eye Model and Stereo Vision
## 3 Methods
### 3D Point-of-Regard Estimation
#### 3.3.1. Eyeball Features and Personal Calibration  
In this paper, pupil center normal is introduced to the estimation of the eye model parameters. The vector of pupil center normal is seemed as the vector from the eyeball center to the pupil center (i.e., the direction of the eye optical axis). We extract the pupil contour and calculate the coordinates of the space circle where the pupil contour is located, and then calculate the pupil center normal. In the personal calibration process, the eyeball center is estimated by using the spatial intersection of the lines where the multiple pupil center normal is located. There is a fixed angle \(\theta\) between the eye's visual axis and the eye's optical axis, which varies from person to person and is usually about five degrees. The optical axis is defined by the spatial line from the eyeball center \(e\) to the pupil center \(p\), while the visual axis is a straight line passing through the pupil center \(p\), and the visual axis is defined as the spatial line-of-sight. Since the actual calculated spatial lines-of-sight do not necessarily intersect, this paper takes the midpoint between the nearest points of the binocular visual axes as the point-of-regard (PoR). The single 3D eye model of this method is shown in Figure 7.  
Firstly, we calculate the spatial coordinate of the eyeball center \(e\). The pupil center \(p\) and the corresponding pupil normal vector \(\overset{\rightarrow}{n}\) at each fixation need to be obtained first. It is assumed that \(n\) times fixation steps are included in one calibration. We calculate the sum of the nearest distance between each pupil normal and other normal lines, and remove the pupil normal that is far away from other lines. The pupil normal vector is defined as \(\overset{\rightarrow}{n}\), and then adding labels to each line \(h_{i}=1,i\in[1,\cdots n]\), at last calculating the sum of the nearest distance between each pupil normal and other normal lines. It can be calculated as:  
\[sum_{di}=\sum_{j=1}^{n}\frac{(\overset{\rightarrow}{n_{i}}\times\overset{ \rightarrow}{n_{j}})\bullet\overset{\rightarrow}{p_{i}}p_{j}}{\|\overset{ \rightarrow}{n_{i}}\times\overset{\rightarrow}{n_{j}}\|},i\in[1,\cdots,n] \tag{1}\]  
Then set the label of the line whose distance is greater than the average distance to 0. \[h_{i}=\left\{ \begin{array}{ll}0,sum_{di}<\frac{1}{n}\sum\limits_{i=1}^{n}sum _{di}&,i\in[1,\cdots,n]\\ 1,otherwise\end{array} \right. \tag{2}\]  
Figure 7: Single 3D eye model in this method and the main eyeball features. The nearest points between each line and other lines are defined as \(pn_{ij}\) (\(x_{ij}\), \(y_{ij}\), \(z_{ij}\)). Then the calculated the eyeball center \(e\) is:  
\[e(x_{e},y_{e},z_{e})=\text{argmin}\left(\sum\limits_{i=1}^{n}\sum\limits_{j=1}^{ n}h_{i}\|e-pn_{ij}\|\right) \tag{3}\]  
We solve the \(x_{e}\), \(y_{e}\), and \(z_{e}\) by using the centroid formula:  
\[x_{e}=\frac{\sum\limits_{i=1}^{n}\sum\limits_{j=1}^{n}h_{i}x_{ij}}{\sum \limits_{i=1}^{n}\sum\limits_{j=1}^{n}h_{i}},y_{e}=\frac{\sum\limits_{i=1}^{n} \sum\limits_{j=1}^{n}h_{i}y_{ij}}{\sum\limits_{i=1}^{n}\sum\limits_{j=1}^{n}h_ {i}},z_{e}=\frac{\sum\limits_{i=1}^{n}\sum\limits_{j=1}^{n}h_{i}z_{ij}}{\sum \limits_{i=1}^{n}\sum\limits_{j=1}^{n}h_{i}} \tag{4}\]  
After obtaining the coordinate of the eyeball center \(e\), the pupil center \(p\) and the spatial coordinate of object \(P_{o}\) are all used to calculate both eyes. The estimation method of \(\theta\) is to calculate the relative spatial position between the fixation point and the pupil center. After multiple fixations, we can obtain multiple sets of fixation points and pupil center data, and then use the nonlinear optimization method to obtain better value of \(\theta\). As shown in Figure 7, the visual axis \(p_{i}\overset{\rightarrow}{P}_{oi}\) is formed by the connection of the spatial coordinates of \(p_{i}\) and \(P_{oi}\), and the direction of the optical axis is defined as the vector \(\overset{\rightarrow}{ep_{i}}\).