# Low-Complexity Pupil Tracking for Sunglasses-Wearing Faces for Glasses-Free 3D HUDs
## 2 Methods
### Nose-Month-Face Boundary Tracking: Alignment and Tracker Checker
If \(\beta_{left}\) is a linear regression matrix, then we can write  
\[E_{left}=X\beta_{left}. \tag{1}\]  
Similarly, the right eye position, \(E_{right}\), can be formulated using \(\beta_{right}\):  
\[E_{right}=X\beta_{right}. \tag{2}\]  
Solving Equations (1) and (2) leads to  
\[\beta=X^{+}E_{left}, \tag{3}\]  
where \(X^{+}\) is the pseudo-inverse of \(X\) when assuming a linear relationship between \(E\) and \(X\). Collecting more data for \(X\) and \(E\) produces a more accurate value of \(\beta\). This process of eye position estimation using a supervised linear regression method from n-samples of the \(11\) nose-mouth-face boundary (NMB) points is illustrated in detail below (Box 1):  
**Box 1. ** Detailed processes of eye position estimation using a supervised linear regression method. \begin{tabular}{l l} \(\checkmark\) & 1-sample \\  & : eye position from \(11\) NMB points \\  & : \(E_{left}=X\beta_{left}\) \\  & : \(e_{1}=\ a_{1}x_{1}+a_{2}y_{1}+\ldots+a_{11}y_{11}\) \\  & : \(e_{2}=\ b_{1}x_{1}+b_{2}y_{1}+\ldots+b_{11}y_{11}\) \\  & : \([e_{1}\ e_{2}]=[x_{1}\ y_{1}x_{2}y_{2}\ldots x_{11}y_{11}]\left[ \begin{array} []{c}a_{1}b_{1}\\ a_{2}b_{2}\\ \ldots\\ a_{11}b_{11}\end{array} \right]\) \\ \(\checkmark\) & n-samples \\  & : n-samples/ eye positions from n-samples/ 11 NMB points by linear regres-sion \\  & : \(E_{left}=X\beta_{left}\) \\  & : \( \begin{bmatrix}e_{11}&e_{21}\\ e_{12}&e_{22}\\ e_{1n}&e_{2n}\end{bmatrix} = \begin{bmatrix}x_{1,1}\ y_{1,1}x_{2,1}y_{2,1}\ldots x _{11,1}y_{11,1}\\ x_{1,2}\ y_{1,2}x_{2,2}y_{2,2}\ldots x_{11,2}y_{11,2}\\ \ldots\\ x_{1,n}\ y_{1,n}x_{2,n}y_{2,n}\ldots x_{11,n}y_{11,n}\end{bmatrix} \left[ \begin{array}{c}a_{1}b_{1}\\ a_{2}b_{2}\\ \ldots\\ a_{11}b_{11}\end{array} \right]\) \\  & \(\begin{bmatrix}\checkmark\) \\and video database entries captured inside and outside an office environment of 10 people wearing different styles of sunglasses. The average aligner precision error was 2 mm for the bare-face images and 10 mm for the sunglasses-wearing images. \begin{table}
\begin{tabular}{c c c} \hline \hline  & **Bare-Face Eye Tracker** &
\begin{tabular}{c} **Sunglasses-Wearing Face** \\ **Eye Tracker** \\ \end{tabular} \\ \hline Precision (mean error) & 2 mm & 10 mm \\ Distance between & \(\sim\)50–200 cm & \(\sim\)50–200 cm \\ camera and user & & \\ Speed (ms/frame) & \(\sim\)2–5 ms (2.5 GHz CPU) & \(\sim\)2–5 ms (2.5 GHz CPU) \\ CPU consumption & 10\% & 10\% \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Performance of the proposed eye-tracking method for bare and sunglasses-wearing faces. Figure 6: Seamless eye tracking for a user wearing sunglasses. A stereo RGB camera was used for 3D position conversion. The two columns show the left and right images from the stereo camera. The two databases were trained using the SIFT and SDM algorithms. Moreover, both bare-face and sunglasses-wearing-face image databases were used to inform the eye position estimation algorithm based on the supervised linear regression algorithm. Our method demonstrated high accuracy and speed, estimating the pupil center position with less than 10 mm error at a speed of 9 ms for sunglasses-wearing faces.