# Ocular Biometrics Recognition by Analyzing Human Exploration during Video Observations
## 3 Proposed Method
### Gaze Vector Estimation  
The region of the image containing the face is the input for the Gaze Vector Estimation block. First of all, 68 2D facial landmarks are detected and tracked on the face region employing a probabilistic patch expert named Conditional Local Neural Fields (CLNF). This way, spatial and non-linear relationships between pixels (neighboring and longer distance) and the alignment probability of a landmark [46] are learned. The employed patch expert is the Local Neural Field (LNF), an undirected graph that models the conditional probability of the probability that a patch is aligned (\(\mathbf{y}\)) depending on the pixel intensity values in the support region. It also includes a neural network layer that can capture complex non-linear relationships between pixel values and the output responses.  
For a particular set of observations of pixel intensities \(\mathbf{X}=\{x_{1},x_{2},\ldots,x_{n}\}\), the set of output variables \(\mathbf{y}=\{y_{1},y_{2},\ldots,y_{n}\}\) is predicted by the model that is the conditional probability distribution with density:  
\[P(\mathbf{y}|\mathbf{X})=\frac{exp(\psi)}{\int_{-\infty}^{\infty}exp(\psi)\,dy} \tag{1}\]  
The potential function \(\psi\) is defined and determined by four model parameters \(\{\alpha,\beta,\gamma,\Theta\}\) that are learned by maximizing the conditional log-likelihood of the LNF on the training sequences. See [46] for more details.  
The locations of facial feature points in the image are modeled using non-rigid shape and rigid global transformation parameters trained on the LFPW [47] and Helen [48] datasets. The CLNF is further optimized with a Non-uniform Regularized Landmark Mean-Shift technique [49]. Once the 2D-3D correspondences are known, the 3D rotation and translation vectors \(R,T\) are found by the Perspective-n-Point algorithm based on Levenberg-Marquardt optimization [50] to map the detected 2D landmark position and a static 3D head pose model.  
In particular, given the camera intrinsic calibration matrix \(K\) as:  
\[K=\begin{bmatrix}f_{x}&0&c_{x}\\ 0&f_{y}&c_{y}\\ 0&0&1\end{bmatrix}\,, \tag{2}\]  
Figure 1: A block diagram representation of the proposed pipeline.  
then, for each correspondence between a landmark in image plane coordinates (\(p_{IP}\)) and 3D point (subscript \(p_{3D}\)) we have:  
\[s\,p_{IP}=K\,[\,R\,|\,T\,]\,p_{3D} \tag{3}\]  
whose least-squares solution represents the 6-DOF pose of the face.  
The information provided by the generic CLNF deformable shape registration is applied to the eye region to find the location of the eye and the pupil and, consequently, to compute the eye gaze vector for each eye as suggested in [51]. For this step, training of the CLNF model is performed on the SynthesEyes [52], a dataset of eye-patches synthetically generated. Gaze vector is computed in the following way: a ray is cast from the camera origin through the center of the 2D pupil pixel position, and its intersection with the eyeball sphere is used to estimate 3D pupil position. The vector passing from the two 3D points represented by the eyeball and the pupil is the estimated gaze vector. In the proposed pipeline, the gaze vector is estimated only for the right eye, extracting its \(x,y,z\) coordinates.