# A Method of Free-Space Point-of-Regard Estimation Based on 3D Eye Model and Stereo Vision
## 3 Methods
### Customized Gaze Tracking System  
Gaze tracking devices require different configurations of cameras and light source to satisfy different methods. Three conditions need to be considered when designing a head-mounted gaze tracking system [33]:  
1. Ensure a large field of view;
2. Reduce or avoid the influence of environment illumination on images; and
3. The device should be miniaturized, light-weight, and low-cost. The proposed head-mounted gaze tracking device is based on three pairs of stereo vision cameras and a pair of semi-transparent mirrors. Two pairs of stereo cameras cover left and right eyes, and another pair of stereo cameras are used to capture scene images, all of which are imaged through semi-transparent mirrors. Two near-infrared light emitting panels are, respectively, arranged between the eyes as light sources, which are used for generating dark pupil images and improving the contrast of the eye images. The structure and the photo of the proposed system are shown in Figure 2.  
Figure 1: The diagram of the proposed 3D point-of-regard estimation method. Figure 2: The structure and the photo of the proposed gaze estimation system. The device is formed by two modules. As shown in Figure 2, each module is equipped with a semi-transparent mirror, which enables the eye cameras to capture eye images taken from the front view, and the scene cameras to obtain the scene images (i.e., scene cameras are fixed under the semi-transparent mirrors and their horizontal field-of-view is 120 degree). The distance between the two modules is not fixed, and can be adjusted according to the pupillary distance. The resolution of a single camera is \(640\times 580\) pixels, and we choose a smaller focal length in order to get a larger field of view. The semi-transparent mirror reduces the size of the whole device, and the optical coating of the mirror also has some special features. On the side close to the eye, the coating of the semi-transparent mirror has a reflectivity of over 90% in the near infrared band. The image captured by the eye cameras through the mirror can avoid noise caused by environment light, while the eye area is illuminated by the near infrared light source, making the eye image clearer. Stereo cameras can acquire the spatial coordinates of eye features accurately. Two pairs of the binocular stereo cameras capture images of both eye regions, respectively. The system calculates the positions of each eye features independently, but these coordinate values are eventually aligned to the device coordinate system. A pair of the scene cameras also compose a stereo vision system. The scene cameras are not directly involved in the gaze estimation in this method, but are used to capture the calibration target images in the calibration process. Additionally, they are also used to evaluate the accuracy of the whole method. The near infrared light source equipped in this device is a pair of low-power light emitting panels within the near infrared band. Traditional LED bulbs can ensure good reflection spots on the cornea, but the human eyes are usually stimulated, resulting in rapid fatigue of the users. The gaze estimation model in this paper does not use corneal reflection, so there is no need for a point light source. Therefore, the purpose of installing these two low-power light emitting panels is to improve the image contrast. The light emitting panel illuminates the whole eye area through reflection mirrors. The enhancement of the image is shown in Figure 3. Moreover, because of the soft light, the user does not feel uncomfortable when wearing the device for a long time. Every camera in the proposed gaze tracking device needs to be calibrated and determine the transformations of all cameras to one device coordinate system. Since all the images of cameras are captured through the semi-transparent mirror, the gaze estimation system in this paper defines the reality of cameras in the mirror as virtual cameras, while all camera calibration refers to the calibration of virtual cameras.