Where Am I Looking: Localizing Gaze In Reconstructed 3D Space We propose a method to estimate the 3D gaze of the observer onto the scene using a portable eye tracker with a monocular camera. We reconstruct the 3D scene using Structure from Motion (SfM) and use camera pose and 3D reconstruction information of the scene to localize the position and pose of the observer in the 3D reconstructed space. Along with the position, we can obtain the 2D gaze of the observer using an eyetracker. Each person may have a different perspective of the same 3D object in the scene, observing it from different positions. We use this information to fuse these multiple perspectives in 3D space to get a better understanding of how differently each observer perceives the same scene, compared to others. In the entire system, we developed a convolutional neural network to detect and track eye movement and a camera relocalization method to localize the camera in 3D environment. Based on our novel eye tracking and camera relocalization methods, we can accurately localize the gaze in the 3D reconstructed environment.

Regionwise Polynomial Regression for 3D Mobile Gaze Estimation In the context of mobile gaze tracking techniques, a 3D gaze point can be calculated as the middle point between two 3D visual axes. To infer gaze directions and eyeball positions, a nonlinear optimization problem is typically formulated to minimize the angular disparities between the training gaze directions and prediction ones. Nonetheless, the experimental results reported by some previous works show that this kind of approaches are very likely to yield large prediction errors hence considered less useful for humanmachine interactions. In this study, we aim to address this widespread issue in three aspects. At first, instead of using a global regression model, a simple local polynomial model is proposed to backproject a pupil center onto its corresponding visual axis. Based on the LeaveOneOut crossvalidation criterion, the partition structure is automatically learned in the process of resolving a homographylike relationship. Secondly, a good starting point for nonlinearoptimization is obtained by the image eyeball center, which can be estimated by systematic parallax errors. Meanwhile, it is necessary to add the suitable constraints for 3D eye positions. Otherwise, the optimization may end up with trivial solutions, i.e., faraway eye positions. Thirdly, we explore a strategy for designing the spatial distribution of calibration points in a principled manner. The experiment results demonstrate that an encouraging gaze estimation accuracy can be achieved by our proposed framework for both the normal vision and eyewear users.