Lane and Xi Zhou InvisibleEye : Fully Embedded Mobile Eye Tracking Using Appearance-Based Gaze Estimation

Human gaze has a long history as a means for fast, accurate, and natural interaction with both ambient [15] and body-worn displays, including smartwatches [3] and has, more recently, also been shown to be a rich source of information about the user [6]. Eye movements are closely linked to everyday human behavior and cognition and can therefore be used for computational user modeling, such as for eye-based recognition of daily activities [2], visual search targets [8], or personality traits [4] – including analyses over long periods of time for lifelogging applications [9]. Interest in gaze has been fuelled by recent technical advances and significant cost reductions of mobile eye trackers that can be worn in daily life and provide insights into users’ everyday gaze behavior [1]. Despite its appeal, mobile eye tracking suffers from several fundamental usability problems. First, current mobile eye trackers are still uncomfortable to wear, especially over long time periods: The required highquality imaging sensors are large and thus often occlude the user’s field of view, are heavy and cause discomfort or even pain. Second, current eye trackers limit users’ mobility given that they require a wired connection to a recording computer. Finally, their obtrusive design leads to low social acceptance and unnatural behavior of both the wearer and people they interact with [7], thus fundamentally limiting the practical usefulness of mobile eye tracking as a tool in the social and behavioral sciences. We argue that it is ultimately necessary to fully integrate eye tracking into regular glasses, i.e., to effectively make eye tracking visually and physically unnoticeable to both the wearer and bystanders. A key requirement for such unnoticeable (invisible) integration is to reduce the size of an eye tracker’s core component: the imaging sensors. Smaller sensors not only significantly reduce the device’s weight, but can also be positioned in the visual periphery to avoid occlusions within the users’ field of view. In addition, the low resolution common to these sensors generates significantly less data that could be processed on the device itself, stored, or transmitted wirelessly, thus removing the need for a separate recording device. Finally, the reduced computation required to process low-resolution images decreases the load on the processor and, as such, helps to extend the recording time beyond the current limit of only a few hours. The eye tracker we developed, InvisibleEye, can be fully embedded into a normal glasses frame (see Figure 1, bottom left). To achieve this, we took a radically different approach to mobile eye tracking. Instead of a single camera and model-based gaze estimation, InvisibleEye uses multiple, millimeter-size imaging sensors positioned around the eye as well as a computational method based on an artificial neural network for so-called appearance-based gaze estimation – estimating gaze of a specific user by automatically analyzing the eye images obtained from the cameras. Here we briefly Despite their potential for a range of exciting new applications, mobile eye trackers suffer from several fundamental usability problems. InvisibleEye is an innovative approach for mobile eye tracking that uses millimeter-size RGB cameras, which can be fully embedded into normal glasses frames, as well as appearancebased gaze estimation to directly estimate gaze from the eye images. Through evaluation on three large-scale, increasingly realistic datasets, we show that InvisibleEye can achieve a person-specific gaze estimation accuracy of up to 2.04° using three camera pairs with a resolution of only 3x3 pixels. Excerpted from “InvisibleEye: Mobile Eye Tracking Using Multiple Low-Resolution Cameras and Learning-Based Gaze Estimation” from Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies with permission. https://dl.acm. org/citation.cfm? id=3139486.3130971 © ACM 2017 [HIGHLIGHTS]