# Low-Complexity Pupil Tracking for Sunglasses-Wearing Faces for Glasses-Free 3D HUDs
## 1 Introduction  
Three-dimensional (3D) displays provide realistic visual experiences with an enhanced sense of image depth [1; 2]. Recent developments of holographic optical elements (HOE) technologies have increased the possibilities for commercialization of augmented reality (AR) devices including wearable AR glasses [3]. Additionally, autostereoscopic 3D displays offer the full benefits of the 3D experience without requiring the observer to wear 3D glasses. This is enabled via the eye-tracking-based autostereoscopic 3D method, which overcomes 3D viewing zone limitations, thereby allowing users a seamless 3D experience with higher 3D-resolved content [1; 2]. The eye-tracking-based autostereoscopic 3D method can be adopted in automobile head-up displays (HUDs) for drivers, which display realistic 3D navigation information about the road via combiners placed on the windshield [4; 5; 6]. Eye-tracking-based autostereoscopic 3D display systems require accurate and fast 3D measurements of the eye positions of the viewer to display 3D images with low 3D crosstalk and high 3D resolution. However, owing to the limited capabilities of automobile computing systems, recent deep-learning-based algorithms are not used because they typically require heavy graphical processing units (GPUs). Fast and accurate eye-gaze-tracking technologies are required not only in glasses-free 3D HUDs, but also in many AR display systems [7]. Other applications such as antispoofing in face recognition [8] and interaction systems with virtual contents [9] also utilize eye-gaze-tracking technologies. To date, many studies of eye tracking have focused on eye-gaze-tracking for wearable devices such as head-mounted devices. These methods calculate thevector between the pupil center and the corneal reflection using near-infrared (NIR) light sources to estimate the viewer's looking direction. In most methods, the NIR camera and light sources must be placed in specific locations to achieve clear bright pupil and corneal reflection images [10; 11; 12; 13; 14; 15; 16]. Remote eye-tracking methods have adopted recent computer vision techniques to detect and track viewers' eyes at greater distances for various consumer electronics such as 3D televisions and monitors, gaming devices, smartphones, driver monitoring systems, and HUDs in automobiles [17; 18; 19]. These methods usually adopt NIR light sources to capture high-quality eye images under various light conditions [20; 21; 22]. The goal of gaze tracking is to estimate the viewer's gaze direction, while our eye position tracking aims to detect and track 3D eye positions using red-green-blue (RGB) web-cameras at remote distances. In previous studies, we published real-time computation-based eye-tracking methods for bare faces [23; 24]. This accurate and fast pupil position tracking provided clear 3D images with high 3D resolution using limited embedded system resources, even when the head movements of the users were considered in real time. However, localization and tracking of the real-time pupil position of a driver can be affected by various occlusion factors, such as the driver wearing a mask, cap, or sunglasses. If the eye-tracking precision error is larger than the 3D margin of the HUD, users can see the 3D crosstalk and will feel 3D fatigue. As the 3D crosstalk margins of our AR 3D HUD prototype are 12 mm [2], the eye-tracking precision must be less than this threshold to allow users to enjoy 3D-crosstalk-free content (Figure 1). Furthermore, a user will experience 3D crosstalk when they move their head if the overall system latency, including eye tracking and 3D rendering, is high. Complex state-of-the-art deep-learning-based algorithms for detecting facial features [25; 26; 27; 28; 29] are very accurate, but incompatible with 3D autostereoscopic display systems, which have limited GPU capacity. Figure 1: Eye-tracking-based 3D light field rendering results based on an input of a red right image and a blue left image.