# Autostereoscopic 3D Display System for 3D Medical Images
## 2 Methods
### Eye Tracking  
The 3D image quality and 3D crosstalk of the proposed 3D medical display system depend on the precision and speed of the eye tracker [17]. With faster and more accurate eye center position tracking, 3D images with higher 3D resolution and lower 3D crosstalk can be displayed in real-time in our autostereoscopic 3D display system. The development of accurate and fast eye tracking systems using RGB cameras remains a challenge due to various light conditions including low light in medical investigation rooms, eye occlusion, eyeglasses reflection, fast head movements, head poses, and limited system computing resources. Figure 4: A horizontal 2D slice of light field in the 3D display. (**a**) Real-world setting, (**b**) equivalent representation in the ray space. Greyscale lines represent light rays passing through slits (lens), red and blue lines represent light rays passing through the left and right eye respectively. Square boxes are intersected points. Figure 3: Parameterization of a light ray in the 3D display environment. Our real-time machine learning-based eye tracking systems aims to provide fast, accurate, robust, and small model sized methods for medical display commercial use. Our proposed eye tracking system classifies human faces into faces with clear eyes and eye-occluded faces such as thick eyeglasses with reflection, wearing hats, sunglasses, and hair occlusion. Depending on user eye shape clearness conditions, corresponding eye landmark point detection methods are applied based on image classification. The eye tracking system switches model designs between two tracking modes. Each eye tracking mode has different machine learning models, and the basic components can be divided into three main stages: (1) eye-nose region detection from RGB webcam images, (2) eye center position tracking from the detected eye-nose region with eye position refinement, and (3) tracker-checker for fast tracking (Figure 5). Our eye-nose region detection was developed by the error-based learning (EBL) method [25], where the cascaded Adaboost classifier [36] with local binary features (LBP) [37] is simple and practical with the use of a conventional CPU in a PC or mobile tablets with limited GPU resources. The EBL method trains only a small part of training face image DBs with a large size. The EBL offers higher accuracy in much shorter training times through multiple stages to find the most difficult training image DB subsets. If eye-nose region detection is successful, the eye center position tracking mode runs depending on the user eye image condition classification, until tracking mode fails with the tracker-checker module. For clear eye faces, a coarse-to-fine strategy was developed to infer pupil center location using the Supervised Descent Method (SDM) [38] with Scale-Invariant Feature Transform (SIFT) [39] with pupil position refinement by pupil segmentation module [26]. Specifically, 11 landmark points encompassing the eyes, nose, and mouth are used to perform the SDM-based shape alignments [22; 23]. The pupil segmentation method was developed as an iris regression module, where pupil boundaries are regressed to refine pupil center positions [25; 26]. For eye occluded faces due to wearing thick eyeglasses, sunglasses, and hair occlusion, we adopted a Convolutional Neural Network (CNN)-based facial keypoint alignment method, the Practical Facial Landmark Detector (PFLD) network [40], which uses MobileNet v2 [41] to improve the alignment speed when using only CPU resources. We revised the PFLD network by adding a re-weight network module to handle eye occlusion problems [26; 27]. To infer the eye pupil position, we utilized 98 whole face feature points, which can handle the eye occlusion cases. While the original PFLD method has the same importance in whole face points, our revised PFLD method with the re-weight sub-network  
Figure 5: Illustration of the proposed eye tracking method, which includes pupil segmentation modules.