# Exploring the Potential of Event Camera Imaging for Advancing Remote Pupil-Tracking Techniques
## 2 Proposed Method
### Pupil-Tracking Algorithm  
In our research, the creation of event frames is an important step. By accumulating events over a 33 ms interval, we translate this information into a visual format familiar to standard frame-based systems. The flowchart in Figure 4 starts with data from the event camera. These data are collected over the 33 ms period to form the event frame. Once the frame accumulates events for the set 33 ms interval, it is passed on to the next stage for processing. After forming the event frame, our method first identifies and localizes the eye-nose region within the frame. This detection step utilizes cascaded Adaboost classifiers with multi-block local binary patterns (LBPs) for the robust and efficient recognition of the eye region. Upon successful detection of the eye-nose region, our system activates the SDM-based eye-nose shape alignment. The next tracking checker block ensures that our tracking is maintained over consecutive frames and verifies the validity of the tracking. For the tracking checker, we utilize Scale-Invariant Feature Transform (SIFT) features extracted around 11 landmarks on bare faces and use a support vector machine (SVM) to ascertain the success of tracking. When the eye tracking fails, it restarts the detection mode, which scans the whole image to find the eye-nose area with a relatively lower speed (16 ms), compared to the tracking mode (5 ms), which utilized a small region of interest from the eye tracking success in the previous camera frame. Therefore, it is desirable to maintain the tracking mode at each frame without the execution of the detection module, in terms of the overall system latency. The above processes yield the final eye coordinates, allowing our system to effectively and efficiently track pupils in dynamic environments. The entire process, from event frame formulation to the extraction of the final eye coordinates, is illustrated in Figure 4.  
We are expanding on the success of our previously developed eye-tracking method [4; 11], which demonstrated effectiveness in diverse environmental settings and with different users. The foundation of our previous approach relied on 11-point eye-nose shape tracking, employing SIFT features [16], the SDM [14]. A concise list of the main features we employed includes the following: (1) 11-point eye-nose shape tracking--this technique selects the most significant landmark points within the entire facial structure, chosen specifically for their role in enhancing the accuracy of eye alignment, (2) SDM regression with the SIFT feature--optimized for central processing unit (CPU) efficiency; SDM uses a 4-stage regression to transition from an average to an optimal facial shape. SDM focuses on a series of descent directions to minimize non-linear square functions of landmarks. This regression-based approach not only reduces computational cost but also enhances the alignment accuracy, presenting shape alignment as an optimization task. We efficiently regress the initial pupil positions from the detected eye-nose regions to their optimal pupil center positions, along with other eye-nose landmark points, enabling swift and accurate tracking. Given the demands of real-time applications and the challenges posed by various light conditions, eye occlusion, head movements, and limited computing resources, we opted for a comprehensive and efficient machine-learning-based computer vision approach. This approach includes utilizing the speed advantage of the SDM [14] and cascaded Adaboost classifiers [17; 18] with multiblock LBP [19] for eye-nose region detection, offering a robust alternative to more computationally intensive methods. Furthermore, our previous studies [4; 11] have demonstrated the substantial speed advantage of the SDM-based eye tracking method over various convolutional neural-network (CNN)-based algorithms.