# FF-YOLO: An Improved YOLO11-Based Fatigue Detection Algorithm for Air Traffic Controllers
## 3 Materials and Methods
### Dataset Construction
#### 3.1.2 Dataset Labeling
For the objective dimension of fatigue assessment, the PERCLOS metric, which is based on the persistence of eyelid closure, was adopted as the standard for judging fatigue. PERCLOS is an effective psychophysiological indicator that assesses an individual's alertness by calculating the proportion of time the eyelids are closed [27], and it holds significant application value in fields such as driver fatigue detection. This metric quantifies fatigue by calculating the percentage of time within a specific period that the eyelids are closed beyond a certain proportion. A higher proportion indicates a greater degree of fatigue in the individual. Compared to traditional methods that count the number of blinks within a fixed time, PERCLOS reflects slow eyelid drooping actions rather than rapid blinking behaviors, and this slow drooping is more indicative of fatigue states; thus, it was chosen as the primary basis for annotating fatigue states. To calculate PERCLOS, it is first necessary to determine the shape of the eyes. This paper employs the method from the Dlib library to identify the key points of the ATCO's eyes based on facial images. Dlib is an open-source machine learning library that provides researchers with complete documentation and corresponding debugging tools for using artificial intelligence technology, and it has wide applications in the fields of computer vision and image processing [28]. This paper utilizes the facial detection tool provided by Dlib by calling its functions to obtain a facial detector. Next, a pre-trained shape_predictor_68_face_landmarks model is loaded, which can accurately locate 68 key points on the face. As shown in Figure 4a, among the 68 key points, six key points related to the eyes are used to calculate the degree of eye closure, thereby inferring the individual's fatigue level. It should be noted that wearing glasses may impact the extraction of key eyelid landmarks. As shown in Figure 4b, this impact is typically minimal when an ATCO wearing glasses views the screen frontally. However, when the ATCO's head is turned, occlusion by the frame and light reflections from the lenses may interfere with the accuracy of PERCLOS calculation during specific periods. Given that fatigue is an inherently continuous physiological process rather than an instantaneous event, the PERCLOS metric is designed to compute the proportion of eyelid closure over an extended sampling window. At this temporal scale, transient measurement errors caused by intermittent factors like occlusion and reflections exert limited influence on the overall result. Therefore, despite the potential for measurement bias introduced by eyewear, the PERCLOS-based objective annotation results employed in this study remain effective for characterizing fatigue states over extended time scales. The fatigue labeling method that combines NASA-TLX subjective data for cross-validation and fusion further reduces the impact of bias on the final fatigue judgment. The eye aspect ratio (EAR) is an indicator used to quantify the degree of eye closure. As shown in Figure 5, this paper uses 6 key points, such as the inner and outer corners of the eyes, the upper and lower eyelids, etc., to calculate the eye aspect ratio EAR. The formula for calculating EAR is given in Equation (2) [29]. \[EAR=\frac{||P_{2}-P_{6}||+||P_{3}-P_{5}||}{2\times||P_{1}-P_{4}||} \tag{2}\]  
In the formula, \(P_{1}\) to \(P_{6}\) represent the position coordinates of the six key points around the eyes, of which \(P_{1}\) and \(P_{4}\) are the points of the outer and inner corners of the eyes, and \(P_{2}\), \(P_{3}\), \(P_{5}\), and \(P_{6}\) are the points of the upper and lower contours of the eyes. By calculating the Euclidean distance between these points, the EAR value can be obtained. When the eyes are open, the EAR value is relatively stable; when the eyes are closed, the EAR value will gradually decrease and approach zero.