# FF-YOLO: An Improved YOLO11-Based Fatigue Detection Algorithm for Air Traffic Controllers
## 3 Materials and Methods  
### Dataset Construction  
#### 3.1.1 Data Collection  
This study was conducted on the radar control simulator at the Key Laboratory of Flight Techniques and Flight Safety CAAC, where a dataset of facial images of ATCOs under working conditions was constructed. The experiment collected facial image data from 10 radar ATCOs. Considering the complex lighting variations in actual air traffic control (ATC) work environments due to natural light changes at different times of day and dynamic interference from indoor equipment and screen reflections, this experiment included data collection of ATCO facial images under different lighting conditions at various times of the day. As shown in Table 1, the experiment adopted three time periods--morning, afternoon, and evening--for data collection to record the impact of natural light changes on the facial images of ATCOs. The design of data collection at different time periods took into account the effects of indoor and outdoor light source transitions on pupil state, facial reflections, and shadow distribution, thereby enhancing the generalization capability of the dataset. During the experiment, the subjects were required to perform ATC tasks involving large, medium, and small traffic volumes, facing various complex flight situations, such as multiple large, medium, and small aircraft entering the control area simultaneously, and flight adjustments under adverse weather conditions. For safety considerations, these scenarios were all conducted using the radar control simulator system. The experiment arranged for staff to act as the pilot-in-command, responsible for simulating air-ground communications and command confirmations with the subject ATCOs, to recreate the real working environment. The data collection environment is shown in Figure 1. Before the experiment began, all subjects underwent unified training to ensure they were familiar with the experimental procedures and equipment operations. The experiment used air traffic radar control tasks to induce fatigue. \begin{table}
\begin{tabular}{c c} \hline \hline
**Shifts** & **Time Periods** \\ \hline Morning & 8:30–11:30 \\ Afternoon & 14:30–17:30 \\ Evening & 19:30–21:00 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Specific time schedule of each shift. The experiment first collected facial video data of the subjects during the control tasks, then split the video frames at a rate of 1 frame per second, and after cropping, obtained facial image data of ATCOs with a resolution of 900 \(\times\) 900, totaling 53,756 images, all stored in JPEG format. Figure 2 displays some of the facial images of ATCOs collected in the experiment. It can be seen from the figure that the experiment captured facial images of ATCOs under various conditions, including occlusion by glasses, hand occlusion, head tilt, and different ambient lighting at various times. After each task, the subjects were required to fill out the NASA-TLX questionnaire. This questionnaire serves as a subjective dimension for assessing fatigue, used to evaluate the workload of individuals during task execution, where higher NASA-TLX scores are correlated with higher levels of fatigue [25]. The NASA-TLX questionnaire used in the experiment is shown in Figure 3. After each air traffic control task, the subjects would receive a pop-up window prompting them to complete the questionnaire. The questionnaire includes six dimensions: mental demand, physical demand, temporal demand, performance, effort, and frustration level. Each dimension has a slidable slider, and the subjects score each indicator by dragging the slider. The score range is from 1 to 10, where 1 indicates \"very low\" and 10 indicates \"very high\". Figure 1: ATCO facial data collection environment. Figure 2: Extracted facial images of ATCOs. The NASA-TLX scale employs six dimensions to assess psychological workload: Mental Demand, Physical Demand, Temporal Demand, Performance, Effort, and Frustration Level.