###### Abstract  
Fatigue affects operators' safe operation in a nuclear power plant's (NPP) main control room (MCR). An accurate and rapid detection of operators' fatigue status is significant to safe operation. The purpose of the study is to explore a way to detect operator fatigue using trends in eyes' blink rate, number of frames closed in a specified time (PERCLOS) and mouse velocity changes of operators. In experimental tasks of simulating operations, the clustering method of Toeplitz Inverse Covariance-Based Clustering (TICC) is used for the relevant data captured by non-invasive techniques to determine fatigue levels. Based on the determined results, the data samples are given labeled fatigue levels. Then, the data of fatigue samples with different levels are identified using supervised learning techniques. Supervised learning is used to classify different fatigue levels of operators. According to the supervised learning algorithm in different time windows (20 s-60 s), different time steps (10 s-50 s) and different feature sets (eye, mouse, eye-plus-mouse) classification performance show that K-Nearest Neighbor (KNN) perform the best in the combination of the above multiple indexes. It has an accuracy rate of 91.83%. The proposed technique can detect operators' fatigue level in real time within 10 s.  
fatigue; nuclear power plant main control room; eyes' blink rate; PERCLOS; mouse velocity; clustering; supervised learning +
Footnote †: journal:  
velocity (AMV) and average value of mouse velocity (AOV) features in a non-invasive detection way to avoid interfering with the operator's normal work. On the one hand, these four indicators can be acquired in a non-invasive way. This approach avoids interfering with the operator's normal work; on the other hand, the combination of using the camera to capture changes in the eye, and to augment it with changes in the velocity of the mouse, and then combining these two types of feature sets, shows a significant improvement in fatigue recognition compared to using a single eye feature. Blink rate and PERCLOS features are used to analyze and detect fatigue in automobile driving [11], aircraft driving [12] and human-computer interaction [13]. Eye aspect ratio (EAR) has been successfully used in operator fatigue detection in automated control systems [14; 15]. First, the EAR values are obtained to generate the features associated with the eyes. Then, these features are used for supervised learning [11]. Using convolutional neural network, after a large number of eye opening and closing pictures are trained, a model for detecting eye opening and closing is obtained. Then, fatigue is determined according to predetermined rules and algorithms [16]. At present, when using eye-related features for non-invasive detection, the methods to obtain the eyelid closure state are roughly the two mentioned above. One is to calculate the eye-width ratio using facial landmark and determine the eye opening and closing by a predetermined threshold. The other uses a trained neural network to judge the opening and closing of the eyes directly from images. Then, the features are extracted again based on the opening and closing state of the eyes, such as blink rate or PERCLOS. Finally, the fatigue detection is implemented using the recognition fatigue algorithm, or blink rate and PERCLOS threshold size are used to determine fatigue according to predetermined rules. The above two single feature set methods are affected by facial images, lighting, motion blur or head deflection during the task. Mental fatigue is a complex and individualized phenomenon that affects psychological, physiological and behavioral aspects [17]. Determining fatigue using a single indicator is unreliable. Few studies have been conducted to detect the fatigue status of operators in the main control room of a nuclear power plant based on the fusion of multiple indicators by machine learning [18]. In [19], mouse, facial features and eye gaze are used to recognize attention.