# Driving Fatigue Detection Based on the Combination of Multi-Branch 3D-CNN and Attention Mechanism
## 1 Introduction  
According to statistics from the National Bureau of Statistics of China, the global car ownership in 2021 is about 1.05 billion, of which 80% are sedans, and it is growing at a rate of 2-5% every year [1]. As the number of cars continues to grow rapidly, traffic accidents also occur more and more frequently. More than 95% of traffic accidents are related to the driver's behavior [2] and the driver's fatigue distracted behavior will affect his operating behavior, thereby causing traffic accidents. Therefore, the research and development of the detection system of driver's fatigue state has strong practical significance. The current mainstream driving fatigue detection methods are mainly divided into three categories: detection based on driver's physiological characteristics [3; 4; 5], detection based on motor vehicle behavior characteristics [6; 7] and detection based on driver's facial feature [8; 9; 10].The detection method based on physiological characteristics is to understand the driver's fatigue state and judge the driver's fatigue degree through the collection, recording, detection and analysis of various physiological signals reflectingdriving fatigue. Although this method has high detection accuracy, it greatly reduces the driving experience. The second fatigue monitoring method is to measure the behavior of the vehicle. This method is mainly to monitor the changes of the data during the driving of the vehicle, such as speed, steering wheel rotation angle and lane departure detection, brake and accelerator pressure changes, etc., to indirectly evaluate the driver's fatigue through these data. This method is difficult to accurately judge the driver's mild fatigue. With the great success of deep neural networks in recent years for tasks such as speech recognition [11] and image classification [12], the rapid rise of neural network-based deep learning has provided new ideas for the development of fatigue driving systems. [PERSON] et al. [9] and [PERSON] et al. [10] have used methods that basically just extract features from the spatial dimension through a 2D convolutional operation kernel, but the limitation of this method is that none of them take into account the continuous inter-frame motion information. In a real driving environment, fatigue is usually a continuous state and extracting features in the spatial dimension without considering the temporal dimension will significantly compromise the detection results. The 3D convolution kernel [13] can extract temporal and spatial information in video data, so it can capture more motion information between video frames. After being proposed, 3D convolution is widely used in behavior recognition [14; 15] and medical imaging [16; 17], but is rarely used in fatigue detection. This paper proposes a fatigue detection model based on a 3D convolution kernel, which includes a face localization and tracking module, a 3D-CNN feature extraction module, a feature recalibration SE module and a fatigue state assessment module. The main contributions of this paper are as follows:  
1. In this study, 3D convolution and attention mechanism modules are used to extract features in both temporal and spatial dimensions and to recalibrate the fatigue features to improve the accuracy and robustness of the model. 2. In this study, we conducted multiple sets of experiments on 3D-SE-Net to evaluate the detection effect of the model, including: comparative experiments with and without attention mechanism, comparative experiments for three-branch and double-branch networks, comparison experiments for different hyperparameters, comparison experiments for with and without frame aggregation strategy and comparison experiments for the proposed model with the state-of-the-art model on the same dataset. 3. In this study, a dataset was collected in a real driving environment with a rich set of driving scenarios including day and night, with and without glasses and male and female, which can be used for subsequent in-depth studies on fatigue driving detection.