# Convolutional Two-Stream Network Using Multi-Facial Feature Fusion for Driver Fatigue Detection
## 1 Introduction  
According to the National Highway Traffic Safety Administration report, 22% to 24% of traffic accidents are caused by driver fatigue. Driver fatigue during driving can increase the risk of accidents by four to six times. Frequent occurrence of traffic accidents seriously threatens the safety of people's life and property. Therefore, the study of driver fatigue detection is of great significance. Research shows that fatigue is closely related to psychophysiological changes, such as blink rate, heart rate, anxiety, etc. [1]. Nowadays, there are various techniques to measure driver fatigue. These techniques can be generally classified into three categories: vehicle-focused, driver-focused, and computer vision-based methods. Driver-focused methods focus on psychophysiological parameters such as using electroencephalogram (EEG) data [2; 3; 4], which would be an intrusive mechanism for detecting driver status. Vehicle-focused methods detect the running condition of the vehicle and the status of the steering wheel [5], which have specific limitation factors such as highway driving. [PERSON] [6] combined vehicle-focused and driver-focused methods, measuring physiological and behavioral indicators to analyze and prevent accidents. Because of the rapid development of deep learning, driver fatigue detection has been an active research topic in the field of computer vision in recent years. In driver fatigue detection based on computer vision, some researchers focus on the driver's mouth movement [7; 8], while others study the relation between fatigue and eye movement [9; 10; 11; 12]. [PERSON] et al. calculated the blink rate as a basis for judging driving [13]. [PERSON] & [PERSON] used support vector machines to classify normal and yawning mouths [14]. [PERSON] et al. [15] combined multiplevisual cues to get a more robust and accurate model, which included eyelid movement, gaze movement, head movement, and facial expression. Although an EEG is highly correlated with the driver's mental state and is most sensitive to fatigue detection, it has a large amount of redundant information, which will affect the efficiency and accuracy of detection. In addition, it also requires drivers to wear related devices, which is an intrusive mechanism for driver. Driver fatigue detection based on the vehicle driving mode is greatly influenced by external factors and the driver's own driving habits; therefore, the detection accuracy is not so high. The driver fatigue detection method based on computer vision not only has high accuracy, but also has no impact on the driver's driving. Therefore, it is more applicable. With the prevalence of convolutional neural networks (CNNs), more and more driver fatigue monitoring algorithms have used convolutional neural networks as underlying algorithms, and more and more variants have been invented. [PERSON] et al. [16] presented the Driver Drowsiness Detection (DDD) network. It integrated the results of three existing networks by support vector machine (SVM) to classify the categories of videos, which cannot monitor driver drowsiness online. Three-dimensional convolutional neural networks (3D-CNNs) were applied to extract spatial and temporal information by [PERSON] et al. [17], but the approach took advantage of global face images, which did not have the flexibility to configure patches that contained most of the drowsiness information. [PERSON] [18] also used global facial information to detect driver fatigue. [PERSON] [19] proposed a sequence-level classification model that was able to simultaneously estimate the status of the driver's eyes, mouth, head, and drowsiness. Long-term multi-granularity deep framework [20] combined long short-term memory network (LSTM) [21] and CNN with multi-granularity; however, this method did not contain dynamic information that behaved in the time dimension.