# Research on a Real-Time Driver Fatigue Detection Algorithm Based on Facial Video Sequences
## 1 Introduction  
Driver fatigue, or drowsiness, contributes to many thousands of deaths and injuries on the roads every year. It has a role in up to 30% of fatal crashes and up to 15% of serious injuries [1]. Driver fatigue occurs when a driver does not get enough sleep or after long-distance driving. The influence of fatigue on driving safety is similar to that of alcohol impairment. A survey of the U.S population found that 37% of workers got less than the recommended minimum of 7 h of sleep [2]. Driver fatigue is responsible for 22.7% of fatal accidents and 20.5% of accidents with injuries on Canadian roads. In 40% of fatigue-related accidents, the driver was awake for over 17 h [3]. The main problem with fatigue is that it is very difficult to be aware of the firchers of drivers before they become too tired to drive safely. Driver fatigue is a major cause of car accidents, since sleepy drivers are unable to make rapid decisions, and they may have slower reaction times [4]. Therefore, recent research also suggests that driver fatigue detection methodologies can be used to prevent such accidents [5]. The research on driver fatigue detection can be classified into two categories: passive driver monitoring system and active driver fatigue control system. Among them, the first is the use of preventive measures, by which a driver is monitored, and an alarm is displayed to advise the driver to stop the vehicle during the fatigue stage [5, 6, 7]. The second is the abnormal fatigue detection in which a driver is deemed to be asleep and the system can actively control the vehicle to avoid accidents [8, 9, 10]. In recent years, researchers have developed methodologies to detect or indicate the driver fatigue state prior to a collision [11; 12; 13]. Generally, data from three different sources have been used to classify driver fatigue or drowsiness. Firstly, vehicle-based measures are used to detect a drowsy or unsafe state. For example, vehicle-based measuring may be obtained by steering wheel angle, acceleration pedal data, and lane departure information by an external sensor [14; 15]. Secondly, observation of the driver behavior method may be determined by continuous recording through a camera installed in the vehicle to monitor eye-closure time, eye blinking frequency, movement and pose of the head, and yawing, etc. [16; 17; 18]. The most recognized measure of these models is the percentage of eye closure time (PERCLOS) [19], although others have used measures such as eye aspect ratio (EAR) [20]. Thirdly, driver-based measures include physiological measurements such as electroencephalography (ECG), as well as heart rate, respiration rate, pulse pattern, and popular cortical signals, etc. [21; 22; 23]. In summary, owing to easy installation and low cost, observation of the driver behavior method has been widely used for fatigue detection. For example, attention technologies and smart eye technology employ the movement of the driver's eyes and position of the driver's head to determine the level of their fatigue [24; 25]. In this study, we propose a real-time observation of the driver behavior method to detect driver fatigue. Our method only uses the inside vehicle camera, which is handy for the driver to carry on, or convenient to install in the vehicle. The novelty of this study is to develop a novel real-time driver fatigue detection system that has two major advantages. First, we introduce a new face-tracking algorithm based on facial landmarks to improve detection accuracy. A tasks-constrained deep convolutional network is constructed to detect the face region based on 68 key points, which solve the optimization problem caused by different convergence speeds of each task. Second, we use K-nearest neighbor (KNN) to classify the state of the driver's eye. According to the real-time facial video images, the eye feature of eye aspect ratio (EAR), mouth aspect ratio (MAR), and percentage of eye closure time (PERCLOS) are calculated based on specific facial landmarks.