## 3 Proposed Approach  
### Driver Fatigue Detection  
For the detection of driver fatigue, we use the random forest algorithm because it performs well, has a fast training speed, and can process high-dimensional data. We first detect faces in the input images. Subsequently, we use the S3 FD [6] for detecting faces because it can provide high-quality performance for the different scales at different ROIs. Then, we use the FAN [7] to extract 68 facial feature points in the image to find features such as the eyes and mouth. This network uses a 2D representation of the face and the coordinates of the facial feature points as inputs, and it is then trained with four hourglass modules [30]. Hence, upsampling and downsampling can be used to obtain the information of each image size, reduce the loss of image information, and finally acquire a heatmap. This heatmap can be used to predict the position of each facial feature point in the image. The advantage of using this network is that it can detect larger or unusual face poses, and is thus generally more effective than Dlib or the cascaded regression method [31]. For closed eye and eye blink detection, six feature points of each eye are used to define the eye aspect ratio (_EAR_) [32], which is calculated using the length and width of the eye as follows:  
\[EAR=\frac{||P_{2}-P_{6}||+||P_{3}-P_{5}||}{2(||P_{1}-P_{4}||)} \tag{1}\]  
where \(P_{i}\), for \(i\) = 1, 2,..., 6, are the feature points. A closed eye is defined as \(EAR\) < 0.15 for a given period. Eye blinking behavior is indicated by the frequent fluctuation of the EAR. For the detection of eye gaze direction, the region of the eye extracted with the six feature points is first converted into a grayscale eye image. The grayscale eye image is then processed with blurring and erosion to eliminate the reflected light, and is finally binarized to derive the enclosing contour and centroid. The eye gaze direction is expressed in terms of the horizontal and vertical directions. Figure 2 illustrates the results from the processing of an eye region extracted with the six feature points. Figure 2: Processing results for an eye region. (**a**) grayscale eye image, (**b**) blurred eye image, (**c**) eroded eye image, and (**d**) binarized eye image. Yawning is such that the mouth is open for a fairly long period. Thus, we use the mouth aspect ratio (_MAR_), similar to the _EAR_, to detect yawning. The _MAR_ is defined as follows:  
\[MAR=\frac{||P_{2}-P_{8}||+||P_{3}-P_{7}||+||P_{4}-P_{6}||}{2(||P_{1}-P_{5}||)} \tag{2}\]  
where \(P_{i}\), for \(i=1\), 2,..., 8, are the eight feature points representing the mouth [33]. A single yawn is indicated by a _MAR_ that exceeds a given threshold for a given period (i.e., if the mouth remains open for too long). Figure 3 presents the flowchart for how yawns are detected. The purpose of calculating the head posture is to determine whether the driver is nodding. To determine the orientation of the head, we first establish 14 correspondences between the 2D facial feature points and the 3D face model. The identified 2D image features are then mapped onto the 3D model to derive the rotational changes of the head. For driver fatigue detection, six fatigue parameters are defined: (a) PERCLOS, the percentage of eye closure over time; (b) Blink frequency, the blink frequency in a period of time; (c) Maximum close duration (MCD), the longest eye closure in a period of time; (d) NodFreq, the frequency of nodding over time; (e) YawnFreq, the frequency of yawning in a period of time; and (f) GazeDir, the gaze direction. We use the YawDD data set for training and testing [34]. This data set contains 322 male and female drivers with or without glasses (or sunglasses). The camera is placed under the front mirror of the vehicle. Three or four video clips are recorded for each participant. Each video shows a different mouth condition, such as normal talking, singing, and yawning. The persons with talking and yawning behaviors are considered as sober and drowsy drivers, respectively.