# Research on a Real-Time Driver Fatigue Detection Algorithm Based on Facial Video Sequences
## 2 Approach
### Tasks-Constrained Deep Convolution Network  
The framework of TCDCN algorithm is shown as Figure 2. The input is a 40 \(\times\) 40 gray image. The feature extraction stage includes four convolution layers, three pooling layers, and one full connection layer. The weight of the filter is not shared in space, which means that different filter sets are used in the input map. The hyperbolic tangent of the absolute value rectification is selected as the activation function; the maximum pooling is carried out in the area without overlap, and an eigenvector is generated with the full connection layer after four convolutions. This eigenvector is shared by multiple tasks in the estimation stage, among which linear regression is used to obtain the location of the landmark points and multiple logistic regression is used to handle multiple other personal tasks, for example, gender, face position, wearing glasses, etc.  
Figure 1: Schematic diagram of our approach.  
Generally speaking, when people look at the face in the picture, they will detect where the face is, and then identify the gender, face position, if the driver is wearing glasses, and so on. When machines perform these tasks, they usually need to design independent algorithms to solve different tasks. However, TCDCN can design a sophisticated network to complete all these tasks at the same time, and make use of the relationship between tasks.  
The difficulty of multi-task learning is that different tasks have different characteristics and convergence speed. The method proposed in this study is superior to the existing method, especially in dealing with the situation of serious occlusion and position change, and reduces the complexity of the model.  
In order to make use of all the information on multi-task data sets, such as face frame, face landmark points, face position, gender, and wearing glasses information, multiple sub networks can train the task-related data sets, and then share the parameters, because no single data set contains all the annotation information required by the face detection task. Through this method, TCDCN can use and adapt parameter sharing to the whole domain, rather than to fit the specific task domain.  
In the training phase of TCDCN, the multi-task facial landmark (MTFL) dataset [24] is used as the training data. This dataset contains 12,995 face images collected from the Internet. The images are annotated with (1) five facial landmarks and (2) attributes of gender, wearing glasses, and head pose.  
The procedure of the training of the TCDCN aims to obtain the optimal model by adjusting the parameters dynamically. To express the influence of different parameters on the performance of the TCDCN, a loss function is introduced during training. In this paper, the least square method is used as the loss function for the main task and the cross-entropy loss function for the auxiliary task. The loss function of the main task is shown as the following:  
\[L(y,f(x))=\sum_{i=1}^{N}{(y_{i}-f(x_{i}))^{2}} \tag{1}\]  
where, \(L\) is the loss function, \(i\) represents the \(i\)th sample, \(N\) represents the total number of samples, \(y\) is the predicted value, and \(f(x)\) is the true value.  
The cross entropy loss function of auxiliary task (gender, face position, and wearing glasses) is shown as the following:  
\[L(Y,p(Y|X))=-\log{p(Y|X)} \tag{2}\]  
where, the loss function \(L\), means that the probability \(p(Y|X)\) of sample \(X\) reaches the maximum value under the condition of classification \(Y\).  
Through the training of the TCDCN, the face of the driver can be obtained accurately, which provides a stable face image for the following algorithms.