# Driver Fatigue Detection Based on Residual Channel Attention Network and Head Pose Estimation
## 3 Materials and Methods
### Facial State Recognition  
Retinaface has good robustness in complex situations and can accurately output the landmarks of the face: the left and right mouth corners, the center of the nose, and the centers of the left and right eyes. Retinaface performs pixel-wise face localization on faces of various scales by taking advantages of joint extra-supervised and self-supervised multi-task learning. Therefore, Retinaface can also accurately locate the five landmarks of human face when the camera is far away from the driver. Figure 2 depicts the results of face detection using Retinaface. According to the five landmarks given by Retinaface, the eye region and the mouth region can be obtained. Section 3.2 shows how to obtain these regions. After obtaining the regions of eyes and the mouth, these regions will be uniformly scaled to 56 \(\times\) 56 and input to the RCAN to judge the state. In this study, the RCAN is proposed to extract the features of the eyes and mouth. The RCAN is composed of three residual channel attention blocks (RCABs) that have the same structure in series, in which channel attention module is integrated. EfficientNet [24] and MobileNet [25] also have a similar structure. In the block of the RCAN, we uniformly use a 3 \(\times\) 3 convolution kernel to extract features and stack multiple 3 \(\times\) 3 convolution kernels to expand the receptive  
Figure 1: The overall structure of driver fatigue state detection. Figure 2: Detection results of Retinaface. field in every RCAB. The RCAN finally achieves the last feature layer, the size of which is \(7\times 7\). Then this layer will be sent to the fully connected layer for feature aggregation. The complete structure of the RCAN is shown in Table 1.  
In an RCAB, the channel attention module is used for feature re-extraction of the current feature layer. As every channel of a feature map is regarded as a feature detector [26], the channel attention module focuses on what is meaningful given a feature map. The significance of the channel attention module is to suppress a useless channel of the feature map and enhance the role of the useful channel of the feature map. Therefore, the channel attention module can suppress the unnecessary background area, which can help the RCAN know \"what to look for\". Figure 3 shows the structure of an RCAB and its channel attention module. This module extracts information by squeezing each channel of a feature map so that the feature layer with stronger semantic information has a higher weight. The ways of squeezing include global average-pooling (GAP) and global max-pooling (GMP). Ref. [26] demonstrated that it is effective to use GAP or GMP for squeezing the feature layer. GAP and GMP have different representation abilities. GMP focuses on the most significant region in the image to compensate the global region that GAP focuses on. Therefore, GAP and GMP have different effects on the compression feature layer. However, the convolutional block attention module (CBAM) proposed in [26] can be regarded as adding GAP to GMP directly for feature extraction, which causes the channel attention module input to be unbalanced. For this reason, the proposed channel attention module multiplies GAP and GMP by a trainable weight in the input stage and then adds them and sends them to a multi-layer perceptron (MLP) with a hidden layer to extract information. A residual connection is included between the input and output of the channel attention module, which makes it easy to converge when training the RCAN. The specific operations of the proposed channel attention module are as follows: Set \(\text{M}_{\text{input}}\) as the input of the channel attention module in the current RCAB. C is the total channels of \(\text{M}_{\text{input}}\). First, \(\text{M}_{\text{input}}\) is squeezed into the GAP layer \(\text{F}_{\text{gap}}\in\text{R}^{\text{C}\times 1\times 1}\) and the GMP layer \(\text{F}_{\text{gmp}}\in\text{R}^{\text{C}\times 1\times 1}\).