# Dual-Branch Multimodal Fusion Network for Driver Facial Emotion Recognition
## 2 Related Work
### Facial Emotion Recognition  
In the research field of facial expression recognition, the first step involves facial detection, followed by classification and recognition of expressions based on it. Regarding facial detection techniques, there are traditional feature-based methods [29; 30; 31] and the currently widely used deep learning-based approaches [32; 33]. Traditional methods require subsequent processing after detecting facial information, including facial feature alignment [34]. Due to the possibility of differences in scale, rotation, or pose of facial images after facial feature alignment, it is necessary to normalize facial images [35; 36]. After alignment and normalization processing of facial images, feature extraction is carried out using feature extraction networks. Facial feature extraction primarily involves two main aspects: facial contour extraction [37] and facial landmark extraction [38]. Facial contour features contain the overall shape information of the face, reflecting its overall expression state [39]. They offer crucial visual information for emotion recognition. Facial landmark extraction involves locating a set of key points in a facial image. The positions and changes of these key points can reveal microexpressions and facial expression alterations [40], offering more detailed information and foundational support for emotion recognition. Compared to traditional methods, deep learning methods have achieved end-to-end detection in face detection [41; 42]. [PERSON] et al. [43] emphasized the powerful capabilities of Convolutional Neural Networks (CNN) in image classification and feature extraction in their research. They pointed out that CNN can learn different features in images layer by layer, from simple edges and lines to complex image semantics. This ability stems from its hierarchical feature learning structure, which gradually extracts key information by stacking multiple convolutional layers. Meanwhile, [PERSON] et al. [44] explored the application of deep neural networks (DNNs) in feature learning, emphasizing the importance of effective loss functions to ensure high feature discrimination between different categories. They proposed that the weight vector of the last hidden layer in the network can be regarded as the geometric center of the category in the feature space. This study also proposed a specific symmetry and explored it through analysis and empirical research, aiming to address existing convergence issues. Simple convolutional networks have demonstrated efficient feature extraction capabilities, while deep learning methods rely on convolutional neural networks (CNNs) for feature extraction. Hence, a series of deep learning network frameworks such as Residual Network (Resnet) [45] and Mobile Neural Network (MobileNet) [46] have emerged. With the ongoing evolution of facial detection, deep learning has progressed towards innovative methodologies and broader applications in the field. [PERSON] et al. [47] implemented a medical IoT model that combines a single-shot multi-box detector (SSD) and a hybrid deep learning method. By optimizing parameters, they achieved the detection of facial masks and facial recognition. [PERSON] et al. [48] combined facial detection and recognition with drones to achieve shooting and detection recognition under various limitations of height, angle, and distance. These automatically extract features from raw data, avoiding subjective design features, and can provide a good feature supply for subsequent emotion recognition. Facial emotion recognition networks have demonstrated excellent performance on multiple public datasets such as CK+ [49], FER2013 [50], and SFEW [51]. In terms of driver facial emotion recognition, challenges such as capturing unique emotional features still exist. [PERSON] et al. [52] proposed a layered weighted random forest (WRF) algorithm, optimized in real-time for embedded applications with limited computing resources, enabling rapid driver face recognition. [PERSON] et al.