# Lightweight Facial Expression Recognition Based on Class-Rebalancing Fusion Cumulative Learning
## 1 Introduction  
Facial expressions, as expressions of emotions, play a major role in interpersonal communication. FER based on lightweight networks is very important for the implementation of human-computer interaction technology. With the continuous development of FER technology, FER is popularly applied in autonomous driving, criminal investigation, medical diagnosis, psychological assessment, and auxiliary classroom teaching and other fields [1].  
Recently, FER methods of deep learning based on the convolutional neural network (CNN) have gained significant achievements [2, 3]. Better results have been achieved for FER on the facial expression datasets CK+ [4], JAFFE [5], and so on taken under controlled conditions (laboratory, no complex background, etc.). However, it is difficult for natural expression datasets affected by noise, lighting changes, posture, and occlusion such as FER2013 [6] and RAF-DB [7] to achieve the expected effect in FER, and scholars have studied this accordingly. [PERSON] et al. [8] showed an attention network based on facial regions, evaluated different region generation strategies, and adaptively integrated weighted features from regions and the entire face through the attention module, which significantly improved the performance of the network under occlusion and complex pose conditions. [PERSON] et al. [9] showed a method to apply depth histogram metric learning to FER in CNNs, which enhanced the accuracy of FER under uncontrolled conditions. [PERSON] et al. [10] showed a new FER framework based on a support vector machine (SVM) classifier andCNNs, which improved the ability of network feature extraction and expression classification. [PERSON] et al. [11] showed a dual-branch multi-feature fusion network based on deep learning, in which one branch extracts multi-level facial features to enhance feature recognition capabilities and the other branch enhances the adaptability of learned features to direction and scope changes, improving the ability of network feature extraction. Although the above methods provide some effective research methods for FER in real scenes, there are still some issues: (1) Due to realistic factors such as complexity, subtlety, and occlusion of facial expression in real scenes, the robustness of the features extracted by shallow networks is poor, and it is difficult to capture the local detailed facial expressions' features. (2) The deep network model promotes the capacity of feature extraction and the accuracy of expression recognition, but simultaneously, it greatly increases the number of network parameters and computational costs. In practical applications, limited by the cost of computing hardware configuration, the method of deepening the network depth in exchange for recognition accuracy is not practical, which is not conducive to the further development of FER.  
In response to the aforementioned problems, a lightweight and efficient method of FER is shown in this paper. The following is an overview of the main innovation points of this paper:  
1. To resolve the issue of the uneven inter-class distribution of facial expression data, a dual-branch network (RLR-CNet) is designed in our paper.
2. To alleviate the issue of easily losing feature information and to cut down on the number of parameters of the RLR-CNet, a lighter Clip_K5_ShuffleNet inverted residual structure is proposed in the RLR-CNet.
3. To facilitate the transfer of facial expression feature information and promote the generalization ability of the RLR-CNet, the \(\beta\)-Mish activation function and the improved random masking method are used in the RLR-CNet, respectively.
4. To extract facial expressions' local key features and further enhance the accuracy of FER without significantly increasing computational complexity, a shuffle attention (SA) module is embedded into the RLR-CNet, which integrates spatial and channel attention.