## 3 Materials and Methods
### Lightweight Expression Recognition System
#### 3.2.2 Selection of CNN Model Structure  
A depthwise separable convolutional network [17] is adopted for facial expression recognition and noise detection in order to improve the operating efficiency of the recognition system. The network decouples the channel correlation and spatial correlation of the convolutional neural network, simplifies the network, and removes parameters with the idea of the residual module, which greatly reduces the model size. Its core processes, including depthwise convolution and pointwise convolution, are shown in Figure 4.  
In order to focus on important feature weights in model learning, this paper adds the attention mechanism module CBAM [29] after the depth separation convolution layer of the first residual module of the CNN model [17]. The CBAM module has two sequential sub-modules: channel attention module and spatial attention module. For the channel attention module, the process can be summarized as:  
\[\mathbf{F}^{\prime} =\mathbf{M_{c}}(\mathbf{F})\otimes\mathbf{F} \tag{16}\] \[\mathbf{F}^{\prime\prime} =\mathbf{M_{s}}\big{(}\mathbf{F}^{\prime}\big{)}\otimes\mathbf{F} ^{\prime}\]  
where \(\mathbf{F}\in\mathbb{R}^{C\times H\times W}\) denotes an intermediate input feature map, \(\mathbf{M_{c}}\in\mathbb{R}^{C\times 1\times 1}\) denotes a 1D channel attention map, \(\mathbf{M_{s}}\in\mathbb{R}^{1\times H\times W}\) denotes a 2D spatial attention map, and \(\otimes\) denotes element-wise multiplication. The channel attention module is computed as:  
\[\mathbf{M_{c}}(\mathbf{F}) =\sigma(MLP(\text{Avg Pool}(\mathbf{F}))+MLP(\text{Max Pool}( \mathbf{F}))) \tag{17}\] \[=\sigma\Big{(}\mathbf{W_{1}}\Big{(}\mathbf{W_{0}}\Big{(}\mathbf{ F}_{\text{avg}}^{c}\Big{)}\Big{)}+\mathbf{W_{1}}(\mathbf{W_{0}}(\mathbf{F}_{ \text{max}}^{c}))\Big{)}\]  
where \(\mathbf{F}_{\text{avg}}^{c}\) and \(\mathbf{F}_{\text{max}}^{c}\) denote average-pooled features and max-pooled features, respectively. \(\mathbf{W_{0}}\in\mathbb{R}^{C/r\times C}\) and \(\mathbf{W_{1}}\in\mathbb{R}^{C\times C/r}\), and they are the weights of the multi-layer perceptron (MLP). \(\sigma\) denotes the sigmoid function. The sigmoid function is calculated as:  
\[y=\frac{1}{1+e^{-x}} \tag{18}\]  
As the specific model structural parameters have been described in the reference [17], they will not be described here. A simple schematic diagram of the model structure is shown in Figure 5.  
BN refers to Batch Normalization, which can solve the problem that the data distribution changes in the training process, so as to prevent the gradient from disappearing or exploding and speed up the training speed. It is calculated as:  
\[y^{(i)}\leftarrow\gamma\odot\xi^{(i)}+\beta \tag{19}\]  
where \(\gamma\) denotes the Stretch parameter, and \(\beta\) denotes the Offset parameter. The Relu function is calculated as:  
\[f(x)=\max(0,x) \tag{20}\]  
The scale test comparison of some lightweight models is shown in Table 1.  
Figure 4: Depthwise separable convolution.  
It can be found that the network of reference [17] is very tiny, which is suitable for building a lightweight facial expression recognition system. However, the proposed CNN [17]+CBAM can improve the accuracy of the reference [17] network by about 1 percentage point in training convergence on the FER2013 [30] dataset, but the model size is almost unchanged, so it is feasible.