## 3 Methods
### Network Architecture
#### 3.3.2 Spatial-Frequency Feature Learning  
Extracting spatial-frequency features is primarily accomplished through the use of the convolutional encoder and the attention module FeaNet discussed in Section 3.3.1, with the module structure illustrated in Figure 6.  
Specifically, 2u frames \(S_{i}\) in each segment are fed into the CNN module sequentially in time order, and the three-dimensional data structure of each frame is 9 \(\times\) 9 \(\times\) 8. To better retain information in the relatively small three-dimensional data structure, two convolutional layers are first implemented. The first layer employs a 1 \(\times\) 1 convolution kernel with 64 kernels, and the second layer utilizes a 3 \(\times\) 3 convolutional kernel with 128 kernels. These different convolutional kernels are utilized to extract deeper information from the three-dimensional data. Subsequently, the residual network is constructed using DC and PC. This combination reduces the number of training parameters while extracting internal features from expanded individual feature maps using DC and expressing cross-feature map relationships using PC. Moreover, the residual structure effectively avoids network degradation. ReLU is utilized as the activation function for each convolutional layer, and BatchNorm processing is performed. Padding operations are carried out for DC  
Figure 5: Comparison of SENet and FeaNet. induction, and \(h_{t}\) is the hidden state. In comparison, the output equation of the Bi-LSTM is shown below:  
\[y_{t}=\sigma(W_{h}\cdot[h_{t},h_{t}^{\prime}])+b_{h} \tag{21}\]  
Bi-LSTM networks combine an LSTM network that moves from the beginning of the sequence and an LSTM network that moves from the end of the sequence, and the backward layer is an extension of the information of past emotions. It is worth mentioning that the parameters of the two LSTM neural networks in Bi-LSTM are mutually independent, and they share only the input vector sequence. This structure merges the gate control architecture and the bidirectional feature perfectly and experimentally proves to be more efficient than a single LSTM for feature extraction of sequences. The network architecture of Bi-LSTM is shown in Figure 7. The time sequences are input to the model, and the forward layer has the information at time t and the previous time, while the backward layer has the information at time t and the subsequent time. The hidden layer data of the two LSTM layers can be processed using summation, average, or connection. Equation (21) is the output value in the form of a connection. To highlight the data at key frame moments, the output of Bi-LSTM is not used as the result of the whole temporal feature learning module in this paper. Instead, a nonlinear transformation is first performed for each frame moment hidden layer state with the following equation:  
\[H_{temp,j}=\tanh(W_{temp,j}h_{j}+b_{temp,j}) \tag{22}\]  
The amount of memory in each LSTM layer is \(\frac{q}{2}\). The hidden layer is processed in a connected form to obtain \(h_{j}\in R^{d}\). \(H_{temp,j}\) is the nonlinear expression of the hidden layer, while \(W_{temp,j}\in R^{d\times q}\) and \(b_{temp,j}\in R^{d}\) are the weights and bias vectors of the \(tanh\) function, respectively, and \(d\) is set to 512. After obtaining \(H_{temp,j}\in R^{d}\), the softmax function is used to calculate the weights for each frame moment to obtain \(A_{temp,j}\). The specific equation is as follows:  
\[A_{temp,j}=\frac{\exp(H_{temp,j}u_{temp,j})}{\sum\limits_{j}\exp(H_{temp,j}u _{temp,j})} \tag{23}\]  
Figure 7: Bi-LSTM structure information. where \(\mathrm{u}_{\mathrm{temp}j}\in\mathrm{R}^{\mathrm{d}}\) is a trainable parameter, and the greater the value of \(\mathrm{A}_{\mathrm{temp}j}\), the more important the corresponding frame is in the timing sequence.