## 3 Proposed Method
### Bidirectional Attention GRU Layer  
GRU (gated recurrent unit) is an improved type of RNN that addresses the issues of gradient explosion and vanishing gradients encountered by traditional recurrent neural networks. By introducing update gates and reset gates, GRU effectively mitigates these problems. Compared to Long Short-Term Memory (LSTM), GRU has a lower parameter count, approximately 2/3 of LSTM. This property makes GRU less prone to overfitting and also demonstrates better performance in terms of convergence time and iteration. To effectively leverage contextual information among different facial regions in facial expression recognition, this study adopts BiGRU to model the feature information of facial subregions. By applying BiGRU to these feature sequences of facial subregions in both forward and backward computations, two different sets of hidden states are obtained. By combining these two sets of states, BiGRU outperforms unidirectional GRU in terms of fitting capability. This combination of forward and backward computations allows for better utilization of the input sequence information, enabling the network to capture subtle variations in facial expressions more effectively. As shown in Figure 1, the cell structures of the forward-propagating GRU and the backward-propagating GRU are given, and the corresponding computational formulas are given as follows:  
\[\begin{cases}\mathbf{z}_{t}^{(p)}=\sigma(\mathbf{W}_{z}^{(p)}x_{t}+\mathbf{U}_ {z}^{(p)}h_{t-1}^{(p)}+b_{z}^{(p)})&\text{(a)}\\ \mathbf{r}_{t}^{(p)}=\sigma(\mathbf{W}_{r}^{(p)}x_{t}+\mathbf{U  
The BiGRU formulation uses the index t to denote the step of the current iteration and the letters \(p\) and \(b\) to denote the forward and backward GRU cells, respectively, to which they belong, where the reset and update gates of the GRU unit are denoted by **r** and **z**, respectively. The amount of information from the preceding stage that is to be saved to the present stage's memory is determined by the update gate, whereas the reset gate determines the amount of information that is to be kept from the preceding stage. \(W_{z}\),\(W_{r}\), and \(W_{h}\) are the initial weights of the neurons of the present input sequence, \(U_{z}\), \(U_{r}\), and \(U_{h}\) represent the initial weights of the hidden layer in the preceding stage, and \(b_{z}\), \(b_{r}\), and \(b_{h}\) are the respective offsets. Specifically, each GRU unit first computes the states of the reset and update gates by obtaining the hidden state \(h_{t-1}\) of the preceding stage and the present input \(x_{t}\). A state vector in the 0-1 range is obtained as a result of the application of the sigmoid activation function. Next, the state vector after forgetting is obtained by computing \(h_{t-1}\odot r_{t}\), where a multiplication operation between individual elements is depicted by \(\odot\). Then, the result of the preceding step is added to the bias and the corresponding weighted input vector \(x_{t}\), and activated by the nonlinear activation function \(\varphi\). The update gate in the present cell decides which information should be kept as the final information. Before passing the intermediate layer results to the attention layer, it is necessary to add the hidden states of the GRU cells and to obtain the final corresponding hidden states, as shown in Equation (5). (1) Attention mechanism: Motivated by the most advanced attention mechanism at present, the attention layer is positioned after the BiGRU layer to enhance the recognition ability of facial expressions. The feature information of the facial region is adaptively weighted to emphasize the salient information via two fully connected layers and a ReLU activation function. The attention module's weight values are displayed in red in Figure 1. The state vector \(s\) is derived by adding the product of the corresponding weight coefficients, which is computed using formula (6), to the final state \(h_{t}\) of every hidden layer. \[