## 3 Proposed Methodology
### The Proposed SEResNet-BiGRU Model
#### 3.4.3 SERSidual Block  
DL networks (LeNet, AlexNet, and VGGNet) frequently start with convolutional layers for detailed descriptions. They terminate with a fully-connected layer for categorization applications, which does not need to ignore connections. Sequential networks are architectures where each layer transmits data to the following layer. The complexity  
Figure 6: Structure of Bidirectional GRU (BiGRU): (**a**) GRU cell and (**b**) unroll BiGRU. of disappearing or bursting gradients increases as the sequential network's layer depth increases. To address this issue, the residual block in a ResNet enables the omission of interconnections between blocks of convolutional layers, improves gradient propagation, and enables the training of even deeper CNNs to avoid gradient vanishing concerns. The following formula could be used to represent a residual layer:  
\[\text{ELU}(x)= \begin{cases}x&x\geq 0\\ \alpha\times(e^{x}-1)&x<0\end{cases}  \tag{11}\]  
\[R(x)=\text{ELU}(x+f(x)) \tag{12}\]  
where \(f(x)\) represents the layer's output, \(x\) represents the input, \(\text{ELU}(x)\) represents the exponential linear unit function and \(R(x)\) represents the output of the residual block. The residual component \(f(x)\) is provided in this block as two consecutive duplicates of three operations: convolution with a 3 \(\times\) 1 filter, batch normalization, and ELU activation. Afterward, the feature map from \(f(x)\) is combined with the input \(x\). Lastly, the combined features are processed to the ELU activation function. In this study, we proposed the SEResidual block to retrieve hybrid features hierarchically by integrating spatio-temporal and channel-wise data [50]. As seen in Figure 5, this residual block was constructed of Conv1D layers, BN layers, ELU layers, SE components, and a direct connection to BiGRU. The SE modules were included to increase the recommended model's representational capability in channel attention. The architecture of an SE module is shown in Figure 7. Following the convolution procedure, many feature maps are obtained. Nevertheless, specific feature maps may include duplicated data. The SE module performs feature recalibration to improve significant features and disable less beneficial ones. This module is divided into two parts: squeeze and excitation stages. Firstly, the squeeze stage excerpts all the data corresponding to their channels. The size \(C\times H\times W\) of the feature map \(U\) corresponding to one channel in \(U\) is \(H\times W\). Feature maps for each channel are squeezed into \(1\times 1\) feature maps employing a channel descriptor function, for example Global Average Pooling (GAP) [51]. This phase generates a scalar value, including global information about the channel. Equation (13) demonstrates the squeeze operation, where \(u_{c}(i,j)\) signifies a feature map relating to channel \(c\) after \(X\) has  
Figure 7: Structure of the SE module. gone through the convolution layer. The channel descriptor method \(F_{squeeze}\) was employed in this investigation, and GAP was included. \[Z_{c}=F_{squeeze}(U_{c})=\frac{1}{H\times W}\sum_{i=0}^{H}\sum_{j=0}^{W}U_{c}(i,j) \tag{13}\]  
Then, during the excitation stage, the channel-wise dependencies are examined utilizing channel descriptors collected during the squeezing stage. This may be accomplished using fully-connected (FC) layers and non-linear functions. Equation (14) illustrates the excitation stage, where \(z\) is the squeezed value, \(W_{i}\) is the \(i\)th FC layer, \(\sigma\) is the sigmoid function, and \(\mathbf{F}_{excite}\) is the excitation function. According to the sigmoid, the excitation step's resulting value is 0 and 1. It can be utilized as a calibration factor. Excitation produces a new weight \(s\), multiplied by the current feature map \(U\). The SE module employed in this investigation is shown in Figure 7, along with the architecture of the squeeze and excitation stages inside the SE unit.