## 3 Proposed Methodology
### The Proposed Att-ResBiGRU Model
#### 3.3.2 Residual BiGRU Block  
Accurate recognition of human activities necessitates more than relying on convolution blocks for spatial pattern extraction, as human behaviors unfold over time in an ordered sequence. Models designed for sequential data, like RNNs, are better suited for processing behavioral time-series inputs. However, as the length of the input time series grows, RNNs face challenges with gradients fading during backpropagation and losing information from earlier time steps. As initially proposed by [PERSON] et al. [44], long short-term memory (LSTM) networks are a type of gated recurrent neural network able to retain long-range dependencies in sequence data. Unlike standard recurrent networks, LSTMs leverage internal gating mechanisms to store information over an extended time. This unique architecture enables LSTMs to model lengthy temporal contexts more effectively. However, human behavior exhibits complex bidirectional dynamics influenced by past and future actions. Although LSTM shows promise as an alternative to RNNs for addressing the vanishing gradient problem, the memory cells in this architecture contribute to increased memory usage. In 2014, [PERSON] et al. [45] introduced a new RNN-based model called the GRU network. GRU is a simplified version of LSTM that lacks a separate memory cell [46]. In a GRU network, update and reset gates are utilized to control the extent of modification for each hidden state. This means they determine which knowledge is essential for transmission to the next state and which is not. The BiGRU network, a two-way GRU network that considers both forward and backward information, enhances the extraction of time-series features compared to the GRU network by incorporating bidirectional dependencies. Consequently, a BiGRU network is appropriate for extracting time-series features from behavioral data. While the BiLSTM network exhibits proficiency in deriving features from time-series data, its capability to capture spatial features diminishes. Moreover, during the training phase, the issue of gradient disappearance becomes more pronounced as the number of layered layers increases. ResNet, a residual network, was introduced by the Microsoft Research team in 2015 as a solution to the issue of gradient disappearance [47]. The 152-layer network in question won the ILSVRC championship in 2015. Each block comprising the residual is denoted as follows:  
\[x^{i+1}=x^{i}+F(x^{i},W_{i}) \tag{8}\]The residual blocks are composed of two components: the direct mapping (\(x_{i}\)) and the residual portion (\(F(x^{i},W_{i})\)). The encoder module of the Transformer model is similarly modified to accommodate the envisioned framework. Our inquiry presents an alternative framework that capitalizes on the advantages of the residual design utilizing the BiGRU network. Normalization techniques can be implemented within the BiGRU network. When considering recurrent neural networks, layer normalization (LN) is found to be more advantageous compared to BN [48]. LN is subjected to a similar computation process as BN and can be expressed as follows:  
\[\hat{x}^{i}=\frac{x^{i}-E(x^{i})}{\sqrt{var(x^{i})}} \tag{9}\]  
where \(x^{i}\) denotes the input vector in the \(i\)-th dimension and \(\hat{x}^{i}\) represents the output subsequent to layer normalization. This research presents ResBiGRU, a novel integration of a residual structure and layer normalization in a BiGRU network (see Figure 4). Detailed below is an explanation of the recursive feature information \(y\):  
\[x_{t}^{f(i+1)}=LN(x_{t}^{f(i)}+G(x_{t}^{f(i)},W_{i})) \tag{10}\]  
\[x_{t}^{b(i+1)}=LN(x_{t}^{b(i)}+G(x_{t}^{b(i)},W_{i})) \tag{11}\]  
\[y^{t}=concat(x_{t}^{f},x_{t}^{b}) \tag{12}\]  
Layer normalization is referred to as \(LN\) in this specific context. In contrast, the management of input states within the GRU network is denoted as \(G\). The variable \(t\) in \(x_{t}^{f(i+1)}\) represents the \(t\)-th moment in the time series. In contrast, \(f\) and \(b\) denote the forward and reverse phases.