## 2 Related Literature Review
### Deep Learning Models
#### 2.2.4 BiGRU  
A forward Gated Recurrent Unit (GRU) and a backward GRU neural network were integrated to form the Bidirectional Gated Recurrent Unit (BiGRU) network, incorporating representations of the hidden layers from both the forward and backward GRUs. In a conventional unidirectional RNN, information progresses from the past to the future. A bivariate RNN (BiGRU) incorporates contextual information from both past and future data in time series analysis by adding a reverse layer to the bidirectional GRU, hence enhancing its capacity to extract contextual information from sequential data [37; 51]. The pertinent information regarding the past and future of the inquiry is comprehensively employed in the input and output throughout the mapping process between sequences, guaranteeing that the context preceding and succeeding the sentence is documented and that the past and future information is thoroughly acknowledged. BiGRU enhances the unidirectional network through the second layer of the network [51].  
Linked characteristics and possible text connections have been identified by researchers [52] using MLCNN and BiGRU with attention mechanisms, respectively. Horizontal fusion has been used for classification, which can lead to an increase in classification accuracy [52].  
Furthermore, the downstream-specific NLP job and the pre-training-generated word vector were methodically transformed into the pre-training-generated word vector utilizing the BERT-BIGRU-CRF entity extraction technique, as evidenced by the researchers in [53].  
BiGRU, an enhanced variant of BiLSTM, utilizes fewer parameters by merging the input and forget gates into a singular update gate. Subsequently, two gates--the reset gate and the update gate--are utilized to alter the GRU's secret state. BiGRU enhances the process while maintaining the ability to manage long-term dependencies in both forward and backward directions. Equations (9)-(12) calculate the gates and states for the forward GRU [54]:  
\[\mathbf{z}_{t}=\sigma(\mathbf{W}_{z}\cdot[\mathbf{h}_{t-1},\mathbf{x}_{t}])\ (\text{Update gate}) \tag{9}\]  
\[\mathbf{r}_{t}=\sigma(\mathbf{W}_{r}\cdot[\mathbf{h}_{t-1},\mathbf{x}_{t}])\ (\text{Reset gate}) \tag{10}\]  
\[\widetilde{\mathbf{h}}_{t}=\text{tanh}(\mathbf{W}_{h}\cdot[\mathbf{r}_{t}*\mathbf{h}_{t-1},\bm {x}_{t}]) \tag{11}\]  
\[\mathbf{h}_{t}=(\mathbf{1}-\mathbf{z}_{t})*\mathbf{h}_{t-1}+\mathbf{z}_{t}*\widetilde{\mathbf{h}}_{t} \tag{12}\]  
For the bidirectional GRU [47], see Equations (13)-(15):  
\[\widetilde{\mathbf{h}}_{t}=\text{GRU}_{forward}\bigg{(}\mathbf{x}_{t},\widetilde{\mathbf{ h}}_{t-1}\bigg{)} \tag{13}\]  
\[\widetilde{\mathbf{h}}_{t}=\text{GRU}_{backward}\bigg{(}\mathbf{x}_{t},\widetilde{\mathbf{ h}}_{t+1}\bigg{)} \tag{14}\]\[\mathbf{h_{t}}=\left[\begin{array}{cc}\xrightarrow{\mathbf{h_{t}}}&\xrightarrow{\mathbf{h_{t}} }\\ \end{array}\right]\text{ (Final hidden state)} \tag{15}\]  
There are two kinds of gates in the GRU design shown in Figure 1: update gates and reset gates. A linear interpolation between earlier hidden states defines the hidden state \(h_{t}\). Bi-GRU's structure is the result of combining two distinct directional GRUs, as seen in Figure 2. Both directional GRUs received the same input in the study, which was also the GRU's input, and both GRUs determined the output.