## 2 Methods
### Bidirectional LSTM  
Conventional RNNs have difficulty capturing long-range dependencies due to the gradient vanishing problem, where gradients shrink during backpropagation, limiting the network's ability to learn long-range dependencies. LSTM networks are introduced to address the gradient vanishing problem. LSTM integrates memory cells, input, forget, and output gates to selectively retain or discard information in long sequences. Despite the enhancements provided by LSTMs over traditional RNNs, unidirectional LSTMs encounter difficulties in effectively capturing information from past and future contexts, particularly in tasks where directional information is essential. The concept of bidirectional processing in BiLSTM utilizes both forward and backward information, enabling the network to capture contextual information more comprehensively [36]. This bidirectional approach allows the LSTM model to capture dependencies  
Figure 3: The attention mechanism process. and patterns from past and future information, providing a more comprehensive understanding of the input sequence. BiLSTM is successfully applied to a variety of tasks, including NLP, speech recognition, and sequence-to-sequence modeling [37; 38; 39]. The computations within the LSTM cell encompass updating cell states, calculating hidden states, and determining the output during both the forward and backward passes. These computations govern the forward and backward passes in an LSTM cell including computations for updating cell states, calculating hidden states, and determining the output. The bidirectional nature allows the model to consider the bidirectional context for each time step. The demonstration diagram of the BiLSTM network is shown in Figure 4.  
During the input sequence period, the input to a BiLSTM consists of a sequence of data points or tokens, each represented as a vector. Then the entire sequence is composed of a series of these vectors. For instance, in natural language processing tasks, individual words within a sentence can be represented as a vector using word embeddings. In the forward LSTM processing phase, the input sequence is processed sequentially from the beginning to the end by a forward LSTM layer. At each time step \(t\), the LSTM takes the input vector \(x_{t}\) corresponding to the \(t\)-th token in the sequence. The LSTM computes an output \(h_{t}\) and a hidden state \(c_{t}\) based on \(x_{t}\), the previous hidden state \(h_{t-1}\), and cell state \(c_{t-1}\). The output \(h_{t}\) and the updated hidden state \(c_{t}\) are passed to the next time step. In the backward LSTM processing period, the input sequence is processed in reverse order using a backward LSTM layer. At each time step \(t\) (from end to beginning), the LSTM takes the input vector \(x_{t}\) and computes an output \(\widetilde{h}_{t}\) and a hidden state \(\widetilde{c}_{t}\) based on \(x_{t}\), the previous hidden state \(\widetilde{h}_{t+1}\), and cell state \(\widetilde{c}_{t+1}\). The output \(\widetilde{h}_{t}\) and the updated hidden state \(\widetilde{c}_{t}\) are passed to the previous time step. In the combining outputs, after processing both forward and backward sequences, the outputs \(h_{t}\) from the forward LSTM and \(\widetilde{h}_{t}\) from the backward LSTM are concatenated for each time step \(t\). The combined outputs \(\left[h_{t};\widetilde{h}_{t}\right]\) capture information from past and future contexts of the input sequence. In the final output period, the concatenated outputs \(\left[h_{t};\widetilde{h}_{t}\right]\) are then typically used as inputs to subsequent layers of the neural network (e.g., fully connected layers) for tasks like sequence tagging, sentiment analysis, and machine translation. In summary, a BiLSTM processes input data using two LSTM layers operating in opposite directions, which enables  
Figure 4: A demonstration diagram of the BiLSTM network. the model to capture dependencies and contexts from past and future information within a sequence [40].