## 3 Materials and Methods
### Model Selection
#### 3.4.1 Long Short-Term Memory (LSTM)  
Introduced by [PERSON] and [PERSON] in 1997 [PERSON] and [PERSON] (1997), LSTM networks are a variant of RNNs that overcome the shortcomings of standard RNNs. These disadvantages include poor performance in handling long-term dependencies and the vanishing or exploding gradient problem. In 1999, a forgotten gate was added to LSTM to restore cell memory, improve the original structure, and become the standard structure for LSTM networks. Unlike deep feedforward neural networks (DFNN), LSTMs contain feedback connections and can process data sequences, not just individual data points such as vectors or arrays [PERSON] and [PERSON] (1997). In LSTM networks, the fundamental building block is called a memory block or LSTM unit. Composed of a cell that acts as the memory component and three gates (input, output, forget/keep), these units can retain information over arbitrary periods. The gates of the LSTM unit are responsible for regulating the flow of data through the cell. One of the most prominent features of the LSTM cell is the \"constant error carousel\" (CEC). An LSTM network is structured similarly to an RNN, except the hidden layers comprise memory blocks instead of neurons [PERSON] and [PERSON] (1997). Input gate: The unit features a sigmoidal function that regulates the inflow of data into the cell. It obtains activation from the previous output h\({}^{(t-1)}\) and the current input x\({}^{(t)}\). By means of the sigmoid function, an input gate produces values ranging from zero to one. A value of zero acts as a complete blockage of information, while a value of one permits the passage of all information. Equation (1) shows this process [PERSON] and [PERSON] (1997). \[i^{t}=\sigma\Big{(}W^{(ix)}\ x^{(t)}+U^{(lh)}\ h^{(t-1)}+b^{i}\Big{)} \tag{1}\]  
Cell input layer: The input to the cell is like the input gate. It takes in the previously hidden state h\({}^{(t-1)}\) and the current input x\({}^{(t)}\). However, a \"tanh\" activation function is used to squash the input values to a range between \(-1\) and \(1\), which is indicated by the symbol l' in the Equation (2) [PERSON] and [PERSON] (1997). \[l^{t}=tanh\ \Big{(}W^{(lx)}\ x^{(t)}+U^{(lh)}\ h^{(t-1)}+b^{i}\Big{)} \tag{2}\]  
Forget gate: A unit using a sigmoidal function decides what information from previous steps of the cell should be remembered or discarded. The forget gate takes input from h\({}^{(t-1)}\) and x\({}^{(t)}\) and assumes values between zero and one. The next step involves a Hadamard product with the old cell state c\({}^{t-1}\) to update to a new cell state c\({}^{t}\) in the below equation. If the forget gate has a value of zero, it is closed and will completely forget the information of the old cell state c\({}^{t-1}\). On the other hand, the value of one will make all information memorable. Thus, the forget gate has the authority to reset the cell state if the old data are deemed irrelevant. Equation (3) summarizes the forget gate mechanism [PERSON] and [PERSON] (1997). \[f^{t}=\sigma\Big{(}W^{(fx)}\ x^{(t)}+U^{(fh)}\ h^{(t-1)}+b^{f}\Big{)} \tag{3}\]  
Cell state: The cell state is responsible for storing a cell's memory over an extended period. Each cell contains a self-connected linear unit known as a constant error carousel (CEC), which recurrently operates to prevent the vanishing or exploding gradient problem in an LSTM network. The CEC incorporates a forget gate that regulates and resets the gate as needed. At time t, the present cell state c\({}^{t}\) is modified by the previous cell state c\({}^{t-1}\), controlled by the forget gate, and the current input and cell input product (i\({}^{t}\circ\) l'). Equation (4) summarizes the overall update of a cell state [PERSON] and [PERSON] (1997). \[c^{t}=f^{t}\circ c^{t-1}+i^{t}\circ l^{t} \tag{4}\]Output gate: A unit equipped with a sigmoidal function has the ability to regulate the passage of information from a cell.