## 2 Methodology
### Long-Short Term Memory (LSTM)  
The long short-term memory (LSTM) networks improve upon standard recurrent neural networks (RNNs) by incorporating a mechanism that selectively retains or discards information via specialized memory cells and gates (Figure 6) [53]. The memory cells can retain information for prolonged periods, with the input, forget, and output gates regulating the data flow in and out [54; 55]. LSTM effectively captures both short-term and long-term dependencies in the input. LSTM networks exhibit enhanced performance over conventional RNNs in tasks requiring long-term dependencies, effectively addressing the vanishing gradient problem. Success has been achieved in various applications, including video summarization [56], hand gesture recognition [57], and ship motion forecasting [58]. Furthermore, LSTMs have been integrated with other deep learning frameworks, including convolutional neural networks (CNNs), to develop hybrid models that effectively manage both spatial and temporal data [59].  
An LSTM network is composed of a series of recurrently connected sub-networks known as memory blocks [60]. The blocks connect to the LSTM layer, which consists of units featuring an input gate, a forget gate, a cell with a self-recurrent loop, and an output gate [55]. These components dynamically add or remove information as required, a method first introduced by [PERSON] and [PERSON] [53] and subsequently expanded by [PERSON] et al. [61]. Various methods exist for optimizing LSTM parameters and reducing the loss function [55]. Similar to recurrent neural networks (RNNs), long short-term memory (LSTM) networks map an input sequence x to an output sequence y by iteratively computing the network unit activations from \(t=1\) to \(t=\tau\), with initial conditions \(C_{o}=0\) and \(h_{o}=0\). The LSTM computations are described by the following equations:  
\[i_{t}=\sigma(W_{i}\:x_{t}+U_{i}\:h_{t-1}+b_{i}) \tag{3}\]  
\[f_{t}=\sigma\Big{(}W_{f}\:x_{t}+U_{f}\:h_{t-1}+b_{f}\Big{)} \tag{4}\]  
\[o_{t}=\sigma(W_{o}\:x_{t}+U_{o}\:h_{t-1}+b_{o}) \tag{5}\]  
\[C_{t}=tanh(W_{c}\:x_{t}+U_{c}\:h_{t-1}+b_{c}) \tag{6}\]  
\[C_{t}=f_{t}\otimes\:C_{t-1}+i_{t}\otimes\:C_{t} \tag{7}\]  
\[h_{t}=o_{t}\otimes\:tanh(C_{t}) \tag{8}\]  
Figure 6: The schematic of LSTM architecture.  
where  
Weight matrices: \(W_{i}\), \(W_{f}\), and \(W_{o}\) represent the weights from the input to the input gate, forget gate, and output gate, respectively. Similarly, \(U_{i}\), \(U_{f}\), and \(U_{o}\) denote the weights connecting the hidden layer to these gates.  
Bias vectors: \(b_{i}\), \(b_{f}\), and \(b_{o}\) are the bias terms associated with the input, forget, and output gates.  
Activation functions: The sigmoid function (\(\sigma\)) serves as the non-linear activation applied element-wise, while the hyperbolic tangent (\(\tanh\)) scales values between \(-1\) and \(1\).  
Gates and cell state: The vectors \(i_{t}\), \(f_{t}\), \(o_{t}\), and \(c_{t}\) correspond to the input gate, forget gate, output gate, and cell state at time step \(t\), all of which are of the same dimension as the hidden state vector \(\operatorname{\mathit{ht}}\).  
Element-wise operations: The symbol \(\otimes\) denotes element-wise multiplication between vectors.  
The LSTM network's architecture uses these gates and the cell state to regulate the flow of information, enabling it to capture both long-term dependencies and short-term patterns effectively. For detailed analyses and further discussions on LSTM mechanisms, references such as [PERSON] et al. (2012), [PERSON] et al. (2013), and [PERSON] et al. (2014) provide valuable insights.  
Hyperparameters belonging to LSTM used in this study are as follows: hidden layers = (8, 16, 32, 64, 128, 256), solver = (Adam), step size = (96), and initial learning rate = (0.001, 0.3).