## 3 The Proposed Framework
### Long Short-Term Memory  
Recurrent neural networks (RNNs) are commonly utilized in deep-learning frameworks owing to their capacity to process sequential data. These networks are composed of interconnected neurons, where the output of one neuron not only influences the next layer, but also feeds back to the previous layer. This arrangement facilitates the retention of the underlying information until it is fed back into the subsequent prediction, making it simpler to implement and train the network. However, RNNs have limited ability to handle long-term dependencies in sequential data, which can result in challenges such as vanishing and exploding gradients. Vanishing gradients occur when gradients during backpropagation significantly diminish, making adequate weight updates challenging, whereas exploding gradients result from large gradients accumulating during backpropagation, leading to unstable models owing to substantial weight updates. Consequently, RNNs have a limited capacity to manage long-term dependencies. LSTM networks enhance predictions for medium- and long-term data by incorporating additional state and gate control units on top of RNNs. This is facilitated by internal mechanisms that regulate the flow of relevant information in both short and long terms. The fundamental structure of an LSTM cell includes a cell state that preserves long-term memory, and a hidden state that retains short-term memory, as illustrated in Figure 2.  
In Figure 2, three gates are incorporated to regulate the flow of information through the memory cell. The input gate controls the amount of information read into the cell, while the forget gate determines the amount of past information that must be forgotten and retained. The output gate was responsible for reading the output from the cell. This design allows the cell to decide when to retain information and ignore inputs, which is crucial for remembering useful information and forgetting less-useful information. Mathematically, these gates are computed as follows. \[ \begin{split}& I_{t}=\sigma(W_{t}\cdot y_{t}+U_{I}\cdot h_{t-1}+b_{ i}),\\ & F_{t}=\sigma(W_{f}\cdot y_{t}+U_{f}\cdot h_{t-1}+b_{f}),\\ & O_{t}=\sigma(W_{o}\cdot y_{t}+U_{o}\cdot h_{t-1}+b_{o}).\end{split}  \tag{2}\]  
In the previous equations, \(I_{t}\), \(F_{t}\) and \(O_{t}\) represent the input, forget, and output gates at the current time step \(t\), respectively. The variable \(y_{t}\) represents the input at time \(t\) and \(h_{t-1}\) is the hidden state at the previous time step. \(W\) and \(U\) represent the weight matrices and  
Figure 2: LSTM cell structure. (Our elaboration). is the bias vector. \(\sigma\) is the sigmoid activation function that sets the values of these gates in the range \([0,1]\). The memory cell and hidden state are updated as follows. \[ \begin{split}&\bar{c}_{t}=\tanh(W_{c1}\cdot h_{t-1}+W_{c2}\cdot y_{t }+b_{c}),\\ & c_{t}=F_{t}\odot c_{t-1}+I_{t}\odot(\bar{c}_{t}),\\ & h_{t}=O_{t}\odot\tanh(c_{t}).\end{split}  \tag{3}\]  
The candidate memory cell, denoted by \(\bar{c}_{t}\), can capture both the previous information stored in \(h_{t-1}\) and current input value \(y_{t}\). The calculation of the content of the memory cell, represented by \(c_{t}\), is determined by the past memory cell state \(c_{t-1}\) and current memory cell candidate, which functions as the current input. The process of determining the amount of past information to be discarded and the amount of current input to be retained is facilitated by element-wise matrix multiplication represented by \(\odot\) and an activation function. In this process, \(F_{t}\) and \(I_{t}\) are used to control the amount of information discarded or retained in the element-wise matrix multiplication. For instance, when the values of \(F_{t}\) are close to zero, the results of the element-wise matrix multiplication between \(F_{t}\) and \(c_{t-1}\) render past information negligible or, in other words, obsolete.