## 2 Theoretical Background  
### Lstm  
The RNN approach is commonly utilized by practitioners in forecasting studies within the ML models due to its acceptable performance with sequential data [36]. However, despite its capacity for processing short-term sequential data, the RNN's success appears unsatisfactory when learning long-term dependencies or prolonged context memorization is desired in time-series forecasting applications. LSTM is an improved version of RNN that has been developed to effectively learn long-term dependencies of time series data while mitigating the effects of 'vanishing gradients' or 'exploding gradients' through its memory cell and gate structures. These distinguishing features set it apart from traditional RNNs [37]. Figure 1 provides an illustration of the LSTM architecture. The memory cell (C\({}_{t}\)) records neuron states at time \(t\), while selected information passes through gates that employ a sigmoid neural layer and point-by-point multiplication system. LSTM achieves information conservation and supervision via the forget gate (f\({}_{t}\)), input gate (i\({}_{t}\)), and out-put gate (o\({}_{t}\)) mechanisms. The forget gate determines whether to retain or eliminate previous information from the cell state C\({}_{t-1}\) using a sigmoid function, and the output of this gate (f\({}_{t}\)) at time \(t\) is computed as follows.  
\[\text{f}_{t}=\sigma\left(\text{W}_{\text{H}}[\text{h}_{t-1}],\text{W}_{\text{ K}}\left[\text{x}_{t}\right],\text{b}_{t}\right)\text{,} \tag{1}\]  
where \(\sigma\) is a nonlinear function (e.g., logistic sigmoid, a hyperbolic tangent function, or rectified linear unit (ReLU)), \(\text{h}_{t-1}\) is the hidden state at time \(t-1\). In a typical LSTM model, x is the input vector; h is the output of the LSTM unit; W\({}_{x}\) and W\({}_{h}\) are weight matrices that are used in DL model, and b is a constant bias. The sigmoid function normalizes all activation values between 0 (all removed) and 1 (all preserved). Secondly, the input gate determines whether to add the new information to the LSTM memory or not. This gate has two layers: a sigmoid layer, which makes decision of values that need to be renewed, and a \"tanh\" layer which forms a vector of candidate values for adding into the LSTM memory. The following equations are formulas for the outputs of these two layers:  
\[\text{i}_{t}=\sigma(\text{W}_{\text{ih}}[\text{h}_{t}-1],\text{W}_{\text{ix}} \left[\text{x}_{t}\right],\text{b}_{t})\text{,} \tag{2}\]  
\[\hat{C}_{t}=\tanh(\text{W}_{\text{ch}}[\text{h}_{t}-1],\text{W}_{\text{cx}} \left[\text{x}\right],\text{b}_{c})\text{,} \tag{3}\]where \(i_{t}\) indicates whether the value is required or redundant, \(\hat{C}_{t}\) denotes the vector of new candidate values suitable for addition to the model's memory, while the necessity/redundancy of the value is indicated. The integration of the two layers provides a revision of the LSTM model. The formulation of this revision is as follows:  
\[C_{t}=[(f_{t}*C_{t-1})+(i_{t}*\hat{C}_{t})], \tag{4}\]  
The outcome of this process ranges from 0 to 1, where 0 signifies full eradication of the value and 1 stands for preservation. The output gate ultimately selects which part of the LSTM memory will add to the output, utilizing a sigmoid layer to multiply the output of this layer with the non-linear tanh function which retains the values in the range of \(-1\) to 1, according to the equations formulated in [38].  
\[o_{t}=\sigma(W_{\text{oh}}[h_{t}-1],W_{\text{ox}}\;[x_{t}],\;b_{o}), \tag{5}\]  
\[h_{t}=o_{t}*tanh(c_{t}), \tag{6}\]