## 2 Background
### Long-Short Term Memory Model  
The purpose for which RNNs were designed [15] is time series forecasting. A normal multilayer perceptron can be compared to the structure of this network, except for the hidden unit connections being enabled after a certain delay. Consequentially, the model can detect correlations between temporal occurrences in the data that are not next to each other and can be far away. Despite the capabilities of the first versions of RNNs, they are not without deficiencies. The training of such models is hindered due to the exploding and vanishing gradient issues. These problems can be mitigated by incorporating newer RNN versions, of which an LSTMcan be considered an example. However, these models can be complicated, as they require a large number of hyperparameters. It should be noted that a basic RNN can outperform newer recurrent models such as the LSTM and gated recurrent units (GRU), due to the smaller number of parameters [16]. On the other hand, an advantage of RNNs is that they do not require input vectors of fixed length for which the output is also fixed. Sequences and other rich structures can be used to take advantage of this characteristic.Put simply, a RNN can, not only work with sequences of input vectors, but also generate sequences for the output. With the hidden state being retained, the RNN can process the data of the sequence. Researchers have applied the LSTM to problems similar to the topic explored in this work. Storage is available for information within the network, which enables the usage of previously learned information. Time series problems require that a model uses long-term dependency. This is achieved by replacing the memory cells of the hidden layers. Data are manipulated using three gates of the LSTM, which are the input, output, and forget gates. The data passes through the forget gate \(f_{t}\), which decides if the data will be forgotten based on Equation (4). \[f_{t}=\sigma(W_{f}x_{t}+U_{f}h_{t-1}+b_{f}), \tag{4}\]  
for which the range of the gate \(f_{c}\) is \([0,1]\), the sigmoid function is displayed as \(\sigma\), while the variable matrices are shown as \(W_{f}\) and \(U_{f}\), and \(b_{f}\) is the bias vector. The second gate is the input gate, which decides which data will be saved, as shown in Equation (5). \[i_{t}=\sigma(W_{i}X_{t}+U_{i}h_{t-1}+b_{i}), \tag{5}\]  
where \([0,1]\) is the range of \(i_{t}\), and the learnable parameters are displayed as \(b_{i}\)\(W_{i}\), and \(U_{i}\), while the potential update vectors \(C_{t}\) are calculated using Equation (6). \[C_{t}=tanh(W_{c}x_{t}+U_{c}h_{t-1}+b_{c}), \tag{6}\]  
for which the parameters for learning are given as \(b_{c}\), \(W_{c}\)\(U_{c}\). Equation (7) describes the process that follows. \[C_{t}=F_{t}\odot C_{t-1}+i_{t}\odot C_{t}, \tag{7}\]  
where \(\odot\) is signification and element-wise multiplication.Data that are to be disposed of are given as \(C_{t-1}\). \(f_{t}\odot C_{t}\) provides the data to be stored, while the new data \(i_{t}\odot C_{t}\) will be stored by \(C_{t}\). The hidden state \(h_{t}\) value \(o_{t}\) output gate is calculated using the sigma function as given in Equation (8)  
\[o_{t}=\sigma(W_{o}x_{t}+U_{o}h_{t-1}+b_{o}), \tag{8}\]  
\[h_{t}=o_{t}\odot tanh(C_{t}), \tag{9}\]  
where \(o_{t}\) is defined in the range \([0,1]\) with \(b_{o}\), \(W_{o}\), and \(U_{o}\) learnable parameters. The \(h_{t}\) output value is represented as the \(o_{t}\) and \(tanh\) product value of \(C_{t}\) in Equation (9). Optimal value selection of hyperparameters requires experimentation, as well as optimization, for which, in this case, the PSO is used. The performance of the model is directly influenced by the hyperparameters. The following text provides a list of the hyperparameters usually optimized when configuring LSTM models indicated in bold:  
**1. The number of hidden layers (\(n_{hid}\)):** The hidden layers determine the model's depth.