## 2 Materials and Methods
### Neural Networks Architectures for TEC Forecasting
#### 2.2.2 Recurrent Neural Networks  
Among the variety of different types of recurrent neural networks, neural networks based on the employment of LSTM and GRU cells are most widely used in solving problems of forecasting and modeling time series. Long short-term memory neural networks were developed to solve the long-term dependency problem and make it possible for neural networks to effectively learn information over long periods of time. This architecture was originally introduced by [PERSON] and [PERSON] [6] and became one of the most widespread architectures that has found an application in almost every area. LSTM networks store the information about learned temporal dependencies between each element of the input sequences during its operation in the internal memory state cell. The  
Figure 5: Illustration of negative disturbances during periods of large negative values of the Dst index: (**a**) influence on behavior of TEC, (**b**) response \(\delta\)TEC.  
main advantage of LSTM is the ability to handle longer time sequences compared to a conventional recurrent neural network, which faces the vanishing gradient problem.  
The set of equations defining the LSTM cell is given as follows:  
\[i_{t}\ =\ \sigma(w_{i}\cdot[h_{t-1},x_{t}]+b_{i}), \tag{1}\]  
\[f_{t}\ =\ \sigma\Big{(}w_{f}\cdot[h_{t-1},x_{t}]+b_{f}\Big{)}, \tag{2}\]  
\[o_{t}\ =\ \sigma(w_{o}\cdot[h_{t-1},x_{t}]+b_{o}), \tag{3}\]  
\[\widetilde{C}_{t}\ =\ tanh(w_{C}\cdot[h_{t-1},x_{t}]+b_{C}), \tag{4}\]  
\[C_{t}\ =\ f_{t}\odot C_{t-1}+i_{t}\odot\widetilde{C}_{t}, \tag{5}\]  
\[h_{t}\ =\ o_{t}\odot tanh(C_{t}), \tag{6}\]  
where \(i_{t}\) represents the input gate, \(f_{t}\) is the forget gate, \(o_{t}\) is the output gate, \(\sigma\) and \(tanh\) are the sigmoid and hyperbolic tangent activation functions, respectively, \(C_{t}\) is the internal memory state, \(\widetilde{C}_{t}\) represents the cell state candidate, \(C_{t-1}\) is the state at a timestamp \(t-1\), \(h_{t-1}\) is the output of the previous LSTM cell at a timestamp \(t-1\), \(h_{t}\) is the output, \(x_{t}\) is the input, \(w_{C}\), \(w_{i}\), \(w_{f}\), and \(w_{o}\) are weights for the respective gates, and \(b_{C}\), \(b_{i}\), \(b_{f}\), and \(b_{o}\) are biases. The \(\odot\) symbol denotes the Hadamard product.  
An alternative solution to the problem of learning long-term dependencies is gated recurrent unit-based neural networks, originally introduced by [PERSON] et al. [7]. The gated recurrent unit is a simplified modification of the LSTM cell that does not store the internal state of the cell but instead uses an update gate and a reset gate to control the progress of the data transformation.  
Data processing using GRU cells is determined by the following equations:  
\[z_{t}\ =\ \sigma(w_{z}\cdot[h_{t-1},x_{t}]+b_{z}) \tag{7}\]  
\[r_{t}\ =\ \sigma(w_{r}\cdot[h_{t-1},x_{t}]+b_{r}) \tag{8}\]  
\[\widetilde{h}_{t}\ =\ tanh(w_{h}\cdot[r_{t}\odot h_{t-1},x_{t}]+b_{h}) \tag{9}\]  
\[h_{t}\ =\ (1-z_{t})\odot h_{t-1}+z_{t}\odot\widetilde{h}_{t} \tag{10}\]  
where \(z_{t}\) represents the update gate, \(r_{t}\) is the reset gate, \(\widetilde{h}_{t}\) is the output candidate, \(w_{z}\), \(w_{r}\), and \(w_{h}\) are weights for the respective gates, and \(b_{z}\), \(b_{r}\), and \(b_{h}\) are biases.  
The structures of individual cells that make up the neural networks of the corresponding type are shown in Figure 6.