## 4 Methodology
### Long Short-Term Memory Network  
The LSTM network, a type of recurrent network, can be viewed as the first choice deep learning architecture for time series prediction due to its ability to learn long-term dependencies [51]. The basic unit in LSTM networks is called a LSTM cell, which remembers values for arbitrary time intervals, while three filters regulate the information flow into and out of the cell [52]. This feature arises with the help of three logic gates: input gate, output gate, and forget gate. Each of these gates obtains input from a sigmoid activation function (\(s\)). Here and later we use the following definitions: \(Wt_{j}\) is a weight matrix, \(bs_{j}\) is a bias vector (\(j=f,i,c,o\)) and \(x_{t}\) is a vector containing input values (the subscript \(t\) indexes the time step). Forget gates decide what information to discard from a previous state of the cell by assigning a previous state, compared to a current input, a value between 0 and 1. This gate is defined as  
\[f_{t}=s(Wt_{f}[h_{t-1},x_{t}]+bs_{f}), \tag{1}\]  
where \(h\) is the number of LSTM cell's units. Input gates decide which pieces of new information to store in the current state, using the same system as forget gates. The functionality of the input gate can be described with two parts: \(i_{t}\) and \(\zeta_{t}\). Here, \(i_{t}\) is information we want to remember, which we compute using a sigmoid function (\(s\)) as follows:  
\[i_{t}=s(Wt_{i}[h_{t-1},x_{t}]+bs_{i}). \tag{2}\]  
To create a vector of new updated values \(\zeta_{t}\), we use a hyperbolic tangent function (\(\sigma\)), i.e.,\[\tilde{c}_{t}=\sigma(Wt_{c}[h_{t-1},x_{t}]+bs_{c}). \tag{3}\]  
Then, the state of the old cell \(c_{t-1}\) is replaced by a new cell state by removing the information generated by the forget gate in Equation (1):  
\[c_{t}=f_{t}\odot c_{t-1}+\dot{t}_{t}\odot\tilde{c}_{t}, \tag{4}\]  
where \(c_{t}\) is the updated cell state. Here, the operator \(\odot\) denotes the Hadamard product (element-wise product). Finally, output gates control which pieces of information in the current state to output by assigning a value from 0 to 1 to the information, considering the previous and current states. The output is passed from a sigmoid layer and then a hyperbolic tangent layer to classify. \[o_{t} =s(Wt_{o}[h_{t-1},x_{t}]+bs_{o}),\] \[h_{t} =o_{t}\odot\sigma(c_{t}), \tag{5}\]  
where \(o_{t}\in(0,1)^{h}\) is an output gate's activation vector and \(h_{t}\in(-1,1)^{h}\) is a hidden state vector, also known as an output vector of the LSTM unit, and the superscript \(h\) refers to the number of hidden units. Note that sigmoid activation function (\(s\)) suppresses the values in the interval \((0,1)\), while the hyperbolic tangent activation function (\(\sigma\)) suppresses the values in the interval \((-1,1)\). After the above steps, the cell state is updated. Lastly, the output of the current state is computed by taking the values of the updated state of the cell state and also values from the sigmoid layer that determine the components of the cell state that need to be included in the output. The architecture of LSTM network used in our study is shown in Figure 3. We used bidirectional version of this recurrent network, it makes a forward pass through time and a backward one with different hidden states and learnable matrices. The model includes a linear input layer of size \(1\times 200\) with the Relu activation function. There are also two LSTM cells, each containing 200 neurons, and the output from these cells, after dropout layer with probability to replace some random elements equaling 0.2 in training, goes to a linear layer with a weight matrix of size \(400\times 1\). This final output allows us to obtain the predicted value. Also, we used 1500 steps as length of the input window (the length before we bring hidden state to zero). Due to their complicated structure, state-of-the-art LSTM networks include a huge number of model parameters, typically exceeding the usual on-chip memory capacity.