## 2 Materials and Methods
### Long Short-Term Memory (LSTM) Network  
LSTM is a recurrent neural network that remembers the problem for longer, having a chain structure to repeat the module. Other networks repeat the module whenever the input receives new information. LSTM interacts in a particular way and contains four layers of neural networks. The data transfer process is the same as standard recurrent neural networks, while the information dissemination operation is different. As information passes through, the operation decides which information needs to be further processed and which is to be discarded. The main operation consists of cells and gates (Figure 5); the former contains various activations called sigmoids, containing certain values ranging from zero to one. They help to forget and retain information. If the data is multiplied by one, the value remains the same; if the data is multiplied by zero, the value becomes zero and disappears.  
There are three types of gates [14; 46; 47; 48; 49; 50]:  
* Forget gate: its function is to decide whether to keep or forget the information. Only information from previously hidden layers and current input remain with the sigmoid function. Any value closer to one will remain, while values closer to zero will disappear: \[\mathrm{f_{t}}=\sigma\Big{(}\mathrm{x_{t}}\mathrm{U^{f}}+\mathrm{h_{t-1}} \mathrm{W^{f}}\Big{)}\] (7) where \(\mathrm{x_{t}}\) is the input vector, \(\mathrm{h_{t-1}}\) are the output of the previous block, \(\mathrm{W}\) and \(\mathrm{U}\) the weight matrices of the hidden state and input respectively for each gate, \(\sigma\) is the sigmoid activation function.
* Input Gate: the front door helps to update the cell condition. Current input and previous state information go through the sigmoid function, which updates the value by multiplying it by zero and one. Likewise, for network regulation, data also pass through the \(\tanh\) function (Equation (9)); \(\mathrm{i_{t}}\) is the input gate vector. \[\mathrm{i_{t}}=\sigma\left(\mathrm{x_{t}}\mathrm{U^{i}}+\mathrm{h_{t-1}} \mathrm{W^{i}}\right)\] (8) \[\mathrm{C_{t*}}=\tanh\mathrm{h}\left(\mathrm{x_{t}}\mathrm{U^{g}}+\mathrm{h_{t -1}}\mathrm{W^{g}}\right)\] (9)  
The cell state vector aggregates the two components (old memory via the forget gate and new memory via the input gate)  
\[\mathrm{C_{t}}=\sigma\left(\mathrm{f_{t}}\mathrm{*C_{t-1}}+\mathrm{i_{t*}} \mathrm{C_{t*}}\right) \tag{10}\]  
Figure 5: LSTM unit architecture.  
C\({}_{t-1}\) is a memory from the previous block, C\({}_{t}\) is defined as a memory from the current block; the \"\(*\)\" operator is the Hadamard product. Output Gate: the next hidden state is set in the output gate. The sigmoid output has to be multiplied by the tanh function; the result of this multiplication decides which information the hidden state h_t should carry. This hidden state is used for the prediction. After, the new hidden state and cell state will move on to the next step:  
\[\mathrm{o_{t}}=\sigma\left(\mathrm{x_{t}U^{\circ}+h_{t-1}W^{\circ}}\right) \tag{11}\]  
\[\mathrm{h_{t}}=\mathrm{tanh}\left(\mathrm{C_{t}}\right)*\mathrm{o_{t}} \tag{12}\]  
where \(\mathrm{o_{t}}\) is the output gate vector, and \(\mathrm{h_{t}}\) the current block output. Table 2 illustrates the used training hyper-parameters for the LSTM neural network.