# Anticipating Future Behavior of an Industrial Press Using LSTM Networks
## 3 Theoretical Background  
The present work uses LSTM networks, considering the referred different studies showing their usefulness for time series predictions [48; 49]. The LSTM is a deep learning recurrent neural network architecture that is a variation of traditional recurrent neural networks (RNNs). It was introduced by [PERSON] and [PERSON] in 1997. The most popular version is a modification refined by many works in the literature [50; 51], which is called vanilla LSTM (hereinafter referred to as LSTM). The LSTM is excellent at handling time series data only with its network parameters. For example, weights and polarization are adjusted or optimized [52]. The primary modification of the LSTM when compared to the RNN architecture is the structure of the hidden layer [53]. The LSTM model is a powerful type of recurrent neural network (RNN), capable of learning long-term dependencies [54]. They became popular due to their power of representation and effectiveness in capturing long-term dependencies [55]. Many networks showed instability when dealing with exploding or vanishing gradient problems during learning. Those problems happen when the gradient of the error is too large or too small. If it is too large, it overflows and the errors cannot propagate properly through different layers during learning. If it is too small, it vanishes and the network does not learn. Different methods were proposed to solve those problems, known as a kind of of \"door control\" that is used in RNN models. For example, Gated Recurrent unit (GRU) algorithms [56; 57], as the LSTMs [58; 59], are to a large extent immune to the gradient problems and learn well. The LSTM network structure is based on three ports whose function is to regulate the flow. Those ports are called the entrance door, the forget gate, and the exit door. The main port of entry is to regulate the entry of new memory data; the forget gate has the function of regulating the storage time in the network memory and the output port intends to regulate how much the value retained in memory influences the activation of the output block [60]. [PERSON] et al. demonstrate some relevant conclusions such as (1) LSTM has a good predictive capacity; (2) their use can significantly improve the profit of service providers, so there is an opportunity when it comes to exploring the forecast in real time [61]. LSTM networks are the _de facto_ gold standard for deep learning algorithms for analyzing time series data [55]. Figure 1 shows the internal architecture of an LSTM unit cell. According to [62; 63], the internal calculation formulae of the LSTM unit are defined as follows:  
\[i_{t}=\sigma(x_{t}U^{i}+h_{t-1}W^{i}+b_{i}) \tag{1}\]  
\[f_{t}=\sigma(x_{t}U^{f}+h_{t-1}W^{f}+b_{f}) \tag{2}\]  
\[o_{t}=\sigma(x_{t}U^{o}+h_{t-1}W^{o}+b_{o}) \tag{3}\]  
\[a_{t}=tan(x_{t}U^{C}+h_{t-1}W^{C}+b_{C}) \tag{4}\]  
where \(U^{i},U^{f},U^{o}\) and \(U^{C}\)are the weight matrices for mapping the current input layer on three ports and the state of the current input cell. \(W^{i},W^{f},W^{o}\) and \(W^{C}\) are the weight matrices for mapping the previous output layer on three ports and the current state of the input cell. \(b_{f},b_{i},b_{o}\), and \(b_{c}\) are polarization vectors for calculating the state of the door and the input cell. \(\sigma\) is the gate activation function, which is normally a sigmoid function. \(tan\) is the hyperbolic tangent function which is the activation function for the current state of the input cell. Then, the current state of the output cell and the output layer can be calculated using the following equations. \[C_{t}=\sigma(f_{t}\times C_{t-1}+i_{t}\times a_{t}) \tag{5}\]  
\[h_{t}=tanh(C_{t})\times o_{t} \tag{6}\]  
To assess the quality of the prediction model, one of the most popular metrics is the Root Mean Square Error (RMSE), which is given by Equation (7):  
\[RMSE=\sqrt{\frac{1}{n}\sum_{t=1}^{n}(Y_{t}-\hat{Y})^{2}} \tag{7}\]  
where \(Y_{t}\) represents the desired (real) value and \(\hat{Y}\) is the predicted (obtained from the model) value.