# Forecasting Steel Production in the World--Assessments Based on Shallow and Deep Neural Networks
## 3 Artificial Intelligence Predictive Models
### Recurrent Neural Networks  
Recurrent, or feedback, networks have their outputs fed back as input signals. This feedback works as a kind of state memory, thus making the networks very suitable to use to process time-varying systems. The signal travels over the network in two directions, having dynamic memory and the ability to represent states in dynamic systems. The present paper particularly focuses on recurrent neural networks known as long short-term memory (LSTM), whose representation is shown in the Figure 5 and gated recurrent unit (GRU).  
[PERSON] and [PERSON] [70] proposed the LSTM cell, which is a popular recurrent model. The LSTM has a capacity to remember, and is controlled by introducing a \"gate\" into the cell. Since this pioneering work, LSTMs were made popular and used by many researchers [71; 72]. According to [73], the internal calculation formula of the LSTM cell is defined as follows:  
\[f_{t}=\sigma(x_{t}W_{f}+h_{t-1}U_{f}+b_{f}) \tag{3}\]  
\[i_{t}=\sigma(x_{t}W_{i}+h_{t-1}U_{i}+b_{i}) \tag{4}\]  
\[o_{t}=\sigma(x_{t}W_{o}+h_{t-1}U_{o}+b_{o}) \tag{5}\]  
\[\mathbf{\tilde{C}_{t}}=tan[(x_{t}W_{C}+h_{t-1}U_{c}+b_{c}] \tag{6}\]  
\[C_{t}=\sigma(f_{t}\times C_{t-1}+i_{t}\times\tilde{C}_{t}) \tag{7}\]  
\[h_{t}=tanh(C_{t})\times o_{t}, \tag{8}\]  
where \(t\) is timestep, \(x_{t}\) is input to the current \(t\), \(W_{f}\) is weight associated with the input, \(h_{t-1}\) is the hidden state of the previous timestamp, \(U_{f}\) is the weight matrix associated with hidden state, \(W_{i}\) is weight matrix of input, \(U_{i}\) is weight matrix of input associated with hidden state, \(C_{t}\) is memory from current block, and \(h_{t}\) output of current block. The \(f_{t}\) is the  
Figure 4: Representation of a multilayer perceptron, with input X, one hidden layer, and output Y.  
forget gate, \(i_{t}\) is input gate, \(o_{t}\) is the output gate, \(C_{t}\) is new cell state, \(\tilde{\mathbf{C}}_{t}\) is values generated by tanh with all the possible values between \(-1\) and \(1\), and \(b_{f},b_{i},b_{o},b_{c}\) are the bias terms.  
The learning capacity of the LSTM cell is superior to that of the standard recurrent cell. However, the additional parameters add to the computational load. [PERSON] et al. (2016) introduced the gated recurring unit (GRU) model, which is a type of modified version of the LSTM.  
Figure 6 shows the internal architecture of a GRU unit cell. These are the mathematical functions used to control the locking mechanism in the GRU cell:  
\[z_{t}=\sigma(x_{t}W^{z}+h_{t-1}U^{z}+b_{z}) \tag{9}\]  
\[r_{t}=\sigma(x_{t}W^{r}+h_{t-1}U^{r}+b_{r}) \tag{10}\]  
\[\tilde{\mathbf{h}}_{t}=tan(x_{t}W_{h}+(r_{t}\times h_{t-1})U_{h}+b)_{h} \tag{11}\]  
\[h_{t}=(1-z_{t})\times\tilde{h}_{t}+z_{t}\times h_{t-1}, \tag{12}\]  
where \(x_{t}\) is input vector, \(h_{t}\) is output vector, \(\tilde{h}_{t}\) is candidate activation vector, \(z_{t}\) is update gate vector, \(r_{t}\) is reset gate vector and, \(W^{z},W^{r},W^{h}\) denote the weight matrices for the corresponding connected input vector. \(U^{z},U^{r},U\) represent the weight matrices of the previous time step, and \(b_{r},b_{z},b_{h}\) are biases.  
Figure 5: Architecture of an LSTM cell; adapted from ([PERSON] et al., 2016; [PERSON] et al., 2017; [PERSON] et al., 2018).  
Figure 6: Architecture of a GRU unit; adapted from ([PERSON] et al., 2016; [PERSON] et al., 2018).