# Direct Multistep Wind Speed Forecasting Using LSTM Neural Network Combining EEMD and Fuzzy Entropy
## 2 Methodology
### RNN and LSTMNN  
LSTMNN is developed on the basis of Recurrent Neuron Network (RNN). Unlike ordinary neural networks, RNN adds a self-connected hidden layer spanning time steps. Therefore, RNN can memorize the input information in front of the time series and apply it to the calculation of the current output.  
A simple RNN consists of three layers: an input layer, a hidden layer and an output layer. Given \(m\) as the number of forecasting steps of direct multistep forecasting, the structure diagram of the RNN forecasting model for wind speed series is shown in Figure 1. In this figure, {\(v_{i}\), \(i\) = 1, 2,..., \(t\)} is the measured input wind speed; {\(V_{i+m}\), \(i\) = 1, 2,..., \(t\)} is the forecasted output wind speed; \(a_{i}\) is the activation value from time-step \(i\); \(W_{ax}\), \(W_{ux}\) and \(W_{ax}\) are the connection weights between neurons, which are the same at each time step.  
In the RNN model, the hidden layers of the front and back time steps are connected. It can make full use of the information of the wind speed series before the forecasted time, thereby improving the forecasting accuracy. However, the RNN model may have the vanishing gradient problem during the training process [37]. The LSTMNN proposed by [PERSON] and [PERSON] [38] can solve this problem of the RNN model through its special structural design. Moreover, the LSTMNN can better handle the long-term dependence of time series and effectively utilize the historical input information of time series.  
The basic LSTMNN also consists of three layers: an input layer, a hidden layer and an output layer. But compared with the RNN, the hidden layer of the LSTMNN adds some threshold units for controlling information transfer, which makes the neural network have a unique memory mode [38; 39]. The structures of the RNN forecasting model and the LSTMNN forecasting model at the time-step \(i\) are shown in Figure 2.  
In the forward propagation process of the LSTMNN forecasting model, besides the activation value \(a_{i}\), a memory cell \(c_{i}\) is transferred from the previous hidden layer to the latter. In addition, three gate structures are added to the hidden layer in LSTMNN, namely forget gate, update gate, and output gate. The three gate structures can control the preservation, reading and modification of the memory cell in the LSTMNN forecasting model.  
Figure 1: The structure of the RNN forecasting model for wind speed series.  
In the forward propagation process of the LSTMNN forecasting model, the forecasted output wind speed of the time-step \(i\) is calculated as follows:  
\[\widetilde{c}_{i}=\tanh(W_{\alpha\cdot}a_{i-1}+W_{\alpha v\cdot}v_{i}+b_{c}) \tag{2}\]  
\[f_{i}=\sigma\Big{(}W_{fa}\cdot a_{i-1}+W_{fv}\cdot v_{i}+b_{f}\Big{)} \tag{3}\]  
\[u_{i}=\sigma(W_{ua}\cdot a_{i-1}+W_{uv}\cdot v_{i}+b_{u}) \tag{4}\]  
\[o_{i}=\sigma(W_{\alpha\cdot}a_{i-1}+W_{\alpha v}\cdot v_{i}+b_{o}) \tag{5}\]  
\[c_{i}=f_{i}c_{i-1}+u_{i}\widetilde{c}_{i} \tag{6}\]  
\[a_{i}=o_{i}\tanh c_{i} \tag{7}\]  
\[V_{i+m}=\sigma(W_{va}\cdot a_{i}+b_{v}) \tag{8}\]  
where the subscript \(i\) represents the time-step \(i\); \(\widetilde{c}_{i}\) is the candidate for replacing the memory cell; \(c_{i}\) is the memory cell; \(f_{i}\), \(u_{i}\) and \(o_{i}\) are the values of forget gate, update gate and output gate, respectively; \(\sigma\) is the sigmoid function; \(W_{\alpha}\), \(W_{\alpha v}\), \(W_{fu}\), \(W_{fu}\), \(W_{ua}\), \(W_{uv}\), \(W_{\alpha u}\), \(W_{\alpha v}\), \(W_{va}\) are weight matrices; \(b_{c}\), \(b_{f}\), \(b_{u}\), \(b_{o}\), \(b_{v}\) are bias vectors.  
After the forward propagation process of the LSTMNN forecasting model is completed, the back propagation process is followed. The model is firstly expanded into a deep network in chronological order. Then, the BPTT algorithm [40] and chain rule are used to iteratively update the connection weights and thresholds in the model until the optimal solution is obtained.