## 2 Prediction of Wind-Power probability Density Based on Quantile Regression LSTM
### Network Model of Long- and Short-Term Memory  
The traditional quantile regression model can effectively reflect the influence of independent variables on the distribution range and shape of dependent variables, but it is not suitable for nonlinear new energy output prediction. [PERSON] proposed a neural network quantile regression model (QRNN) by combining the nonlinear fitting of the neural network with the data analysis ability of QR. Based on the training mechanism of the original recurrent neural network (RNN), the network training results are more inclined to new information but show weak memory function. To solve the long-term dependence on RNN training, [PERSON] and [PERSON] proposed the long-term and short-term memory network.  
As a variant of the recurrent neural network, LSTM has a basic structure, as shown in Figure 1. LSTM is represented as a chain structure by time expansion, and the repetitive module of LSTM contains four neural network layers. The hidden layer in the RNN is replaced by a memory unit in the LSTM to realize effective memory of past information.  
In the RNN network, the hidden layer has only one state, \(h\), and a cell state, \(c\), is added to the LSTM. Compared with the original RNN network, the LSTM is expanded according to the timeline, and the inputs are increased to three, including the cell state \(c_{t-1}\) at the previous time, the output \(h_{t-1}\) at the previous time, and the input \(x_{t}\) at the current network. The outputs are increased to two, namely, the output \(h_{t}\) of the LSTM at the current time and the cell state \(c_{t}\) at the current time.  
The LSTM has four gates in total. At each time, the LSTM receives the current time input \(x_{t}\), the previously hidden state output \(H_{t-1}\), and the memory cell state \(C_{t-1}\) through three gates.  
The first layer of neurons is the forgetting gate control layer, which helps LSTM decide to delete some information in the memory cell state, as shown in Formula (7).  
\[f_{t}=\sigma(W_{f}[h_{t-1},x_{t}]+b_{f}) \tag{7}\]  
Figure 1: Schematic diagram of LSTM gate control structure.  
The second layer and the third layers are the input gate and tanh layer, respectively, as shown in Formulas (8) and (9). Using the input gate, \(i_{t}\), to determine the new information, \(C_{t}^{\prime}\), to be stored in the new cell state, the tanh layer creates a new candidate value--\(C_{t}^{\prime}\).  
\[i_{t}=\sigma(W_{i}[h_{t-1},x_{i}]+b_{i}) \tag{8}\]  
\[C_{t}=\tanh(W_{C}[h_{t-1},x_{t}]+b_{C}) \tag{9}\]  
\[C_{t}^{\prime}=f_{t}^{*}C_{t-1}+i_{t}^{*}C_{t} \tag{10}\]  
where \(f_{t}^{*}C_{t-1}\) determines how much information is forgotten \(C_{t-1}\); \(i_{t}^{*}C_{t}\) determines how much information is stored in the new cell state \(C_{t}\).  
Finally, calculate \(\sigma\) through the output gate, \(H_{t}\). As shown in Formula (11), after Formula (10) passes through the tanh layer, it is multiplied by the output gate to bring the forgetting and memory parameters to the final output.  
\[O_{t}=\sigma(W_{o}[h_{t-1},x_{t}]+b_{o}) \tag{11}\]  
\[h_{t}=O_{t}\text{tanh}(C_{t}) \tag{12}\]  
where \(W_{f},\ b_{f},\ W_{i}b_{i}\) are the weight and offset of the forgetting gate and the input gate, respectively; \(W_{C},\ b_{C},\ W_{O},\ b_{O}\) are the updated value and the weight and offset of the output gate, respectively; \(\sigma(\cdot)\) is an activation function; \(\text{tanh}(\cdot)\) is a tangent activation function. Reference numeral \([h_{t-1},x_{t}]\) denotes a combination of matrices having the same number of rows.  
As a deep neural network with memory ability, LSTM has been widely used in nonlinear sequence prediction. Compared with a static neural network, LSTM can better explore the correlation between sequence data and is very suitable for dealing with problems highly related to time series [7; 8].