# Load Forecasting Based on LVMD-DBFCM Load Curve Clustering and the CNN-IVIA-BLSTM Model
## 3 Methodology
### The Proposed CNN-IVIA-BLSTM Forecasting Model
#### 3.3.2 BLSTM Bidirectional Long Short-Term Memory Network  
BLSTM can remember a sequence well, can solve the dependency problem of longer time spans, and has a strong advantage in improving time series correlation. Based on the LSTM (long short-term memory) network, the forward and reverse network structures form a closed loop of information, which can better verify and correct the process error information while maintaining the bidirectional data information, which has stronger robustness [41]. The LSTM network structure is shown in Figure 10, and the relevant calculation equations are shown in (39)-(44).  
\[Input\ gate:\quad i_{t}=\sigma(W_{i}x_{t}+U_{i}h_{t-1}+b_{i}) \tag{39}\]  
\[Forget\ gate:\quad f_{t}=\sigma\Big{(}W_{f}x_{t}+U_{f}h_{t-1}+b_{f}\Big{)} \tag{40}\]  
\[Output\ gate:\quad o_{t}=\sigma(W_{0}x_{t}+U_{o}h_{t-1}+b_{o}) \tag{41}\]  
\[New\ memory\ unit:\ c_{t}t=\tanh(W_{c}x_{t}+U_{c}h_{t-1}+b_{c}) \tag{42}\]  
\[Final\ memory\ unit:\ c_{t}=f_{t}\otimes c_{t-1}+i_{t}\otimes c_{t}t \tag{43}\]  
\[Output:\quad h_{t}=o_{t}\otimes\tanh(c_{t}) \tag{44}\]  
where \(W_{i}\), \(W_{f}\), and \(W_{o}\) represent input weight vectors; \(U_{i}\), \(U_{f}\), and \(U_{o}\) represent upper output weight vectors; and \(b_{i}\), \(b_{f}\), \(b_{o}\), and \(b_{c}\) are bias vectors. Sigmoid is generally selected as the excitation function for \(\sigma\), which mainly plays a role of gating. It has an output between 0 and 1, which matches the physical definition of gating and is very close to 1 or 0 when the input is large or small, thus ensuring that the gate is open or closed. The tanh is an option to generate the new memory unit \({c_{t}}^{\prime}\) due to a faster convergence rate with an output between \(-1\) and 1, which coincides with the center of the feature distribution being 0 in most scenarios. The related formulas are given in Equations (45) and (46):  
\[sigmoid(x)=\frac{1}{1+e^{-x}} \tag{45}\]  
\[\tanh(x)=\frac{e^{x}-e^{-x}}{e^{x}+e^{-x}} \tag{46}\]  
Figure 9: Overall structure of a CNN.  
LSTM can only be passed in one direction, and the unit computation of the bidirectional neural network is connected with the unidirectional one. BLSTM consists of an input layer, forward LSTM layer, reverse LSTM layer, and output layer. However, the hidden layer of the bidirectional neural network must save two values, i.e., \(A\), which is involved in the forward calculation, and \(A\prime\), which is involved in the reverse calculation. The final output value depends on the sum. The BLSTM network structure is shown in Figure 11.  
However, a common problem is that BLSTM networks have many parameters, and without proper optimization, the model may be overfitted or slow to train, resulting in inefficiency. Too few nodes in the hidden layer will cause the model to not have the necessary learning ability and information processing capability, and too many nodes will increase the complexity of the network structure and make the learning process easily fall into local minima, which makes the network slow. When the learning rate is too high, the cost function is not easy to reduce to the lowest point, it is not easy to converge at the lowest point, and the convergence effect is poor. When too much training is performed, the gradient descent process may cross the nadir, which causes the training rate to be too low. In the case of reasonable tuning parameters, the more layers and neurons there are, the higher the accuracy rate; however, this can also lead to the overfitting phenomenon, and a regularization process can be used to solve the overfitting problem. In this paper, the number of implicit layer neurons, training times, learning rates, and regularization coefficients in BLSTM are optimized using the IVIA to improve the performance of the sequence modelling task.