# Prophet-CEEMDAN-ARBiLSTM-Based Model for Short-Term Load Forecasting
## 2 Method
### Advanced Residual BiLSTM Block  
The paper adopts an advanced residual BiLSTM (ARBiLSTM) block to capture temporal information influencing load variations from features extracted by Prophet and CEEMDAN. The ARBiLSTM block integrates batch normalization, a BiLSTM layer, and PReLU activation, as presented in Figure 2. The ARBiLSTM block can enhance its approximation capabilities by replacing ReLU with PReLU. In addition, the ARBiLSTM block adopts the shortcut connections, which can alleviate the degradation of the model and make the ARBiLSTM block surpass traditional BiLSTM-based methods. The following section will detail the theory of BiLSTM and further explain the residual structure and PReLU activation.  
In contrast to LSTM, the BiLSTM method facilitates bidirectional data flow, featuring two distinct hidden layers that connect to the inputs and outputs [1]. One hidden layer in BiLSTM comprises a forward LSTM, which transfers data from the input load series to the output load series. Simultaneously, a backward LSTM transfers future information from the output load series to the input load series. This process extracts long-term dependencies and enhances forecasting performance [1]. The hidden state of Bi-LSTM at time \(t\) includes the forward \(h_{t}^{f}\) and the backward \(h_{t}^{b}\). In addition, \(H_{t}\) represents the ultimate hidden output of the BiLSTM layer, derived from the amalgamation of the forward and backward outputs. The updating formulas for \(h_{t}^{f}\), \(h_{t}^{b}\), and \(H_{t}\) are as follows.  
\[h_{t}^{f}=\text{LSTM}^{f}(h_{t-1},x_{t},c_{t-1}),t\in[1,T], \tag{10}\]  
\[h_{t}^{b}=\text{LSTM}^{b}(h_{t-1},x_{t},c_{t-1}),t\in[1,T], \tag{11}\]  
\[H_{t}=\left[h_{t}^{f},h_{t}^{b}\right]. \tag{12}\]  
The design of the residual architecture effectively mitigates a notable challenge in deep learning methods known as the degradation problem. This issue arises when increasing network depth fails to improve forecasting performance beyond a certain threshold. The introduction of \"shortcut connections\" allows the model to skip certain layers without adding parameters or computational complexity. When network degradation occurs, the model not only preserves performance equivalent to shallower depths but also effectively mitigates degradation.  
PReLU function serves as an enhanced activation layer. Compared with the widely adopted ReLU, PReLU enhances the approximation capabilities of the methods and further relieves model degradation. The formulation of ReLU \(f_{\text{R}}\) is defined as:  
\[y=f_{\text{R}}(x)=max(0,x), \tag{13}\]  
where \(x\) and \(y\) represent the inputs and outputs, respectively. Nevertheless, due to the risk of neuron deactivation, ReLU may be vulnerable, potentially resulting in non-updatable gradient states and exacerbating degradation. Thus, the PreLU is designed to address the above issue. The formula for PReLU \(f_{\text{R}}\) is defined as:  
\[y_{i}=f_{\text{PR}}(x_{i})=max(0,x_{i})+\beta_{i}min(0,x_{i}), \tag{14}\]  
Figure 2: Advanced residual BiLSTM (ARBiLSTM): BN, batch normalization; BiLSTM, Bidirectional Long Short-Term Memory layer.  
where \(x_{i}\) and \(y_{i}\) represent the inputs and outputs, respectively. \(\beta_{i}\) is the trainable parameter between 0 and 1, co-updated with other layers, contributing to the convergence of the loss function.  
Compared with ReLU, PReLU incorporates non-zero slopes on the negative axis, mitigating the risk of neuron inactivation and concurrently improving the generalization ability. Moreover, PreLU incorporates a negligible number of parameters in the method, serving as a preventive measure against overfitting issues.  
In summary, the application of PReLU within the shortcut connection can boost the learning ability of the ARBiLSTM block. In addition, the introduction of a residual structure effectively alleviates model degradation. By replacing ReLU with PReLU, the ARBiLSTM block integrates an activation layer that enhances the model's generalization ability.