## 2 Methodology
### Development of Integrated Energy Consumption Prediction Model
#### 2.2.2 Bilstm-Attention Method  
BiLSTM is a recursive neural network architecture, which is designed to capture sequential patterns in data. Unlike the traditional LSTM depicted in Figure 3, the BiLSTM processes input data in both forward and backward directions, enabling it to capture dependencies in both past and future contexts. This architecture consists of two LSTM layers: one for forward processing and the other for backward processing of the input sequence. The outputs from both directions are then concatenated or combined, providing the input sequence. BiLSTM, with its advantages in capturing temporal relationships and strong adaptability, is chosen in this study for energy consumption prediction [43].  
The forget gate is responsible for selectively discarding certain information from the previous time step, while the input gate determines which new information can be passed to the next state [44]. The output gate specifies the output from the current state to the next. In LSTM, these three key components (forget gate, input gate, and output gate) collaborate to precisely control the flow of information.  
\[i_{t}=\sigma(W_{xi}x_{t}+W_{hi}h_{t-1}+b_{i}) \tag{13}\] \[f_{t}=\sigma(W_{xf}x_{t}+W_{hf}h_{t-1}+b_{f})\] (14) \[o_{t}=\sigma(W_{xo}x_{t}+W_{ho}h_{t-1}+b_{o})\] (15) \[\tilde{c}_{t}=\tanh(W_{xc}x_{t}+W_{ho}h_{t-1}+b_{c})\] (16) \[c_{t}=f_{t}\odot c_{t-1}+i_{t}\odot\tilde{c}_{t}\] (17) \[h_{t}=o_{t}\odot\tanh(c_{t}) \tag{18}\]  
In the equation: \(i_{t}\), \(f_{t}\), and \(o_{t}\) represent the input gate, forget gate, and output gate, respectively; \(x_{t}\) is the input vector at the current time step; \(\sigma\) denotes the sigmoid function; \(\tanh\) is the hyperbolic tangent activation function; \(W_{xi}\), \(W_{xf}\), \(W_{xo}\), and \(W_{xc}\) are the weight matrices from the input layer to different gate mechanisms; \(b_{f}\), \(b_{i}\), \(b_{o}\), and \(b_{c}\) are bias vectors for different gate mechanisms; \(W_{hi}\), \(W_{hf}\), \(W_{ho}\), and \(W_{hc}\) are the weight matrices from the hidden layer to different gate mechanisms; \(c_{t}\) and \(c_{t-1}\) are the storage unit information at  
Figure 3: The structure of one LSTM neuron.  
the current time t and the previous time, respectively; \(\overset{\sim}{c}_{t}\) is the cell state value at the current time step without using the input gate and forget gate for information adjustment; and \(\odot\) denotes element-wise multiplication.  
BiLSTM is a combination of forward and backward LSTMs, considering historical data in both positive and negative temporal directions. It is equivalent to adding a backward LSTM, allowing the hidden layer to read future sequence information from the input. By incorporating information from both forward and backward directions in the computation, BiLSTM can effectively capture patterns in data bidirectionally. The structure is illustrated in Figure 4, where \(\overset{\leftarrow}{h}\) and \(\overset{\rightarrow}{h}\) represent the sequences processed in the forward and backward directions, respectively. The output of BiLSTM can be expressed as follows:  
\[h_{t}=(H_{c,t-1},H_{c,t}),t\in[1,i] \tag{19}\]  
In Equation (19), \(H_{c,t}\) represents the output of the CNN layer at time t.  
Attention Mechanism (AM) is a commonly used mechanism in deep learning models. Its primary objective is to enhance the model's focus on different parts of the input sequence, enabling the model to more accurately learn and comprehend input information [45]. Attention achieves this by assigning different weights to feature vectors, concentrating attention to highlight key features, and obtaining better results [46]. The structure of the Attention Mechanism is illustrated in Figure 5. Here, \(x_{t}(t\in[1,n])\) represents the input of the BiLSTM network, \(h_{t}(t\in[1,n])\) corresponds to the hidden layer output obtained through BiLSTM for each input, and y is the output of the BiLSTM with the incorporated Attention Mechanism.