# Power Grid Load Forecasting Using a CNN-LSTM Network Based on a Multi-Modal Attention Mechanism
## 4 Case Study
### Comparative Experiments  
To fully demonstrate the effectiveness of our proposed model, we conducted a comprehensive comparison with various mainstream forecasting models, including traditional single models (e.g., ANN, RNN, LSTM, BiLSTM, and GRU) as well as representative hybrid forecasting models (e.g., CNN-LSTM, CNN-RNN) and some recently proposed advanced hybrid models such as SBACT-Seq2 Seq [46] and EWT-CNN-S-RNN+LSTM [47]. These models have been widely applied in the field of time series forecasting, ranging from simple neural networks to complex deep learning architectures. Specifically, ANN (Artificial Neural Network), as a basic neural network model, can extract fundamental patterns from \begin{table}
\begin{tabular}{c c c} \hline \hline
**Layer Category** & **Neurons** & **Remark** \\ \hline Input layer & 24 \(\times\) 9 & \\ Convolution layer & 24 \(\times\) 64 & Kernel 1 \(\times\) 6/1 \\ Pooling layer & 24 \(\times\) 64 & Max-pooling 1 \(\times\) 6/1 \\ Dropout layer & 24 \(\times\) 64 & Retention 0.8 \\ Multi-head self-attention layer & 24 \(\times\) 128 & Heads 4 \\ LSTM layer & 24 \(\times\) 128 & \\ Global attention layer & 24 \(\times\) 128 & Softmax \\ Channel attention layer & 24 \(\times\) 128 & \\ Flatten layer & 1 \(\times\) 5120 & \\ Output layer & 1 \(\times\) 24 & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The specific structure parameters of this model. time series data for prediction but struggles to handle complex dynamic load data due to its limited ability to model temporal dependencies. LSTM (Long Short-Term Memory) and BiLSTM (Bidirectional Long Short-Term Memory), as variants of RNN (Recurrent Neural Network), utilize memory cells to capture long-term dependencies in time series. BiLSTM, in particular, models both forward and backward temporal information, providing a more comprehensive representation of time series. GRU (Gated Recurrent Unit), a lightweight version of LSTM, reduces computational complexity by simplifying its structure while maintaining a comparable performance in some scenarios. In contrast, RNN, as a foundational model for recurrent neural networks, often underperforms compared to LSTM and GRU due to the vanishing gradient problem. In terms of hybrid models, CNN-LSTM combines the spatial feature extraction capability of Convolutional Neural Networks (CNNs) with the temporal modeling capability of LSTM, making it a classic spatiotemporal hybrid forecasting model. The CNN component excels at extracting local spatial features, while the LSTM component captures the temporal dependencies of these features. Similarly, CNN-RNN integrates CNN with RNN, enhancing its ability to predict complex time series data to some extent. SBACT-Seq2 Seq employs a hybrid forecasting approach that integrates XGBoost and SVMD while optimizing the Seq2 Seq model, significantly improving the accuracy and practicality of power load forecasting. EWT-CNN-S-RNN+LSTM enhances short-term power load forecasting accuracy and stability by using EWT decomposition for feature extraction and Bayesian optimization for parameter tuning. However, these hybrid models generally face two major challenges: (1) high model complexity, which increases the risk of overfitting, and (2) limitations in capturing long-term dependencies and key features due to constraints in the structural design of individual components. In contrast, the model proposed in this paper combines CNN and LSTM with a multi-modal attention mechanism, effectively overcoming the shortcomings of traditional models in spatial feature extraction and time series modeling. Additionally, by introducing multi-head self-attention, global attention, and channel attention mechanisms, the model significantly enhances its ability to capture dynamic features and global dependencies. Furthermore, the model achieves a balance between model complexity and forecasting accuracy. Table 2 shows the forecasting performance of our model compared to other models on three datasets.