# Power Grid Load Forecasting Using a CNN-LSTM Network Based on a Multi-Modal Attention Mechanism
## 4 Case Study
### Ablation Experiment  
To fully demonstrate the effectiveness of the proposed attention mechanisms, we conducted an ablation study on three datasets. Table 3 shows the results of the ablation study for each attention mechanism in the proposed model. By gradually incorporating multi-head self-attention (MHSA), global attention mechanism (GAM), and channel attention mechanism (CAM), the model's performance was significantly improved. Figure 9 visually presents the performance of the attention mechanisms on the three datasets. The ablation study indicates that each attention mechanism makes a unique contribution to the overall performance. Specifically, multi-head self-attention (MHSA) enhances the model's ability to capture temporal dependencies, the global attention mechanism (GAM) extracts global features, and the channel attention mechanism (CAM) effectively focuses on the importance of different channels, thereby optimizing feature representation. After gradually incorporating these attention mechanisms, key performance indicators (MAE, RMSE, and Max Error) showed significant improvements. This demonstrates that the proposed attention mechanisms work synergistically to significantly enhance the accuracy and robustness of power grid load forecasting. Further experiments revealed that the role of each attention mechanism is consistent across different datasets, confirming the universality and effectiveness of the proposed model. This also validates that combining multiple attention mechanisms for feature enhancement is an effective approach to improving the performance of complex time-series prediction models. To investigate the impact of the number of heads in the multi-head self-attention mechanism on model performance, we conducted multiple sets of experiments on three datasets, training and evaluating the model with different numbers of heads (1, 2, 4, 8, 16). The experimental results, as shown in Table 4, indicate that the choice of the number of heads significantly affects the model's prediction accuracy and computational efficiency. Figure 8: The load prediction results obtained from experiments involving the Household Dataset. (**a**) shows a comparison of weekly load prediction results, and (**b**) presents the load prediction results for one specific day. When the number of heads increased from 1 to 4, the model's prediction accuracy improved significantly. This suggests that the multi-head mechanism can capture more interfeature relationships, enhancing the model's global modeling capability. However, when the number of heads exceeded four, the performance improvement gradually plateaued. For instance, when the number of heads increased from 4 to 8, the performance gain was no longer significant. This is because an excessive number of heads can lead to model overfitting or increased model complexity, making the training process more challenging. As the number of heads increased, the computational overhead of the model also rose accordingly. In particular, when the number of heads increased from 8 to 16, the computation time increased significantly, but the performance improvement was limited. Selecting an appropriate number of heads is crucial for balancing model performance and computational efficiency. Our experimental results demonstrate that four heads generally provide good performance while maintaining reasonable computational costs. Although the proposed model combining multiple attention mechanisms has shown significant performance improvements in power load forecasting tasks, there are still some limitations and challenges. Firstly, as the number of attention mechanisms increases, the model's complexity and computational overhead also rise. In particular, in the multi-head self-attention mechanism, while increasing the number of heads can enhance the model's global modeling capability, it also leads to a significant increase in computation time, especially when the number of heads exceeds four, where performance improvements plateau while computational costs rise substantially.