# Power Grid Load Forecasting Using a CNN-LSTM Network Based on a Multi-Modal Attention Mechanism
## 3 Methodology
### CNN Module  
The main goal of Convolutional Neural Networks (CNNs) is to extract significant features from input data [16; 17]. CNNs are primarily composed of convolutional layers and pooling layers. Due to the local connectivity and weight-sharing characteristics of convolutional layers, the key objective of CNNs is the automatic extraction of key features and dimensionality reduction [39; 40]. To enhance the depth of the model and improve the ability to extract abstract features, the proposed model alternates between convolutional and pooling layers in the CNN module, forming the feature extraction layer. Because the fixed connection pattern of CNNs somewhat limits the number of feature extraction layers, typically, 1 to 3 feature extraction layers are used in short-term load forecasting, with the specific number of layers determined based on the model's prediction performance. Convolutional layers effectively extract complex dynamic features and local non-linear features from historical load data using different convolutional kernels [41; 42]. Meanwhile, pooling layers reduce the data dimension through max pooling and average pooling operations, preserving more meaningful information by compressing the extracted features. Max pooling can extract the most significant local features from time-series data, such as sharp fluctuations or sudden events in load variations over the short term. This helps the model focus on key data changes at critical time points, thereby avoiding excessive interference from noise. Meanwhile, average pooling smooths the time-series data by pooling the average value within local time windows, helping the model capture the overall trend and long-term patterns in the data, thereby reducing the impact of short-term fluctuations on predictions. The combination of these pooling strategies allows the model to retain important dynamic information while balancing noise and trends, improving  
Figure 1: A schematic diagram of a CNN-LSTM network based on a multi-modal attention mechanism. the feature extraction capability from time-series data and enhancing the accuracy and robustness of load forecasting. This module employs the max pooling method, aiming to retain more fluctuation information, thereby improving the model's generalization ability. However, CNNs are not effective at capturing the temporal dependencies in time-series data. Therefore, to enhance the model's performance, Recurrent Neural Network (RNN) techniques must be incorporated, combining CNN with Long Short-Term Memory (LSTM) networks. The combination of LSTM (Long Short-Term Memory) with CNN is primarily recommended due to the fact that CNN excels at extracting local features and patterns from data, but it performs poorly in capturing the long-term dependencies and temporal relationships in time-series data. CNN effectively extracts local dynamic features and non-linear patterns from time-series data through convolutional operations, and in particular, it excels at capturing short-term fluctuations and sudden events. However, power grid load data typically exhibit long-term temporal dependencies, meaning that the current load state is often influenced by the load history and trends from a previous period. In this case, LSTM is well suited to handling such long-term dependencies. Through its gating mechanisms, LSTM can retain past information and capture long-term patterns and trends over extended periods. Therefore, combining CNN with LSTM allows the strengths of CNN in local feature extraction to be integrated with LSTMs' strengths in temporal modeling, enabling the model to effectively capture both local features and long-term dependencies in load data. This enhances the accuracy and robustness of power grid load forecasting. Taking one feature extraction layer as an example, the preprocessed historical load sequence is input into the CNN module, denoted as \(f_{in}=[f_{in,1},\cdots,f_{in,t-1},f_{in,t},\cdots,f_{in,n}]\).