## 3 Results and Discussion  
### Prediction Results and Comparisons  
The Adam (Adaptive Moment Estimation) optimizer is among the most widely used optimization algorithms for training deep learning models. It combines the advantages of two other extensions of Stochastic Gradient Descent (SGD): AdaGrad and RMSProp. By adjusting the learning rate for each parameter individually, Adam accelerates the convergence of the training process and can enhance the overall performance. Therefore, Adam optimizer has been chosen for all DL models in this study. Two error indicators are used in this study to assess the performance of the proposed model and other models. They are MAE and MAPE and are defined as follows:  
\[MAE=\frac{\sum_{i=1}^{n}\left|y_{i}-x_{i}\right|}{n} \tag{3}\]  
\[MAPE=\frac{1}{n}\sum_{i=1}^{n}\left|\left(y_{i}-x_{i}\right)/x_{i}\right|*100 \tag{4}\]  
where \(y_{i}\) is the predicted value, \(x_{i}\) is the actual value at a given time, and \(n\) is the number of samples. First, as a benchmark to compare all models, Figure 3 shows the forecasting results using the DNN model. As shown, the electricity prediction using the DNN model only is not promising. Figure 3: Electricity demand prediction using the DNN model. \begin{table}
\begin{tabular}{l l} \hline \hline
**Hyper Parameter** & **Setting** \\ \hline Epochs & 500 \\ Batch Size & 128 \\ Activation Function & ReLU \\ FC Layer & 64 \\ LSTM Layer & 64 \\ Optimizer & Adam \\ Loss & MAE \\ Attention Type & Soft \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Configuration of the proposed model. Figure 4 illustrates the prediction results using hybrid CNN and LSTM models with and without an attention mechanism. The HDD and CDD are integrated into the hybrid prediction models. As shown, the integration of CNNs and LSTM networks in the proposed hybrid model has shown significant promise in capturing the complex temporal and spatial dependencies inherent in electricity demand forecasting. CNNs are particularly effective in extracting local patterns and spatial features from the input data, such as the variations in the HDD and CDD, which are critical in understanding the impact of temperature on energy consumption. By passing these extracted features into the LSTM layers, the model can capture long-term dependencies and sequential relationships within the data, which are essential for accurate forecasting over time. This dual approach ensures that both spatial and temporal aspects of the data are effectively utilized, leading to more accurate predictions than models relying solely on one type of neural network. Table 4 lists the performance of various models, including the baseline DNN, CNN, LSTM, and the proposed hybrid models, for the 2020 dataset. It offers a detailed comparison of MAE, MAPE, and the percentage improvement of the proposed method compared with other methods. First, the MAE and MAPE of CNN are 838 and 11.9, respectively; the MAE and MAPE of DNN are 807 and 11, respectively. Second, to enhance prediction accuracy, three hybrid models with two from CNN, DNN, and LSTM models are trained and compared. The MAE and MAPE of a hybrid \begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Model** & **MAE** & \
\begin{tabular}{c} **MAE Improvement** \\ **of the Proposed Model** \\ \end{tabular} & **MAPE** &
\begin{tabular}{c} **MAPE Improvement** \\ **of the Proposed Model** \\ \end{tabular} \\ \hline CNN & 838 & 85.44\% & 11.9 & 85.80\% \\ DNN & 807 & 84.88\% & 11 & 84.64\% \\ DNN and CNN & 153.3 & 20.42\% & 2.1 & 19.52\% \\ DNN and LSTM & 138 & 11.59\% & 1.89 & 10.58\% \\ CNN and LSTM & 135 & 9.63\% & 1.83 & 7.65\% \\ CNN and LSTM & 122 & \(\sim\) & 1.69 & \(\sim\) \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Evaluation metrics for the dataset of year 2020. Figure 4: Electricity demand predictions using CNN-LSTM and CNN-LSTM-Attention models.