## 4 Results
### Comparative Prediction Results  
In order to compare the accuracy of several prediction models, the normalized mean square error (NMSE), the median square error (NMDSE) and coefficient of determination (\(R^{2}\)) are selected as the evaluation of the performance of hyper-parameters, the calculation formula is as follows:  
\[NMSE=mean[(\hat{y_{t}}-y_{t})^{2}/y_{t}{}^{2}] \tag{21}\]  
\[NMDSE=median[(\hat{y_{t}}-y_{t})^{2}/y_{t}{}^{2}] \tag{22}\]  
\[R^{2}=1-\sum(\hat{y_{t}}-y_{t})^{2}/\sum(y_{t}-\hat{y_{t}})^{2} \tag{23}\]  
where \(y_{t}\) is the actual power consumption at time \(t\), \(\hat{y_{t}}\) is the predicted power consumption at time \(t\). Finally, the values of \(R^{2}\) in Table 2 of models are close to 1, which means that all of these models achieve an excellent fitting effect. BO-PSO performs better than others methods in the view of \(R^{2}\). A visual comparison of the values of NMSE is shown in Figure 5.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Hyper-Parameters** & **Type** & **Range** \\ \hline Feature length & Integer & \((5,50)\) \\ Number of network units & Integer & \((5,50)\) \\ Batch size of training data & Integer & \((32,4096)\) \\ \hline \hline \end{tabular}
\end{table}
Table 1: Type and range of hyper-parameters.  
Figure 4: Flow chart of load forecasting.  
It can be seen that after optimizing the model, the NMSE value of the BO-PSO method, whether the maximum value, the minimum value or the average value, is smaller than that of the other two methods, which shows that the BO-PSO method is effective in this experiment and is better than the other two methods. After comparing the NMSE and \(R^{2}\) values of the three methods, the final hyper-parameter (Feature length, Number of network units, Batch size of training data) values are shown in Table 3.  
To show the comparison of the effects of the three methods more intuitively, the hyper-parameters in Table 3 are used to train the RNN and LSTM models. The top 300 predictions for the test set using the proposed method and the comparison method as shown in Figures 6 and 7, it can be seen that there is an obvious fluctuation law of power load and the prediction results are closest to the true values. At the same time, the point-by-point prediction error of each model is shown. It can be observed that the prediction curves fit well for these six models. The error curves of RNN-BO-PSO and LSTM-BO-PSO fluctuate  
\begin{table}
\begin{tabular}{c c} \hline \hline
**Model** & \(\mathbf{R^{2}}\) \\ \hline RNN-BO-PSO & 0.9909 \\ RNN-BO-L-BFGS-B & 0.9908 \\ RNN-BO-TNC & 0.9907 \\ LSTM-BO-PSO & 0.9951 \\ LSTM-BO-L-BFGS-B & 0.9945 \\ LSTM-BO-TNC & 0.9948 \\ \hline \hline \end{tabular}
\end{table}
Table 2: The value of \(R^{2}\) for different models.  
Figure 5: NMSE box line diagram for different models. (**a**) Value of NMSE of RNN models for each method. (**b**) Value of NMSE of LSTM models for each method.  
\begin{table}
\begin{tabular}{c c c c} \hline \hline
**Model** & **Feature Length** & **Number of Network Units** & **Batch Size of Training Data** \\ \hline RNN-BO-L-BFGS-B & 44 & 34 & 64 \\ RNN-BO-TNC & 49 & 34 & 64 \\ RNN-BO-PSO & 47 & 38 & 64 \\ LSTM-BO-L-BFGS-B & 23 & 33 & 64 \\ LSTM-BO-TNC & 23 & 32 & 64 \\ LSTM-BO-PSO & 45 & 38 & 32 \\ \hline \hline \end{tabular}
\end{table}
Table 3: The hyper-parameters of each network.  
smoothly. The error of RNN-BO-PSO and LSTM-BO-PSO are obviously less than the other models in Figures 6 and 7. After calculation, the NMSE value and NMDSE value of models are obtained, as shown in Table 4. Figure 8 shows the comparison of the two indicators, respectively. As can be seen from the chart, the two models optimized based on the BO-PSO method have the smallest NMSE and NMDSE. Thus, it can be seen that the optimization method in this paper can not only replace the manual selection of hyper-parameters, but also improve the performance of the model. From the results of Figures 6-8 and Table 4, we can see that BO-PSO is an effective hyper-parameter optimization method.