gradient with fast convergence speed. The convolution layers are followed by a maximum one-dimensional sampling (MaxPooling1D) layer, three BiLSTM hidden layers, a dropout layer, and a dense layer. The one-dimensional CNN layer is mainly adopted for the sequential data processing of the load demand. The sampling size of MaxPooling1D is 1. The number of features of the BiLSTM hidden layers is 150, 50, and 32, respectively. The proposed CBiLSTM takes advantage of the nonlinear fitting ability from BiLSTM and the feature extraction ability from CNN. The final structure of the SG-CBiLSTM model is resumed in Table 4.3

<table><caption>Table 4.3: Layer description of CBiLSTM [4].</caption><thead><tr><th>Layer</th><th>Configuration/Neurons/Filters</th><th>Activation function</th></tr></thead><tbody><tr><td>Input layer</td><td>Sequential</td><td>-</td></tr><tr><td>Convolution 1-D</td><td>32, k1×k1</td><td>ReLU</td></tr><tr><td>Convolution 1-D</td><td>64, k1×k1</td><td>ReLU</td></tr><tr><td>Convolution 1-D</td><td>64, k1×k1</td><td>ReLU</td></tr><tr><td>MaxPooling 1-D</td><td>k1×k1</td><td>ReLU</td></tr><tr><td>Flatten</td><td>64</td><td>-</td></tr><tr><td>BiLSTM</td><td>150</td><td>ReLU</td></tr><tr><td>BiLSTM</td><td>50</td><td>ELU</td></tr><tr><td>BiLSTM</td><td>32</td><td>ELU</td></tr><tr><td>Fully connected</td><td>1</td><td>Softmax</td></tr></tbody></table>

The general architecture of reference models is fixed through repeated training. While, the units' number, the batch size, and the activation function are selected using the ES method.

## Experimental results

The feasibility of the proposed methodology is depicted in this part. The numerical outcomes at this level are initially led by introducing the exhibition of the proposed structure independently and with several contextual analyses of various SG filter configurations. Second, a comparative study is computed and presented where the computational configurations are unified for the sake of reasonable assessment. The conclusive remarks and interpretations are based on the real results simulated in this work. During the approval analysis of this investigation, the selection of the most appropriate Polynomial (P) degree is based on the trial and error method of 4 P values. For the consistency and coherence of this section, the validation
of our proposed SG-CBiLSTM model is initially led compared with the single CBiLSTM using the proposed framework explained in the previous section. Secondly, the supremacy of the proposed method was verified compared with various reference models. To yield believable outcomes, each case study is repeated 10 times and the results are averaged. The experimental outcomes as demonstrated in Fig. 4.9.

<figure><img src="image_46.png" /><figcaption>Figure 4.9: Performance comparison of SG-CBiLSTM model with a variety of polynomial filters p={3,5,7,10} [4].</figcaption></figure>

As indicated by Fig. 4.9, the proposed SG-CBiLSTM can trace the actual load demand with high accuracy by only utilizing the temperature and the synthetic data. Further, the SG filter verified its capacity to enhance the performance of the prediction framework. As shown in Fig. 4.9, the SG-CBiLSTM method achieved the best accuracy for P=10 where the difference between the estimated curve and the real curve is the littlest compared to the single CBiLSTM and the other models for different P values. Nonetheless, the polynomial degree variates the mistakes from a forecast to another. It can be said that the higher the signal is smoothed, the more accuracy is acquired. A quantitative comparison of the proposed method execution under different P values is resumed in Table 4.4.

<table><caption>Table 4.4: Performance analysis on the impact of different SG parameters on the forecasting accuracy [4].</caption><thead><tr><th>Models</th><th>MSLE</th><th>MAPE(%)</th><th>nRMSE(%)</th></tr></thead><tbody><tr><td>3 SG-CBiLSTM</td><td>9.97</td><td>59.85</td><td>1.36</td></tr><tr><td>5 SG-CBiLSTM</td><td>5.82</td><td>60.14</td><td>1.04</td></tr><tr><td>7 SG-CBiLSTM</td><td>6.87</td><td>59.51</td><td>1.13</td></tr><tr><td><strong>10 SG-CBiLSTM</strong></td><td><strong>4.92</strong></td><td><strong>56.66</strong></td><td><strong>0.93</strong></td></tr><tr><td>Original CBiLSTM</td><td>8.38</td><td>61.17</td><td>1.23</td></tr></tbody></table>
As per Table 4.4, the best results have been accorded to 10 SG-CBiLSTM, which accomplishes the most reliable forecasts compared to 3 SG-CBiLSTM, which ranges to RMSE=1.36% and MAPE=59.85%. This prompts presume that the SG filters are vital components for accomplishing better accuracy. Interestingly, the choice of the most appropriate parameters s compulsory to accomplish the ideal outcomes.

In the pursuit of a fair evaluation, it will be viewed as the ten SG-CBiLSTM as a source of perspective for the proposed prediction framework. For the sake of straightforwardness and stream of perusing, the 10 polynomial SG-CBiLSTM is referred to as the SG-CBiLSTM hereinafter. The loss function is the training and validation of the model simulation is a fundamental basis to follow the learning accumulation in the model development. The parametric equation of the loss function is determined as given [141]:

$$Loss(\hat{y}, y) = \sqrt{\frac{1}{B} \frac{1}{O} \sum_{i=1}^{B} \sum_{j=1}^{O} (\hat{y} - y)^2} \quad (4.1)$$

where B, O denote the output matrix size. Here, the proposed model is trained with a training set stating from 2^nd^, March 2003 with the first 75% portion of the whole set. To follow the loss values when the proposed SG-BiLSTM fetches the next batch of data. It has been adopted the MAPE as a loss function with an Adam optimizer [142]. The training process was selected at hundred epochs with an early-stopping function. This function is adopted to automatically follow the best-trained model without looking for the most appropriate epoch number. Fig. 4.5 displays the variation of the training loss of the first foury epochs for the training and testing sets.

<figure><img src="image_47.png" /><figcaption>Figure 4.10: Loss trend of the SG-CBiLSTM model [4].</figcaption></figure>

As shown in Fig. 4.5, the training loss value begins at 6.93% and constantly diminishes until the 10^th^ epoch begins to settle. At this level, the loss value reaches 2.08 %. At this level, the training loss value diminishes gradually to accomplish 1.59 % at the 40^th^ epoch. Alternatively, the validation loss started at 4.415 % to reach 2.407 % in the 10^th^ epoch, where it diminishes gradually aside from the training loss. This
demonstrates that the training and validation loss reaches their equilibrium at the same iteration. Besides, it is checked that the convergence speed is reasonably quick. The training and validation loss follow the exact error distribution. The simulation results of the proposed method in tandem with its single components (CNN, BiLSTM, and CBiLSTM) with 10-CV have been registered and appeared in Tables 4.5-4.6.

<table><caption>Table 4.5: Forecasting accuracy of different models [4].</caption><thead><tr><th colspan="2">Model</th><th colspan="4">BiLSTM</th><th colspan="4">CNN</th></tr><tr><th>Fold</th><th>R<sup>2</sup>(%)</th><th colspan="2">nMAE(%)</th><th colspan="2">nRMSE(%)</th><th colspan="2">R<sup>2</sup>(%)</th><th colspan="2">nMAE(%)</th><th colspan="2">nRMSE(%)</th></tr><tr><th>Sets</th><th>Train</th><th>Test</th><th>Train</th><th>Test</th><th>Train</th><th>Test</th><th>Train</th><th>Test</th><th>Train</th><th>Test</th><th>Train</th><th>Test</th></tr></thead><tbody><tr><td>1</td><td>95.47</td><td>95.23</td><td>3.11</td><td>3.22</td><td>2.07</td><td>2.13</td><td>90.56</td><td>90.62</td><td>3.11</td><td>3.22</td><td>3.21</td><td>3.21</td></tr><tr><td>2</td><td>95.40</td><td>95.82</td><td>3.14</td><td>2.99</td><td>2.08</td><td>2.02</td><td>86.38</td><td>87.03</td><td>3.14</td><td>2.99</td><td>3.93</td><td>3.88</td></tr><tr><td>3</td><td>95.67</td><td>95.89</td><td>3.05</td><td>2.95</td><td>1.99</td><td>1.98</td><td>89.67</td><td>89.76</td><td>3.05</td><td>2.95</td><td>3.34</td><td>3.31</td></tr><tr><td>4</td><td>95.54</td><td>95.19</td><td>3.09</td><td>3.19</td><td>2.06</td><td>2.12</td><td>88.58</td><td>88.10</td><td>3.09</td><td>3.19</td><td>3.55</td><td>3.59</td></tr><tr><td>5</td><td>95.34</td><td>95.23</td><td>3.16</td><td>3.20</td><td>2.12</td><td>2.12</td><td>89.27</td><td>88.86</td><td>3.16</td><td>3.20</td><td>3.47</td><td>3.51</td></tr><tr><td>6</td><td>95.58</td><td>95.24</td><td>3.07</td><td>3.22</td><td>2.02</td><td>2.10</td><td>89.57</td><td>89.66</td><td>3.07</td><td>3.22</td><td>3.40</td><td>3.42</td></tr><tr><td>7</td><td>95.31</td><td>95.47</td><td>3.17</td><td>3.11</td><td>2.12</td><td>2.10</td><td>90.10</td><td>90.11</td><td>3.17</td><td>3.11</td><td>3.24</td><td>3.25</td></tr><tr><td>8</td><td>95.24</td><td>95.17</td><td>3.19</td><td>3.24</td><td>2.13</td><td>2.15</td><td>90.51</td><td>90.10</td><td>3.19</td><td>3.24</td><td>3.30</td><td>3.42</td></tr><tr><td>9</td><td>95.51</td><td>95.43</td><td>3.11</td><td>3.10</td><td>2.08</td><td>2.09</td><td>90.27</td><td>90.24</td><td>3.11</td><td>3.10</td><td>3.26</td><td>3.26</td></tr><tr><td>10</td><td>95.37</td><td>95.36</td><td>3.15</td><td>3.14</td><td>2.14</td><td>2.18</td><td>89.77</td><td>89.79</td><td>3.15</td><td>3.14</td><td>3.30</td><td>3.30</td></tr><tr><td>Mean</td><td>95.45</td><td>95.41</td><td>3.13</td><td>3.14</td><td>2.09</td><td>2.10</td><td>89.47</td><td>89.43</td><td>3.13</td><td>3.14</td><td>3.41</td><td>3.42</td></tr><tr><td>SD</td><td>0.13</td><td>0.25</td><td>0.04</td><td>0.10</td><td>0.05</td><td>0.06</td><td>1.18</td><td>1.05</td><td>0.04</td><td>0.10</td><td>0.20</td><td>0.19</td></tr><tr><td>Time(s)</td><td colspan="4">3238.38</td><td colspan="4"></td><td colspan="4">3643.24</td></tr></tbody></table>
<table><caption>Table 4.6: Forecasting results of the proposed model [4].</caption><thead><tr><th colspan="2">Model</th><th colspan="4">CBiLSTM</th><th colspan="4">Proposed</th></tr><tr><th>Fold</th><th>R<sup>2</sup>(%)</th><th colspan="2">nMAE(%)</th><th colspan="2">nRMSE(%)</th><th>R<sup>2</sup>(%)</th><th colspan="2">nMAE(%)</th><th colspan="2">nRMSE(%)</th></tr><tr><th>Sets</th><th>Train</th><th>Test</th><th>Train</th><th>Test</th><th>Train</th><th>Test</th><th>Train</th><th>Test</th><th>Train</th><th>Test</th></tr></thead><tbody><tr><td>1</td><td>88.54</td><td>87.79</td><td>4.96</td><td>5.16</td><td>2.84</td><td>2.95</td><td>99.18</td><td>99.05</td><td>0.78</td><td>1.43</td><td>0.89</td><td>0.90</td></tr><tr><td>2</td><td>88.11</td><td>88.46</td><td>5.05</td><td>4.97</td><td>3.00</td><td>3.96</td><td>99.27</td><td>99.09</td><td>0.73</td><td>1.38</td><td>0.65</td><td>0.66</td></tr><tr><td>3</td><td>85.84</td><td>85.52</td><td>5.52</td><td>5.54</td><td>3.17</td><td>3.18</td><td>99.02</td><td>99.28</td><td>0.91</td><td>1.22</td><td>0.70</td><td>0.71</td></tr><tr><td>4</td><td>87.63</td><td>87.79</td><td>5.16</td><td>5.09</td><td>2.92</td><td>2.90</td><td>99.12</td><td>99.05</td><td>0.77</td><td>1.43</td><td>0.70</td><td>0.70</td></tr><tr><td>5</td><td>86.55</td><td>85.84</td><td>5.37</td><td>5.53</td><td>3.12</td><td>3.18</td><td>99.22</td><td>98.97</td><td>0.79</td><td>1.48</td><td>0.62</td><td>0.64</td></tr><tr><td>6</td><td>86.55</td><td>85.02</td><td>5.71</td><td>5.73</td><td>3.24</td><td>3.29</td><td>99.16</td><td>99.33</td><td>0.73</td><td>1.20</td><td>0.68</td><td>0.69</td></tr><tr><td>7</td><td>84.80</td><td>89.16</td><td>4.76</td><td>4.81</td><td>2.74</td><td>2.77</td><td>99.21</td><td>99.16</td><td>0.78</td><td>1.33</td><td>0.73</td><td>0.73</td></tr><tr><td>8</td><td>89.47</td><td>86.9</td><td>5.23</td><td>5.34</td><td>2.95</td><td>3.04</td><td>99.13</td><td>99.29</td><td>0.73</td><td>1.23</td><td>0.67</td><td>0.67</td></tr><tr><td>9</td><td>87.26</td><td>84.91</td><td>5.68</td><td>5.64</td><td>3.31</td><td>3.33</td><td>99.29</td><td>99.02</td><td>0.76</td><td>1.45</td><td>0.74</td><td>0.75</td></tr><tr><td>10</td><td>85.13</td><td>85.49</td><td>5.65</td><td>5.56</td><td>3.26</td><td>3.26</td><td>99.37</td><td>99.55</td><td>0.64</td><td>0.98</td><td>0.70</td><td>0.70</td></tr><tr><td>Mean</td><td>86.84</td><td>86.96</td><td>5.31</td><td>5.34</td><td>3.06</td><td>3.09</td><td><b>99.20</b></td><td><b>99.18</b></td><td><b>0.77</b></td><td><b>1.32</b></td><td><b>0.71</b></td><td><b>0.72</b></td></tr><tr><td>SD</td><td>1.54</td><td>1.45</td><td>0.31</td><td>0.29</td><td>0.18</td><td>0.18</td><td>0.09</td><td>0.17</td><td>0.07</td><td>0.15</td><td>0.07</td><td>0.07</td></tr><tr><td>Time(s)</td><td colspan="4">2416.56</td><td colspan="4"></td><td colspan="2"><b>1808.4</b></td></tr></tbody></table>

As per Tables 4.5-4.6, the 10-CV outcomes obviously exhibit that our proposed system altogether beat stand-alone models according to the score errors. The SG-CBiLSTM accomplishes a high mean testing value of R² =99.18% compared to 86,96%, 89,43%, and 95,41% for CBiLSTM, CNN, BiLSTM individually. The general increment of the performance for the DL models revealed in Table 4.5 is clarified by embracing the data augmentation methodology for the first database. Moreover, employing the S-G filter with the 10th polynomial fitting supports the CBiLSTM precision with an R²12.22%. An outline of the 10-CV outcomes regarding the R² has appeared in Fig. 4.11 feature the accuracy upgrade of the SG-BiLSTM model.
<figure><img src="image_48.png" /><figcaption>Figure 4.11: Box plot of ten-fold CV of four models in terms of R²(%).</figcaption></figure>

It is apparent from Fig. 4.11 that the SG-BiLSTM model successfully achieves a state-of-art performance compared to individual models according to the R² results. The SG smoothing technique helps to see patterns better and detects trends. As indicated by the CV outcomes, the SG-BiLSTM model is well suited for tracking load demand dependencies. The accomplished outcomes demonstrate that the SG-BiLSTM stands closely tied to STLF at various meteorological and spatial conditions. The simulation outcomes of the proposed model are illustrated in Fig. 4.12.

<figure><img src="image_49.png" /><figcaption>Figure 4.12: Hourly predictions of the forecasting system [4].</figcaption></figure>

According to Fig. 4.12, the proposed framework accomplishes high exactness and tracks the measured load demand with extraordinary accuracy. The performance of the proposed method is analyzed with nine reference models conserving the same experimental environment for a reliable assessment. The reference
models evaluated incorporate ELM, BiLSTM, BiGRU, ConvLSTM, CNN, BiLSTM-Autoencoders (BiLSTM-AE), RNN, CNN, and CBiLSTM. It ought to be noticed that the utilized database and the generated data representation from the proposed model construction process are saved for the comparison of all the models and the rest of the approval examination. The experimental outcomes are resumed in Fig. 4.13.
<figure><img src="image_50.png" /><figcaption>Figure 4.13: Simulation results of SG-CBILSTM and reference models over multiple steps ahead [4].</figcaption></figure>

Regarding Fig. 4.13, the test results obviously prove that the high supremacy of the proposed model
compared to the state-of-art models for hourly STLF due to the slight difference between the real and produced predictions. To assess the prediction accuracy, Table 4.7 shows the score measurement of the proposed SG-CBILSTM and benchmarks.

<table><caption>Table 4.7: Forecasting results of different models on ISO-New England data set [4].</caption><thead><tr><th>Mean values of 10 times</th><th>nRMSE(%)±SD</th><th>nMAE(%)±SD</th><th>R<sup>2</sup>(%)±SD</th></tr></thead><tbody><tr><td>SG-CBILSTM</td><td>1.22±0.20</td><td>0.89±0.23</td><td>99.22±0.29</td></tr><tr><td>BiGRU</td><td>2.71±0.27</td><td>2.94±0.37</td><td>95.93±1.10</td></tr><tr><td>BiLSTM-AE</td><td>1.76±0.11</td><td>1.16±0.05</td><td>98.56±0.18</td></tr><tr><td>ConvLSTM-AE</td><td>4.13±0.31</td><td>3.13±0.11</td><td>85.21±0.23</td></tr><tr><td>RNN</td><td>3.77±0.32</td><td>2.82±0.25</td><td>93.35±1.06</td></tr><tr><td>ELM</td><td>5.86±0.30</td><td>4.45±0.24</td><td>83.98±1.74</td></tr></tbody></table>

Regarding Table 4.7, the proposed SG-CBILSTM model achieves the best accuracy comparing to reference models on the scale of the nRMSE, nMAE, MAPE, and R² execution measures. The R² comes to 99.22% due to SG filtering. Moreover, the model performance was marginally near the BiLSTM-AE model, which accomplishes an R² = 98.56%. Due to the complex nonlinear relationship between the load demand and its derivatives, the ELM model accomplishes the littlest execution precision with an R² = 83.98%. Additionally, it is clear that the proposed model is best compared to reference algorithms. The proposed SG-CBILSTM is proved to be more prominent to improve the unit commitment between the load and the supply sides.

## Multi-step Validation

In this work, the model execution is evaluated for multi-step forecasting utilizing numerous case studies. The multi-step validation is pivotal to assess the robustness under an extended time range. The analyzed prediction ranges comprise 1h, 12h, one day, and 36h ahead. In the simulations results, we conserve the similar repartition of the database with 75%, 10%, and 25% for the training, validation, and testing sets of the entire dataset. For multi-step-ahead forecasting, the structure of the proposed SG-CBILSTM model conserves the shape of CNN layers compared with one-step forecasting. For the on-step-ahead, the n anticipated yields $\hat{x}_{i+1} = f(x_i, x_{i-1}, ..., x_{i-n})$ takes into consideration the real values of the load data. Notwithstanding, in multi-step-ahead forecasting, the previously produced predictions are included as inputs as $\hat{x}_{i+2} = f(\hat{x}_{i+1}, x_i, x_{i-1}, ..., x_{i-n})$. The experimental results of multi-step forecasting for ISO-NE data were repeated ten times, and the results are averaged in Table 4.8. Besides, the error between the forecasted
and real load demand referring to the prediction range is illustrated in Fig. 4.14.

<table><caption>Table 4.8: Results of Multistep forecasting on ISO-NE dataset [4].</caption><thead><tr><th>Mean values of 10 times</th><th>One-step</th><th>Twelve-step</th><th>Twenty four-step</th><th>Thirty-six-step</th></tr></thead><tbody><tr><td>nRMSE(%)</td><td>0.54</td><td>3.3</td><td>2.26</td><td>3.59</td></tr><tr><td>nMAE(%)</td><td>1.23</td><td>2.14</td><td>1.34</td><td>2.02</td></tr><tr><td>R<sup>2</sup>(%)</td><td>99.11</td><td>87.39</td><td>94.26</td><td>87.22</td></tr></tbody></table>

<figure><img src="image_51.png" /><figcaption>Figure 4.14: Multi-step STLF error variation with the Error (%) = 100*(ŷnorm - ynorm) [4].</figcaption></figure>

As indicated in Table 4.8, it can be remarked that the most reasonable scale is the hourly LF. Focusing on the error measures, the hourly ahead produced outcomes presents an $R^2$ = 99.11% and nRMSE=0.54% compared to $R^2$ = 87.22% and nRMSE=3.59% for 36h ahead. Therefore, the minimum error measure is generated by single-step forecasting, and the maximum error value is produced by 36h forecasting ahead, as illustrated in 4.14. In this study, it is noteworthy to say that the minimal accuracy is accorded to sequential dependencies of 36h ahead with an $R^2$ = 87.22%. This shows the limitation of the proposed for long prediction ranges, which require more examination.

### 4.2.2 Case study 2

In this investigation, a Stacked Generalization method between XGB, LGBM, and Multilayer Perceptron (MLP) models, Stacked XGB-LGBM-MLP, is first and foremost analyzed for STLF. To the authors' best knowledge, no earlier work has tackled this technique for time series prediction. The Stacked XGB-LGBM-
MLP model is preferred among the available solutions for its high exactness, superb execution, and simplicity.

Also, the simulation results with an open data portal verified that the Stacked XGB-LGBM-MLP model figures out how to beat eleven reference models for STLF.

## Proposed methodology

One of the most captivating problems this thesis faces is finding the right combination of models ingredients to boost the overall forecasting system without compromising its computational efficiency and memory limitations. To bridge this knowledge gap, an efficient ensemble model is comprehensively introduced for STLF.

**Stacked XGB-LGBM-MLP Method:** In this part, the proposed model lies in the fusion of XGB, LGBM, and MLP networks to construct an excellent meta-learner. Stacked Generalization (Stacking) is characterized as an undeniable level nonlinear strategy for combination [143]. This mix procedure applies nonlinear weightings for low-level predictors to support the prediction framework exactness. The numerical simulations verify that in many cases, Stacking procedure accomplishes preferred outcomes rather than any other base learners [144, 145]. The Stacked scheme has two-layered structures (level 0, and level 1) as illustrated in Fig. 4.15.

<figure><img src="image_52.png" /><figcaption>Figure 4.15: Graphical representation of the stacking generalization approach [5].</figcaption></figure>

Regarding Fig. 4.15, the proposed methodology lies in producing a sequence of forecasts by a set of learners in the primary level, called base layer [143]. At level-0, the generalizing biases are collectively forecasted. In level-1, named Meta-layer, these forecasts are fed to a Meta-learner to compute the forecast yields utilizing a Cross-Validation (CV) technique. This method targets filtering the outcomes from the primary level of universality. Assuming that *N* samples in training set ($S_i, y_i$) with $1 \le i \le N$ is the sample time. The training set is haphazardly divided into *r* folds with practically equivalent size to constitute ($S_i, y_i$)$_k$ where $k' = (1, ..., r)$ is the fold number. The ($S_i, y_i$)$_k$ fulfill the accompanying condition:
$$ \left\{ \begin{array}{l} (S, y)_k \cup \overline{(S, y)_k} = (S, y) \\ (S, y)_k \cap \overline{(S, y)_k} = \emptyset \end{array} \right. \qquad (4.2) $$

Level-0 takes the second part $N/r$ of the data set $(S_i, y_i)_k$. The weak learners $(L_1, \dots, L_N)$ predict the $Y_i$ using $S/S_i$ feature vectors. At this level, the test part $Si$ is computed using weak learners. The yield results with the real data set $y_i$ shape the meta-level data set $MSi$ with a new feature vector. The $MSi$ data set is fed to a meta-learner to form a meta-level vector from the base-level regressors. The Stacking concept adopted in level-0 consists of two training models with 5 folds as demonstrated in Fig. 4.16.

<figure><img src="image_53.png" /><figcaption>Figure 4.16: Schematic representation of the level-0 of stacked architecture framework built from base learners: XGB and LGBM [5].</figcaption></figure>

Regarding Fig. 4.16, the level-0 learners comprise XGB and LGBM models while the meta-learner is an MLP model. The idea of the proposed approach is hierarchically coupling ensemble methods and Neural Networks represented by MLP model to frame a multimodal forecasting framework. The idea of choosing XGB and LGBM as base models are conducted due to their high performance in other forecasting applications. It has been remarked that the stronger the forecasting potential of each heterogeneous base model, the higher performance of the overall stacking ensemble. The goal here is to construct an excellent prediction paradigm to cope with nonlinear system variations, explicitly for STLF task. A binary combination of level-0 models takes into consideration the input parameters, precisely, the temperature and the DateTime to be introduced to the XGB and LGBM models. These two methods produce expectations for each test set using an Out-Of-Fold (OOF) method. In other words, the training part uses a sub-fold for a CV-such as fashion training which is different from the validation part to prevent the over-fitting issue. A mean value
of all folds is calculated for the temporary test sets. The collected features are considered as parameters for
the next level. At level-1, the MLP model fits the model with temporary predictions in the training set.
Once MLP is trained, the forecasting framework only considers the output results generated at this stage.
It merits saying that the training system takes more time to fit all the sub-folds with different meta-learners
and base learners. Moreover, the higher level of predictors does not guarantee better outcomes as well as
the total stacking concept. The detailed model synoptic is outlined in Fig. 4.17.

<figure><img src="image_54.png" /><figcaption>Figure 4.17: Main framework of the proposed stacking model [5].</figcaption></figure>

As per Fig. 4.17, four stages are processed, explicitly, Feature engineering, object determination, for-
casting, and assessment stages. In the engineering stage, the data is prepared with data cleaning, feature
extraction, and feature selection. The dataset, explicitly, temperature, month, day, hour and load feature
vectors are integrated into the object determination stage. This stage contains the problem formulation, the
data split into training and testing. Besides, the ML models’ hyperparameters are optimized to choose the
most appropriate values according to the single learners and the meta-learner individually. In the forecasting
stage, XGB and LGBM are trained in the level-0 of the Stacking approach. At that point, the meta-data
outputs are fed for the MLP meta-learner model. The meta-learner analyses the attributes inputs to process
the last yield. Fig. 4.18 presents a flowchart of the whole procedure.

<figure><img src="image_55.png" /><figcaption>Figure 4.18: Flowchart of the proposed Stacked XGB-LGBM-MLP method [5].</figcaption></figure>

The evaluation stage is conducted to analyze the performance of the forecasting system using point forecasting assessment, K-fold cross-validation, and visualization graphs.

## Simulation results

In view of a publicly accessible datasets, a real case scenarios were led to evaluate the proposed method and outline the prediction performance of the hybrid model. Besides, a comparative analysis with other reference methods is conducted. Further, the high exactness of the proposed strategy with the recent hybrid STLF procedure for the equivalent database has been demonstrated.

**Data Analysis and Processing:** The high performance of a forecasting system is basically linked to two factors: the quality of the input data and the forecasting engine. In this manner, data analysis and feature engineering are mandatory for enhancing the data quality of the underlying system. To check the effectiveness
of the proposed approach, two real datasets are utilized. For the primary contextual investigation, the data are taken on an open-access base from a power supply industry in the city of Johor, Malaysia [146]. The gathered data comprises high-resolution data with 17519 samples and a sampling frequency of one hour. The database comprises a time period from 01/01/2009 to 01/01/2011. The measured hourly temperature (°C) and load demand (MW) as demonstrated in Fig. 4.19.

<figure><img src="image_56.png" /><figcaption>Figure 4.19: The time series data between 2009-2011 (17519 samples) [5].</figcaption></figure>

As per Fig. 4.19a, it is noteworthy to say that the load demand presents a high sensibility in tandem with a few erroneous measurements (samples=[14748-14749]). The load demand utilization unpredictably changes after some time with high variability and nonlinear behavior. The load data is utilized to confirm the practicality of the stacking framework using the temperature as an indicator introduced in Fig. 4.19b. The temperature behavior is presented as the vital marker of the load demand variation.

For the second scenario, the utilized dataset comes from ISO New England control area, and its eight-wholesale load (ISO-NE) records [147]. The underlying database contains Boston, Bridgeport, Burlington, Concord, Portland, Providence, Windsor Locks, and Worcester registrations. The data reported from ISO-NE are employed for energy management and market enabling. The obtained information incorporates the hourly temperature and load taken from 2003 to 2014. The total vector rows utilized is 103775. The two datasets were partitioned into training data and testing data at a ratio of 80% and 20%, respectively. The testing set for the first and second case scenario contains 1755 and 103774 vector rows individually. The objective of utilizing these datasets with various sizes lies in investigating the model accuracy on small and large training samples.

For the information preprocessing, the dataset was cleaned from records anomalies and strange values. Subsequently, some temporal inputs have been uploaded in the categorical shape. Therefore, the DateTime
was transformed into numerical quantities and partitioned to individual system inputs (hour, day, month, year) to improve the continuous forecasting of temporal labels and facilitate the prediction procedure. At that point, the information is divided into two folds for training and testing. The final data representation incorporates the yearly, monthly, daily, and hourly DateTime inputs liked to the temperature variation while the outcomes lie in the load estimations.

**Hyperparameter Optimization:** In this examination, multiple HO methods have been analyzed to choose the most reasonable method for STLF. Customarily, the hyperparameter values were chosen using on trial and error method. This is computed by interpreting the model performance and dynamically limiting the search space. Meanwhile, manual tuning is often a tedious task leading to inadmissible results. To tackle these issues, Grid Search method (GS) is utilized as an automatic HO procedure to track all the possible values in the solution space. The GS method technique is possibly utilized when the dimension of the search space is moderately low. Moreover, GS is computationally problematic since all cases in the solution space are explored and mimicked. On the other hand, the utilization of meta-heuristics for HO can essentially decrease the computational expense and the fair amount of time associated with GS. In this investigation, five methods were surveyed and compared for HO. Specifically, PSO, SA, ES, RS, and BO [119].

## Comparative results

The determination of the most appropriate HO techniques is vital for achieving satisfactory performance without compromising computational cost-effectiveness. Despite the importance of this topic, the HO does not receive significant attention. In this regard, 5 HO methods are examined in order to choose the most appropriate HO technique for the proposed framework. A predefined search space for all the elements of the proposed Stacked XGB-LGBM-MLP has been computed separately. The simulated environment is conducted utilizing a Lenovo Intel ® i7 ® Nvidia Geforce GTX 1650@ 2.30 GHz. For the simulation, Python programming language and Hyperactive library have been utilized [119]. The search space comprises nine chosen parameters, explicitly, the number of estimators, the learning rate, maximum depth of both XGB and LGBM, the maximum of iterations, and the hidden layer size for MLP model. The search space is enormous due to the number of candidate solutions from assigned hyperparameter values to reach the near-ideal solutions. Table 4.9 illustrates the model configuration.
<table><caption>Table 4.9: hyperparameter settings of experimental models [5].</caption><thead><tr><th>ML Method</th><th>Score Function</th><th>Default value</th><th>Search space</th><th>SA</th><th>PSO</th><th>BO</th><th>RS</th><th>ES</th></tr></thead><tbody><tr><td rowspan="3">XGB</td><td>Number of estimators</td><td>100</td><td>[100, 500, 750, 1000, 1500, 2000, 2500, 3000]</td><td>1500</td><td>1000</td><td>100</td><td>2000</td><td>2000</td></tr><tr><td>Learning rate</td><td>0.1</td><td>[10<sup>-3</sup>,10<sup>-2</sup>, 0.1, 0.3, 0.2, 0.5, 0.8, 1]</td><td>0.1</td><td>0.3</td><td>0.2</td><td>0.1</td><td>0.3</td></tr><tr><td>Max depth</td><td>3</td><td>-2 ~ 12/step = 1</td><td>4</td><td>2</td><td>7</td><td>12</td><td>3</td></tr><tr><td rowspan="3">LGBM</td><td>Number of estimators</td><td>100</td><td>[100, 500, 750, 1000, 1500, 2000, 2500, 3000]</td><td>750</td><td>2000</td><td>1500</td><td>500</td><td>2500</td></tr><tr><td>Learning rate</td><td>0.1</td><td>[10<sup>-3</sup>,10<sup>-2</sup>, 0.1, 0.2, 0.3, 0.5,0.8, 1.0]</td><td>0.5</td><td>0.2</td><td>0.3</td><td>0.3</td><td>0.001</td></tr><tr><td>Max depth</td><td>-1</td><td>-2 ~ 12/step = 1</td><td>2</td><td>-1</td><td>4</td><td>11</td><td>3</td></tr><tr><td rowspan="3">MLP</td><td>Max iterations</td><td>200</td><td>1000 ~ 3000/step = 500</td><td>2500</td><td>2000</td><td>2000</td><td>1000</td><td>1500</td></tr><tr><td>Hidden layer sizes</td><td>100</td><td>[100, 200, 300, 400, 500]</td><td>100</td><td>200</td><td>500</td><td>200</td><td>300</td></tr><tr><td>R<sup>2</sup>(%)</td><td></td><td></td><td>90</td><td>91</td><td>88</td><td>98</td><td>89</td></tr><tr><td></td><td>Time(min)</td><td></td><td></td><td>133</td><td>204</td><td>212</td><td>259</td><td>82</td></tr><tr><td></td><td>Time/iteration(min)</td><td></td><td></td><td>4.42</td><td>6.79</td><td>7.07</td><td>8.62</td><td>2.71</td></tr></tbody></table>

As per Table 4.9, it is noteworthy to say that the fine-tuning of the hyperparameters is computationally requesting for all the proposed models but with different ranges. The quickest calculations for HO are ES and SA, which experimentally decreased the computational expense compared to the other techniques with a consuming time of 82 minutes and 133 minutes, respectively. On the opposite side, the best procedure, as indicated by the highest R² value, is RS technique. This is clarified by the fact that expanding the running time prompts higher opportunities to acquire near-ideal results. Despite the RS prevalence, evolution strategy is computationally costly compared with the other HO techniques. Considering the HO method concurred with a higher R² value for the rest of the hyperparameter tuning problems in this study, The RS method is embraced since it produces the best outcomes.
1st case study

The performance evaluation of the proposed model for different time scales ahead is conducted in this part. The proposed model was computed using Python coding. Several Python libraries were incorporated in the model design, including Scikit-learn, LightGBM, and XGB [118, 148]. Besides, the stacked generalization was built using Vecstack package [149]. The simulation results were generated from a Lenovo Intel ® i7 9th Generation ® Nvidia Geforce GTX 1650@ 2.30 GHz@16GB RAM. The HO was conducted using RS tool for the sake of enhancing the forecasting accuracy by the comprehensive selection of the most suitable parameters [150]. To evaluate the proposed method effectiveness, the model is trained and compared to individual models, explicitly, XGB, MLP, and LGBM [151]. Initially, these methods were utilized as benchmarks to check the viability of the proposed approach. Clearly, the stacked estimations significantly depend mainly on the prediction performance of the base models. The simulation procedure was repeated ten times for providing higher reliability to the forecasting system. To make the comparison more intuitive, a single step ten-fold CV(10-CV) was led to follow the varieties of the models' exhibition. The results of 10-fold CV is computed and illustrated in Table 4.10 for the ingredients of the proposed method. Furthermore, Table 4.11 is resumed the 10-CV outcomes for the Stacked XGB-LGBM-MLP model.
<table><caption>Table 4.10: Cross-validation results for case study 1 [5].</caption><thead><tr><th rowspan="2">Model</th><th colspan="3">MLP</th><th colspan="3">LGBM</th><th colspan="3">XGB</th></tr><tr><th>MAE<br>(MW)</th><th>RMSE<br>(MW)</th><th>R<sup>2</sup><br>(%)</th><th>MAE<br>(MW)</th><th>RMSE<br>(MW)</th><th>R<sup>2</sup><br>(%)</th><th>MAE<br>(MW)</th><th>RMSE<br>(MW)</th><th>R<sup>2</sup><br>(%)</th></tr></thead><tbody><tr><td>0</td><td>771,15</td><td>1003,47</td><td>88,05</td><td>714,62</td><td>943,51</td><td>89,44</td><td>802,91</td><td>1059,83</td><td>86,67</td></tr><tr><td>1</td><td>766,97</td><td>1017,14</td><td>87,16</td><td>724,45</td><td>973,56</td><td>88,24</td><td>814,18</td><td>1090,52</td><td>85,24</td></tr><tr><td>2</td><td>736,62</td><td>991,25</td><td>88,44</td><td>705,14</td><td>949,31</td><td>89,40</td><td>790,72</td><td>1057,65</td><td>86,84</td></tr><tr><td>3</td><td>758,88</td><td>1013,63</td><td>87,68</td><td>726,33</td><td>970,30</td><td>88,71</td><td>798,82</td><td>1060,68</td><td>86,51</td></tr><tr><td>4</td><td>744,16</td><td>988,11</td><td>88,36</td><td>705,20</td><td>939,96</td><td>89,47</td><td>792,17</td><td>1056,48</td><td>86,69</td></tr><tr><td>5</td><td>759,27</td><td>1007,75</td><td>87,77</td><td>710,56</td><td>949,88</td><td>89,14</td><td>791,84</td><td>1056,93</td><td>86,55</td></tr><tr><td>6</td><td>762,86</td><td>1000,71</td><td>87,91</td><td>709,99</td><td>942,76</td><td>89,27</td><td>793,99</td><td>1054,44</td><td>86,58</td></tr><tr><td>7</td><td>780,16</td><td>1028,25</td><td>87,44</td><td>716,76</td><td>959,15</td><td>89,07</td><td>799,85</td><td>1068,04</td><td>86,45</td></tr><tr><td>8</td><td>790,56</td><td>1029,11</td><td>87,74</td><td>727,56</td><td>962,48</td><td>89,27</td><td>807,24</td><td>1068,88</td><td>86,77</td></tr><tr><td>9</td><td>744,61</td><td>989,23</td><td>88,30</td><td>711,90</td><td>941,29</td><td>89,41</td><td>788,60</td><td>1046,00</td><td>86,92</td></tr><tr><td>Mean</td><td>761,52</td><td>1006,87</td><td>87,89</td><td>715,25</td><td>953,22</td><td>89,14</td><td>798,03</td><td>1061,95</td><td>86,52</td></tr><tr><td>SD</td><td>15,92</td><td>14,35</td><td>0,39</td><td>7,91</td><td>11,72</td><td>0,37</td><td>7,78</td><td>11,37</td><td>0,45</td></tr><tr><td>Computational<br>Time (second)</td><td colspan="3">3238.38</td><td colspan="3">3664.78</td><td colspan="3">3643.24</td></tr></tbody></table>
<table><caption>Table 4.11: Evaluation errors for case study 1 [5].</caption><thead><tr><th colspan="4">Stacked XGB-LGBM-MLP model</th></tr><tr><th>Fold number</th><th>MAE(MW)</th><th>RMSE(MW)</th><th>R<sup>2</sup>(%)</th></tr></thead><tbody><tr><td>0</td><td>489,74</td><td>707,29</td><td>94,05</td></tr><tr><td>1</td><td>477,22</td><td>683,82</td><td>94,45</td></tr><tr><td>2</td><td>480,60</td><td>687,10</td><td>94,24</td></tr><tr><td>3</td><td>487,03</td><td>707,94</td><td>94,17</td></tr><tr><td>4</td><td>481,47</td><td>690,45</td><td>94,35</td></tr><tr><td>5</td><td>483,04</td><td>718,25</td><td>93,88</td></tr><tr><td>6</td><td>488,27</td><td>696,69</td><td>94,33</td></tr><tr><td>7</td><td>475,68</td><td>675,73</td><td>94,47</td></tr><tr><td>8</td><td>475,64</td><td>680,11</td><td>94,84</td></tr><tr><td>9</td><td>471,58</td><td>666,61</td><td>94,84</td></tr><tr><td>Mean</td><td>481,03</td><td>691,40</td><td>94,31</td></tr><tr><td>SD</td><td>5,75</td><td>15,29</td><td>0,25</td></tr><tr><td>Computational Time (second)</td><td colspan="3">2033.08</td></tr></tbody></table>

As indicated by Table 4.10, the investigation of the prediction execution of each base model exhibits that LGBM produces the best forecasting results in terms of the error measurements. The average R² for LGBM model is R²=89,14% compared to R²=87,89% and R²=86,52% for MLP and XGB models respectively. The registered results comprises a mean RMSE=1006,87 MW, 953,22 MW and 1061,95 MW for MLP, LGBM, and XGB, respectively. Consequently, the stacked XGB-LGBM-MLP prevails to perform best of all stand-alone models with a mean R²=94,31% as revealed in Table 4.11. The high forecasting precision accomplished by the stacked generalization approach is confirmed from a lower mean RMSE=691,40 MW. The stacked XGB-LGBM-MLP model can utilize the merits of different models with various training mechanisms and effectively achieve the complementarity between the benefits of various models. The relative stability of the proposed technique with a minimum Standard Derivation SD=0,25% compared to its derivatives reveals the model's effectiveness against the unpredictable perturbations. For a real testing environment on a 24-hour ahead, Table 4.12 illustrate the score measurement results.
<table><caption>Table 4.12: Evaluation measures for the stacked model and other benchmarks for one day ahead [5].</caption><thead><tr><th>Model</th><th>RMSE (MW)</th><th>MAE<br>(MW)</th><th>R<sup>2</sup><br>(%)</th><th>MdAE<br>(MW)</th><th>MAPE<br>(%)</th><th>MSLE<br>(10<sup>-2</sup>)</th></tr></thead><tbody><tr><td>XGB</td><td>5641.67</td><td>5220.20</td><td>0.87</td><td>4455.34</td><td>12.77</td><td>1,8</td></tr><tr><td>LGBM</td><td>2143.23</td><td>1765.14</td><td>0.98</td><td>1404.88</td><td>3.85</td><td>0,23</td></tr><tr><td>MLP</td><td>5135.50</td><td>4623.32</td><td>0.89</td><td>4344.50</td><td>10.36</td><td>1,47</td></tr><tr><td>Stacked XGB-LGBM-MLP</td><td>1509.74</td><td>1070.67</td><td>0.99</td><td>597.64</td><td>2.69</td><td>0,17</td></tr></tbody></table>

According to Table 4.12, the RMSE value of the proposed approach is the minimum by a value of RMSE= 1509.74 MW compared with an RMSE =5641.67 MW, RMSE=2143.23 MW, and RMSE=5135.50 MW for the single XGB, LGBM, and MLP models, respectively. The MAE value of the proposed approach is likewise the lowest, equal to 1070.67 MW compared to 5220.20 MW and 1764.14 MW for XGB and LGBM individually. Moreover, the Stacked XGB-LGBM MLP model confirms its ability to deliver a more robust prediction solution compared to the rest of the models in terms of MAE, R², MAPE, and MSLE error values. A daily STLF is computed and appeared in Fig. 4.20.

<figure><img src="image_57.png" /><figcaption>Figure 4.20: Actual and predicted demand for 31/12/09 [5].</figcaption></figure>

Generally, it can be noticed that the predicted values follow the real values as demonstrated in Fig. 4.20. The largest error (4652.57 MW) is reported in hour 16, while the minimum error is equal to 297.71 MW. The increase of the error is due to the high value at the load peak. The relationship between real load and
forecast points is investigated with the marginal distribution displayed in Fig. 4.21.

<figure><img src="image_58.png" /><figcaption>Figure 4.21: Scatter graph of the joint distribution of the real load demand and its forecast.</figcaption></figure>

Concerning Fig. 4.21, the majority of the forecasts during the testing phase are very close to the zero references, which refers that the system does not have an unbiased probability. The graphical mapping of the proposed model as demonstrated in Fig. 4.22.

<figure><img src="image_59.png" /><figcaption>Figure 4.22: Actual and predicted load demand with XGB, LGBM, MLP, and stacked model [5].</figcaption></figure>

It is apparent from Fig. 4.22 that the proposed model is the closest to the ground truth compared to the rest of the models. It is significantly seen that the stacked generalization ameliorates the system
performance compared to single methods. The simulation results have been compared to 11 techniques proposed with the same testing conditions and settings and reported in [152]. These techniques include hybrid Convolutional Neural Networks-Fuzzy Time Series (FTS-CNN), Seasonal Auto-regressive Integrated Moving Average (SARIMA), Probabilistic Weighted Fuzzy Time Series (PWFTS), Weighted Fuzzy Time Series (WFTS), Integrated Weighted Fuzzy Time Series (IWFTS), and LSTM Neural Networks [152]. The LSTM architecture includes two layers with 256 and 64 units respectively, sequence input layer, dropout layer with a forgetting rate of 0.4, a Softmax function, a batch size of 72, a fully connected layer and regression layer, a maximum of iterations of 1000 with an early stopping function. The reference models were embraced from [152] where it has been accounted that the lagged hours remarkably influence the system effectiveness. Subsequently, the tested LSTM 1,2,3 were assigned to three configurations according to 24, 48, 72 former lags fed as feature inputs. The best performance of LSTM is attributed to lagged hour 168. Besides, KNN, and RF are added to the list of reference models. The configurations of benchmarks are resumed in Table 4.18. A delineation of the proposed strategy compared with the reference models is displayed in Fig. 4.23 Moreover, Fig. 4.24 shows a graphical histogram shape of MAPE and RMSE values of the Stacked XGB-LGBM-MLP compared to the benchmarks' models.
<table><caption>Table 4.13: hyperparameters settings for reference models [5].</caption><thead><tr><th>Base models</th><th>hyperparameter settings</th></tr></thead><tbody><tr><td>XGB</td><td>The number of estimators is 2000; The learning rate is 0.001; The tree complexity is 2; The gamma value is 0.5; The max depth is -1</td></tr><tr><td>LGBM</td><td>The number of estimators is 2000; The learning rate is 0.3; The max depth is 11; The tree complexity is 3; Number of leaves 130</td></tr><tr><td>MLP</td><td>The maximum iterations are 1000; The hidden layer sizes are 500,100,50, and 30; the solver is Adam</td></tr><tr><td>FTS-<br>CNN</td><td>The image size is 32; the batch size is 100; the number of epochs is 20; the learning rate is 0.001; The convolutional layers' number are 2; the fully connected layers are 5; the dropout layer is 40%; the activation function is Rectified Linear Units (ReLU) the max pooling is 2*2</td></tr><tr><td>SARIMA</td><td>The structure is (1,0,1)(1,1,2)</td></tr><tr><td>PWFTS</td><td>Default parameters from PyFts</td></tr><tr><td>WFTS</td><td>Default parameters from PyFts</td></tr><tr><td>IWFTS</td><td>Default parameters from PyFts</td></tr><tr><td>LSTM</td><td>The number of layers is 2; The activation function is ReLU; The data used contains 24, 48, 72, and 168 lagged hours for LSTM, LSTM1, LSTM2, and LSTM3</td></tr><tr><td>KNN</td><td>The algorithm is KDTree; the nearest neighbor number is 7; the leaf size is 90; the distance function is Minkowski distance</td></tr><tr><td>RF</td><td>The maximum depth is 50; the minimum samples split is 10; The number of estimators is 50</td></tr></tbody></table>
<figure><img src="image_60.png" /><figcaption>Figure 4.23: Comparison of MAPE performance for ML models [5].</figcaption></figure>

<figure><img src="image_61.png" /><figcaption>Figure 4.24: RMSE and MAPE values for the reference models and the proposed technique [5].</figcaption></figure>

Regarding Fig. 4.23, it is significantly observed that the stacked approach achieves the best results compared to the rest of the methods in terms of MAPE values. The Stacked XGB-LGBM-MLP model succeeds to eliminate the poor prediction aspects in single models. As illustrated in Fig. 4.24, the proposed method is giving the lowest MAPE value, which confirms its superiority for STLF. Table 4.14 presents the numerical RMSE and MAPE values of all of the cited methods.
According to Table.4.14, The superiority of the proposed method is expressed by a decrease of an RMSE of 192,96 MW compared to FTS-CNN. The stacked approach proves its ability to forecast accurately and robustly since the meta-learner and base learners analyses data patterns from different feature spaces and structures. The extension of the forecasting period was investigated to explore the influence of the prediction horizon on prediction accuracy. Fig. 4.25 illustrates the prediction results for 48 hours.

<table><caption>Table 4.14: Forecasts and actual load demand [5].</caption><thead><tr><td>Model</td><td>RMSE (MW)</td><td>MAPE (%)</td></tr></thead><tbody><tr><td>Stacked XGB-LGBM-MLP</td><td>1509.74</td><td>2.69</td></tr><tr><td>FTS-CNN</td><td>1702.70</td><td>2.89</td></tr><tr><td>SARIMA</td><td>2501.25</td><td>4.23</td></tr><tr><td>PWFTS</td><td>2162.57</td><td>4.00</td></tr><tr><td>WFTS</td><td>4419.11</td><td>9.09</td></tr><tr><td>IWFTS</td><td>4663.17</td><td>7.69</td></tr><tr><td>LSTM</td><td>2037.49</td><td>3.45</td></tr><tr><td>LSTM model 1</td><td>2044.68</td><td>4.21</td></tr><tr><td>LSTM model 2</td><td>2483.71</td><td>4.23</td></tr><tr><td>LSTM model 3</td><td>2279.23</td><td>3.88</td></tr><tr><td>RF</td><td>5308.89</td><td>8.16</td></tr><tr><td>KNN</td><td>5851.56</td><td>9.24</td></tr></tbody></table>
<figure><img src="image_62.png" /><figcaption>Figure 4.25: Actual and predicted demand for 31/12/09 and 01/01/10 [5].</figcaption></figure>

Regarding Fig. 4.25, it can be concluded that the horizon extension for more than 24 hours decreases the accuracy of the forecasting system, especially after the first 30 hours of prediction. Therefore, the proposed architecture only fits the short-term forecasting horizon for one day. Table 4.15, resumes the quantification of the score metrics for two days of forecasts.

<table><caption>Table 4.15: Influence of the prediction range of the proposed model and benchmarks [5].</caption><thead><tr><th>Horizon</th><th>RMSE (MW)</th><th>MAE (MW)</th><th>R<sup>2</sup></th><th>MdAE (MW)</th><th>MAPE (%)</th></tr></thead><tbody><tr><td>24 hours</td><td>1509.74</td><td>1070.67</td><td>0.99</td><td>597.64</td><td>2.69</td></tr><tr><td>48 hours</td><td>3033.57</td><td>1984.33</td><td>0.94</td><td>1253.52</td><td>4.96</td></tr></tbody></table>

According to Table 4.15, the 48-hours ahead forecasting is conducted with an RMSE=3033.57 MW compared to an RMSE=1509.74 MW for one day-ahead forecasting. The MAE is 1984.33 MW compared to 1070.67. The RMSE value approximately decreases to half, which presents a major problem for longer forecasting dependencies. However, the proposed model still follows the real load demand with the same behavior. In general, the effect of longer forecasting dependencies is a severe dilemma that limits the capabilities of ML models associated with the scalability of forecasting systems.
## 2<sup>nd</sup> case study

In the second case study, ISO-NE dataset has been used in order to assess the effectiveness of the proposed model on a larger set. The goal of this section is to quantify the sensitivity and robustness of the model with large data for 11 years. The aforementioned adopted methodology is kept the same for the model construction. To make the comparison more intuitive, a single step ten-fold CV was conducted to follow the variations of the models' performance. Here, the results were performed using Google Collaboratory (Collab) in order to alleviate the computational effort from using the large ISO-NE dataset [147]. Collab was used as a cloud service with a GPU-centric application. The output results of the proposed models individually assessed are given in Table 4.16. Additionally, the proposed XGB-LGBM-MLP model results for 10 fold CV are resumed in Table 4.17, and the resulted mean $R^2$ score is plotted shown in Fig. 4.26.

<table><caption>Table 4.16: Performance evaluation results for reference models with 10-CV [5].</caption><thead><tr><th rowspan="2">Model</th><th colspan="3">XGB</th><th colspan="3">LGBM</th><th colspan="3">MLP</th></tr><tr><th>MAE (MW)</th><th>RMSE (MW)</th><th>R<sup>2</sup> (%)</th><th>MAE (MW)</th><th>RMSE (MW)</th><th>R<sup>2</sup> (%)</th><th>MAE (MW)</th><th>RMSE (MW)</th><th>R<sup>2</sup> (%)</th></tr></thead><tbody><tr><td>0</td><td>680,86</td><td>923,13</td><td>89,67</td><td>499,24</td><td>699,55</td><td>94,07</td><td>983,12</td><td>1223,75</td><td>81,85</td></tr><tr><td>1</td><td>679,10</td><td>931,23</td><td>99,78</td><td>499,49</td><td>711,96</td><td>94,03</td><td>950,65</td><td>1196,69</td><td>83,12</td></tr><tr><td>2</td><td>668,59</td><td>912,48</td><td>90,10</td><td>483,88</td><td>682,19</td><td>94,47</td><td>946,90</td><td>1185,69</td><td>83,28</td></tr><tr><td>3</td><td>664,57</td><td>921,06</td><td>89,92</td><td>501,88</td><td>722,47</td><td>93,80</td><td>992,68</td><td>1248,93</td><td>81,47</td></tr><tr><td>4</td><td>681,95</td><td>943,98</td><td>89,36</td><td>503,28</td><td>726,16</td><td>93,71</td><td>839,45</td><td>1116,10</td><td>85,13</td></tr><tr><td>5</td><td>657,03</td><td>900,76</td><td>90,49</td><td>498,37</td><td>701,10</td><td>94,24</td><td>927,56</td><td>1168,89</td><td>83,99</td></tr><tr><td>6</td><td>650,13</td><td>884,33</td><td>90,67</td><td>487,10</td><td>685,76</td><td>94,39</td><td>982,65</td><td>1226,55</td><td>82,04</td></tr><tr><td>7</td><td>673,56</td><td>914,08</td><td>89,64</td><td>495,53</td><td>693,51</td><td>94,04</td><td>999,57</td><td>1234,09</td><td>81,12</td></tr><tr><td>8</td><td>666,06</td><td>905,12</td><td>90,49</td><td>491,19</td><td>684,10</td><td>94,57</td><td>1017,51</td><td>1274,83</td><td>81,14</td></tr><tr><td>9</td><td>661,34</td><td>90,02</td><td></td><td>490,69</td><td>699,92</td><td>94,03</td><td>979,11</td><td>1223,37</td><td>81,75</td></tr><tr><td>Mean</td><td>668,32</td><td>90,01</td><td>0,90</td><td>495,06</td><td>700,67</td><td>94,13</td><td>961,92</td><td>1209,89</td><td>82,49</td></tr><tr><td>SD</td><td>10,07</td><td>15,96</td><td>0,40</td><td>6,20</td><td>14,64</td><td>0,27</td><td>48,13</td><td>42,68</td><td>1,27</td></tr></tbody></table>
<table><caption>Table 4.17: Performance evaluation results for THE proposed framework with 10-CV [5].</caption><thead><tr><th colspan="4">Stacked XGB-LGBM-MLP model</th></tr><tr><th>Fold number</th><th>MAE(MW)</th><th>RMSE(MW)</th><th>R<sup>2</sup>(%)</th></tr></thead><tbody><tr><td>0</td><td>312,39</td><td>439,58</td><td>97,66</td></tr><tr><td>1</td><td>310,47</td><td>443,55</td><td>97,68</td></tr><tr><td>2</td><td>301,60</td><td>421,25</td><td>97,89</td></tr><tr><td>3</td><td>306,20</td><td>470,11</td><td>97,37</td></tr><tr><td>4</td><td>307,31</td><td>458,15</td><td>97,49</td></tr><tr><td>5</td><td>311,68</td><td>434,53</td><td>97,87</td></tr><tr><td>6</td><td>300,63</td><td>422,21</td><td>97,87</td></tr><tr><td>7</td><td>301,56</td><td>421,81</td><td>97,79</td></tr><tr><td>8</td><td>301,18</td><td>414,26</td><td>98,01</td></tr><tr><td>9</td><td>298,03</td><td>430,46</td><td>97,74</td></tr><tr><td>Mean</td><td>305,01</td><td>435,59</td><td>97,73</td></tr><tr><td>SD</td><td>5,00</td><td>16,84</td><td>0,18</td></tr></tbody></table>

<figure><img src="image_63.png" /><figcaption>Figure 4.26: 10-fold CV graph for Stacked XGB-LGBM-MLP [5].</figcaption></figure>
<figure><img src="image_64.png" /><figcaption>Figure 4.27: Box-and-Whisker plot of RMSE errors for STLF models [5].</figcaption></figure>

According to Table 4.16, The LGBM generates the best performances compared to XGB and MLP models with a mean $R^2$=94,13%. Hence, a significant enhancement has been demonstrated for the proposed technique that achieves a mean $R^2$=97,73%. The forecasting results can always be kept above $R^2$=97,37%. Fig. 4.26 presents the high stability of the proposed XGB-LGBM-MLP model with a minimum $R^2$ equal to 95%. The mean RMSE and MAE for Stacked XGB-LGBM-MLP are 481,03 MW and 691,40 MW, respectively. Fig. 4.27 presents a Box-and-Whisker plot of RMSE values for STLF.

According to Fig. 4.27, the proposed technique achieved a significant improvement compared to stand-alone models with a lower RMSE value. The simulation results show that the prediction potential could be enhanced by augmenting the size of the data. Nevertheless, the Stacked XGB-LGBM-MLP also has good robustness under the condition of the small dataset. It can be seen that the stacking ensemble learning model combines the merits of every stand-alone model to overcome the limitations of low-precision prediction of single models. Therefore, it can be deduced that the proposed stacking technique is perfectly tailored for STLF.
## 4.3 Power grid stability assessment

In this subsection, power grid stability prediction has been emphasized using two case studies and multiple testing scenarios as shown in fig4.28. The proposed models are dedicated to both small and large data sizes. The first proposed model takes advantage of the enormous capabilities of DL by implementing a deep bidirectional GRU model and Simulated Annealing algorithm for data smoothing. The second model uses Cataboost, XGB, and LGBM to build an effective classifier for grid stability.

<figure><img src="image_65.png" /><figcaption>Figure 4.28: Methodology of grid stability prediction.</figcaption></figure>

### 4.3.1 Case study 1

For having improved generalized SG Stability Prediction (SGSP), this study employs a DL model, specifically, Bidirectional Gated Recurrent Unit (BiGRU) for SGSP. Both point and interval forecasting of the PGS are depicted. This study employed Simulated Annealing (SA) optimization algorithm for automatic hyperparameter tuning to select the most suitable parameters from a predefined search space.

#### Proposed methodology

The proposed model is conducted via the combination of SA, Bidirectional mechanism, and Gated recurrent unit. The acquired information by the proposed method is proceeded by six phases as shown in Fig. 4.29.

In the data processing phase, the acquired data is cleaned from empty and misleading samples. The data is transformed into time-series data, and the feature inputs are evaluated to guarantee the relevance of feature representation. Then, the data normalization is conducted to unify the range of data. In the split data phase, the data is divided into three folds: training, validation, testing. The HO of the proposed model by the SA algorithm is conducted in the optimization phase. Next, the BiGRU model employs the set of optimal hyperparameters for the model constriction phase. The final results generation consists of employing the prediction engine to produce point and interval SGSP. The final phase consists of assessing the quality of
<figure><img src="image_66.png" /><figcaption>Figure 4.29: Main framework of the proposed SA-BiGRU model [6].</figcaption></figure>

the forecasts via error measures and reference models. To sum up, the flow chart of the proposed forecasting
process is illustrated in Fig. 4.30.

Model constriction

In this section, feature engineering of the publicly available data set has been conducted to tailor the provided
data to the ML tool. The model constriction was simulated in a Python 3.7 environment running using a
Lenovo laptop i-7 GPU Nvidia Geforce 1650 GTX operating at 3.40 GHz with 16 GB RAM. Next, the used
error metrics were presented to verify the model performance compared to the state-of-the-art.

Dataset and model constriction

In our experiment, the Electrical Grid Stability Simulated dataset is employed to validate the proposed SEC approach. This dataset is collected from the 4-node star system by the Department of Computer Science, Karlsruhe Institute of Technology [153]. The data is open-sourced from the UCI Database Repository
<figure><img src="image_67.png" /><figcaption>Figure 4.30: Flowchart of the proposed forecasting process [6].</figcaption></figure>

Electrical Grid Stability Simulated Data Set [153]. Each item stands for the predictive attributes on a scale of [0,10]. The dataset contains 10000 instances representing local stability analysis. The system inputs include 12 features, specifically, the reaction time of the energy producer (tau 1) and three consumers (tau2, tau3, tau4), the nominal power produced (p1) or consumed (p2, p3, p4), the price elasticity coefficient for each network participant (g1, g2, g3, g4). The system's output is the maximum real part of the characteristic differential equation root (stab). The acquired data is cleaned from empty and misleading samples. Since the data has different scales, data normalization is essential to unify the range of data. The best solution for a large dataset is to eliminate the total rows continuing missing values. The final data is divided into three folds, 70% of the data is devoted to training and validation. The remaining 30% are used for testing purposes. The HO for BiGRU model is conducted by SA optimizer. The other benchmarks are tuned using Random Search (RS) method. Table 4.18 resumes the models' hyperparameters.

## Simulation results

In this section, the performance comparisons between our BiGRU and some other data-driven forecasting methods, including LSTM, Extreme Learning Machine (ELM), Deep artificial Neural Network (ANN), Light Gradient Boosting Machine (LGBM), and Extreme Gradient boosting (XGB) have been demonstrated. Point forecasting predicts single values of stability index values, while interval forecasting quantifies the uncertainties associated with the stability index values. The simulation results are conducted via a point
<table><caption>Table 4.18: hyperparameters settings for the simulated models [6].</caption><thead><tr><th>Base models</th><th>hyperparameter settings</th></tr></thead><tbody><tr><td>BiGRU</td><td>The number of LSTM units for the first and second layers is 512 and 256 with a dropout of 0.1<br>The optimizer function is Adam; The activation is Softplus.</td></tr><tr><td>LSTM</td><td>The number of LSTM units for the first and second layers are 256 and 128 with a dropout of 0.1 respectively;<br>The optimizer function is Adam; The activation is Sigmoid.</td></tr><tr><td>XGB</td><td>The number of estimators is 50; The learning rate is 0.001; The tree complexity is 2; The gamma value is 0.5; The max depth is 6.</td></tr><tr><td>ANN</td><td>The maximum iterations are 200; The hidden layer size is 100, the solver is Adam.</td></tr><tr><td>ELM</td><td>The maximum iterations are 1000; The hidden layer sizes is 96, and 30; The activation function is relu, the solver is Adam.</td></tr><tr><td>LGBM</td><td>The number of estimators is 100; the learning rate is 0.002.</td></tr></tbody></table>

and interval mechanism.

## Point prediction

The point forecasting assessment of the proposed model is conducted in this subsection. The scatter plot and the error distribution of the proposed BiGRU model are shown in Fig. 4.31.

<figure><img src="image_68.png" /><figcaption>Figure 4.31: (a)Scatter plot and (b)error distribution of BiGRU model [6].</figcaption></figure>

Regarding Fig. 4.31, the scatter plot in Fig. 4.31(a) shows that the predicted points follow the actual results with high exactitude. The proposed model's error distribution proves that the proposed model is highly effective with minimum error values. To show the performance of the proposed model, a hundred epochs were used to train the model. To verify the competitiveness of the proposed BiGRU model, performance comparison with other benchmarks is conducted. The simulation results are plotted in Fig. 4.32.

From Fig. 4.32, the variations of the loss function show that the proposed model is protected from
<figure><img src="image_69.png" /><figcaption>Figure 4.32: Training and testing loss variation for epochs [6].</figcaption></figure>

overfitting. This result mainly refers to the best hyperparameters generated by the SA algorithm. To verify the competitiveness of the proposed BiGRU model, the performance comparison with other benchmarks is conducted. The simulation results are plotted in Fig. 4.33.

The simulation results projected in Fig. 4.33 show that the proposed BiGRU model has the best accuracy among the other models. For a quantitative assessment, Table 4.19 is plotted to provide the RMSE, MAE, and $R^2$ values of the proposed model and other benchmarks.

From Table 4.19, The proposed model is found to outperform the rest of the models. The provided results of the proposed model show an RMSE=0.005 and MAE=0.003. The acquired RMSE and MAE results are illustrated in Fig. 4.34.

From Fig. 4.34, it has been clearly shown that the proposed BiGRU model has the lowest error RMSE and MAE values. Consequently, the proposed BiGRU model's performance optimized by the SA algorithm is demonstrated to be an excellent competitive model for point GSI prediction.

## Interval prediction

The inherent uncertainty in predicting GSI is essentially conducted to quantify the uncertainties of the PVPF associated with point forecasting. Interval forecasting is a challenging task due to the weather parameters'

<table><caption>Table 4.19: Comparative study of various prediction techniques [6].</caption><thead><tr><th>Model</th><th>BiGRU</th><th>LSTM</th><th>XGB</th><th>LGBM</th><th>ANN</th></tr></thead><tbody><tr><td>R<sup>2</sup>(%)</td><td>97.98</td><td>90.99</td><td>47.25</td><td>29.49</td><td>92.72</td></tr><tr><td>RMSE</td><td>0.005</td><td>0.011</td><td>0.019</td><td>0.018</td><td>0.009</td></tr><tr><td>MAE</td><td>0.003</td><td>0.007</td><td>0.015</td><td>0.015</td><td>0.007</td></tr></tbody></table>
<figure><img src="image_70.png" /><figcaption>Figure 4.33: Performance analysis of prediction models [6].</figcaption></figure>

<figure><img src="image_71.png" /><figcaption>Figure 4.34: Error variations [6].</figcaption></figure>

stochastic variation during the days and seasons of the year. The unavoidability of point forecasting errors has significantly intensified the inherent role of interval forecasting. In this study, prediction intervals are used for quantitative characterizations of GSI forecasts. The loss function $\ell$ of single-point forecasts in quantile regression is defined as:

$$ \ell(\xi|\alpha) = \begin{cases} \alpha\xi_i & \xi \ge 0 \\ (\alpha - 1)\xi_i, & \xi < 0 \end{cases} \qquad (4.3) $$

where $\xi$ is the quantile value quantified between 0 and 1 and $\xi$ is calculated as:

$$ \xi = y_i - f(x_i) \qquad (4.4) $$
<table><caption>Table 4.20: Performance evaluation of interval forecasting [6].</caption><thead><tr><th>Model</th><th>PINC(%)</th><th>PICP(%)</th><th>ACE(%)</th></tr></thead><tbody><tr><td>LSTM</td><td>90</td><td>92.06</td><td>2.06</td></tr><tr><td>GRU</td><td>90</td><td>91.67</td><td>1.67</td></tr><tr><td>BiGRU</td><td>90</td><td>90.23</td><td>0.23</td></tr></tbody></table>

Here $f(x)$ is the forecasted quantile model. The average function for a given data set is computed as:

$$ \ell(y, |\alpha) = \frac{1}{N} \sum_{i=1}^{N} \ell(y_i - f(x_i)|\alpha) \qquad (4.5) $$

After adding a risk uncertainty-based BiGRU model, the forecasting system becomes able to perform quantile regression. The interval forecasting model is implemented in Keras by modifying the loss function as Eq. 4.5 to provide uncertainties from point GSI prediction. The experimental results are the lower/upper quantiles obtained by the BiGRU model are estimated and shown in Fig. 4.36.

<figure><img src="image_72.png" /><figcaption>Figure 4.35: The constructed prediction intervals for grid stability index [6].</figcaption></figure>

As seen in Fig.4.36, Statistical analysis of the results shows that the interval forecasts follow the actual values with high accuracy. The interval error measures for the proposed BiGRU model compared to the basic LSTM and GRU models are resumed in Table 4.20.

The error measures are computed and shown in Table 4.20. Notably, for PINC=90%, the BiGRU generates a PICP=90.23%. The calculated ACE of the proposed BiGRU equals 0.23% which is the least error compared to LSTM and GRU, respectively. Regarding the error measures, a general conclusion can be drawn that the interval forecasting with BiGRU generates a higher performance in terms of lower ACE than the state-of-the-art.
4.3.2 Case study 2

This study tends to supplement the recent works by proposing an efficient Ensemble Stacking Classifier
(SEC)-based to cope with the grid instability with a minimum error ratio.

Proposed methodology

Stacking paradigm is a model combination technology that realizes the diversity of heterogeneous base
learners to generate an efficient and robust classifier [5]. The core intuition behind SEC model implies that
incorporating the outputs of different classifiers renders more accurate predictions. To this end, the proposed
model introduces a combination of XGBoost, Catboost, and LGBM to enhance the overall generalization
ability of SEC technique. By interpreting training data from various angles, the SEC model would achieve
an excellent performance of classification. A simplified flowchart of the detailed ensemble steps as illustrated
in Fig. 4.36:

<figure><img src="image_73.png" /><figcaption>Figure 4.36: Flowchart of the proposed ensemble method [7].</figcaption></figure>

According to Fig. 4.36, XGBoost and Catboost models are selected in the first base learners to their
high performance in related power systems applications [5]. In the first stage, each of these models conducts
a leave-one-out or cross-validation (CV) in training set to generate new meta-features for the meta-learner
bloc [5]. In the second stage, these meta-features were mapped to the real classification tag. A new training
set and testing set were formed to train the meta-learner. LGBM was selected as the meta-learner to train a new model to perform the final prediction. It can be mentioned that the goal of the five-fold CV in the first stage is to ensure that the diversity distribution of the new training set and the new testing set. Thus, the meta-learner model acquires diverse feature representations and directly combines the multiple sources of features as inputs.

## Feature engineering

A grid stability simulation database is employed to validate the proposed SEC approach. This dataset is collected from the 4-node network with star topology by the Department of Computer Science, Karlsruhe Institute of Technology [153]. The data is open-sourced from the UCI database repository electrical grid stability simulated dataset [153]. Each item stands for the predictive attributes on the scale of [0, 10]. The dataset contains 10000 samples representing local stability analysis. The collected inputs include 12 features, specifically, : the reaction time of the energy producer (tau 1) and three consumers (tau2, tau3, tau4), the nominal power produced (p1) or consumed (p2, p3, p4), the price elasticity coefficient for each network participant (g1, g2, g3, g4). The system's output (stab) denotes the state of the grid (stable/unstable). The acquired data is cleaned from empty and misleading samples. Since the data has different scales, data normalization is essential to unify the range of data. In our problem, the data ranges between 0 and 1 according to the following formula [4].

$$x_n = \frac{x_r - x_{min}}{x_{max} - x_{min}} \qquad (4.6)$$

Where; $x_t$ denotes the normalized value, $x_r$ is the real value. Here, $x_{min}$, $x_{max}$ the minimum and maximum values. The final data is divided into three folds; 70% of the data is devoted to training and validation. The remaining 30% are used for testing purposes.

## Simulation results

The simulation results are conducted in Python in a Lenovo laptop; i9 generation, NVIDIA GeForce GTX 1650. The hyperparameters of the proposed model and the benchmarks are tuned using Random Search method. The ten-fold cross-validation (10-CV) results are provided in Table 4.21.
<table><caption>Table 4.21: Accuracy measures for SEC for 10-CV. FN, M, and SD are the fold number, mean, and standard derivation respectively [7].</caption><thead><tr><th>FN</th><th>Accuracy (%)</th><th>AUC (%)</th><th>Recall (%)</th><th>Prec. (%)</th><th>F1 (%)</th><th>Kappa (%)</th></tr></thead><tbody><tr><td>0</td><td>99.01</td><td>99.95</td><td>99.29</td><td>99.14</td><td>99.22</td><td>97.83</td></tr><tr><td>1</td><td>98.74</td><td>99.94</td><td>99.14</td><td>98.88</td><td>99.01</td><td>97.27</td></tr><tr><td>2</td><td>99.05</td><td>99.96</td><td>99.29</td><td>99.22</td><td>99.25</td><td>97.94</td></tr><tr><td>3</td><td>98.88</td><td>99.96</td><td>98.92</td><td>99.33</td><td>99.12</td><td>97.58</td></tr><tr><td>4</td><td>98.93</td><td>99.95</td><td>99.18</td><td>99.14</td><td>99.16</td><td>97.68</td></tr><tr><td>5</td><td>98.9</td><td>99.95</td><td>99.37</td><td>98.92</td><td>99.14</td><td>97.63</td></tr><tr><td>6</td><td>99.21</td><td>99.97</td><td>99.37</td><td>99.4</td><td>99.38</td><td>98.3</td></tr><tr><td>7</td><td>99.1</td><td>99.96</td><td>99.25</td><td>99.33</td><td>99.29</td><td>98.04</td></tr><tr><td>8</td><td>98.76</td><td>99.92</td><td>99.1</td><td>98.96</td><td>99.03</td><td>97.32</td></tr><tr><td>9</td><td>99.05</td><td>99.96</td><td>99.07</td><td>99.44</td><td>99.25</td><td>97.94</td></tr><tr><td>M</td><td>98.96</td><td>99.95</td><td>99.2</td><td>99.18</td><td>99.19</td><td>97.75</td></tr><tr><td>SD</td><td>0.14</td><td>0.01</td><td>0.14</td><td>0.19</td><td>0.11</td><td>0.31</td></tr></tbody></table>

The overall accuracy achieved by the proposed SEC model is 98.96%±0.14, while the precision is 99.18%±0.19. In order to verify the competitiveness of the proposed model, a series of tests have been conducted. The proposed model is compared to single models, including LGBM, XGB. Furthermore, Random Forest (RF), Multilayer Perception (MLP), Support Vector Classifier (SVC), and K-Nearest Neighbors (KNN) were simulated and compared to the proposed model. Table 4.22 presents the confusion matrix of the proposed model and benchmarks.
<table><caption>Table 4.22: Confusion Matrix for the proposed model and benchmarks [7].</caption><thead><tr><th>Classifier</th><th>Label</th><th>Stable</th><th>Instable</th></tr></thead><tbody><tr><td rowspan="2">Proposed</td><td>Stable</td><td>6426</td><td>90</td></tr><tr><td>Instable</td><td>79</td><td>11406</td></tr><tr><td rowspan="2">LGBM</td><td>Stable</td><td>6093</td><td>423</td></tr><tr><td>Instable</td><td>293</td><td>11192</td></tr><tr><td rowspan="2">XGB</td><td>Stable</td><td>5621</td><td>895</td></tr><tr><td>Instable</td><td>385</td><td>11100</td></tr><tr><td rowspan="2">RF</td><td>Stable</td><td>5967</td><td>549</td></tr><tr><td>Instable</td><td>802</td><td>10683</td></tr><tr><td rowspan="2">MLP</td><td>Stable</td><td>6394</td><td>122</td></tr><tr><td>Instable</td><td>346</td><td>11139</td></tr><tr><td rowspan="2">ET</td><td>Stable</td><td>5943</td><td>573</td></tr><tr><td>Instable</td><td>182</td><td>11303</td></tr><tr><td rowspan="2">SVC</td><td>Stable</td><td>4916</td><td>1600</td></tr><tr><td>Instable</td><td>1910</td><td>9575</td></tr><tr><td rowspan="2">KNN</td><td>Stable</td><td>4589</td><td>1927</td></tr><tr><td>Instable</td><td>1306</td><td>10179</td></tr></tbody></table>

According to Table 4.22, the numerical results validate the proposed SEC model's superior performance. The proposed SEC model is found to be very efficient for the SG stability system. From the total number of stable instances, 6426 instances are accurately classified as true positives and 11406 true negatives. The proposed model misclassifies only 90 false positives and 79 false negatives. Comparing with the single LGBM, 6093 true positives and 11192 true negatives are only detected. For more comparison analysis, Table 4.23 illustrates the proposed technique's metric scores with other benchmarks based on the 10-CV analysis.
<table><caption>Table 4.23: Comparative study of various prediction techniques [7].</caption><thead><tr><td>Model</td><td>Accuracy(%)</td><td>Recall(%)</td><td>Precision(%)</td><td>F1 score(%)</td></tr></thead><tbody><tr><td>LGBM</td><td>96</td><td>97.56</td><td>96.22</td><td>96.88</td></tr><tr><td>Catboost</td><td>98.36</td><td>98.97</td><td>98.47</td><td>98.72</td></tr><tr><td>XGB</td><td>93.06</td><td>96.97</td><td>92.52</td><td>94.69</td></tr><tr><td>ET</td><td>95.59</td><td>98.32</td><td>94.94</td><td>96.6</td></tr><tr><td>MLP</td><td>97.25</td><td>98.13</td><td>97.59</td><td>97.85</td></tr><tr><td>RF</td><td>94.49</td><td>96.57</td><td>94.87</td><td>95.71</td></tr><tr><td>SVC</td><td>79.82</td><td>88.27</td><td>82.03</td><td>84.74</td></tr><tr><td>KNN</td><td>82.03</td><td>88.57</td><td>84.11</td><td>86.28</td></tr><tr><td>Proposed</td><td>98.96</td><td>99.2</td><td>99.18</td><td>99.19</td></tr></tbody></table>

Regarding Table 4.23, the proposed SEC model is clearly outperforming the rest of the models. The proposed method achieved almost 3% higher accuracy when compared to LGBM. The Closest performance to the proposed technique is conducted by Catboost model, which achieved 98.36% of overall accuracy. The supremacy of the proposed SEC can also be presented through a bar graph representation in Fig. 4.37.

<figure><img src="image_74.png" /><figcaption>Figure 4.37: Comparative results [7].</figcaption></figure>
As clearly seen in Fig 4.37, the proposed SEC model achieved the best accuracy, recall, and precision results. To visualize the F1 score variation, Fig. 4.38 is presented.

<figure><img src="image_75.png" /><figcaption>Figure 4.38: Radar plot comparison for the proposed model with other benchmarks based on F1 score [7].</figcaption></figure>

As illustrated from 4.38, the proposed SEC model achieved the highest F1 score. In order to quantify the metric scores for stable and unstable classes, Table 4.24 is presented.
<table><caption>Table 4.24: Classification report for the proposed model and benchmarks [7].</caption><thead><tr><th>Classifier</th><th>Label</th><th>Precision (%)</th><th>Recall(%)</th><th>F1-score(%)</th></tr></thead><tbody><tr><td rowspan="2">GRU</td><td>Stable</td><td>93</td><td>100</td><td>96</td></tr><tr><td>Instable</td><td>100</td><td>96</td><td>98</td></tr><tr><td rowspan="2">LSTM</td><td>Stable</td><td>93</td><td>99</td><td>96</td></tr><tr><td>Instable</td><td>99</td><td>96</td><td>98</td></tr><tr><td rowspan="2">MLSTM</td><td>Stable</td><td>97</td><td>100</td><td>99</td></tr><tr><td>Instable</td><td>100</td><td>99</td><td>99</td></tr><tr><td rowspan="2">LGBM</td><td>Stable</td><td>95.4</td><td>93.7</td><td>94.6</td></tr><tr><td>Instable</td><td>96.5</td><td>97.4</td><td>97</td></tr><tr><td rowspan="2">XGB</td><td>Stable</td><td>98.5</td><td>97.4</td><td>97.9</td></tr><tr><td>Instable</td><td>98.5</td><td>99.1</td><td>98.8</td></tr><tr><td rowspan="2">RF</td><td>Stable</td><td>78.9</td><td>93.9</td><td>85.8</td></tr><tr><td>Instable</td><td>83.8</td><td>55.8</td><td>67</td></tr><tr><td rowspan="2">MLP</td><td>Stable</td><td>96.1</td><td>96</td><td>96</td></tr><tr><td>Instable</td><td>97.7</td><td>97.8</td><td>97.7</td></tr><tr><td rowspan="2">ET</td><td>Stable</td><td>95.1</td><td>98.3</td><td>96.7</td></tr><tr><td>Instable</td><td>96.8</td><td>91.1</td><td>93.9</td></tr><tr><td rowspan="2">SVC</td><td>Stable</td><td>83.8</td><td>55.8</td><td>67</td></tr><tr><td>Instable</td><td>78.9</td><td>93.9</td><td>85.8</td></tr><tr><td rowspan="2">KNN</td><td>Stable</td><td>77.8</td><td>70.6</td><td>74</td></tr><tr><td>Instable</td><td>84.1</td><td>88.6</td><td>86.3</td></tr><tr><td rowspan="2"><strong>Proposed</strong></td><td>Stable</td><td>99.1</td><td>98.9</td><td>99</td></tr><tr><td>Instable</td><td>99.4</td><td>99.5</td><td>99.4</td></tr></tbody></table>

According to Table 4.24, the proposed SEC model achieved the highest results with an accuracy of 99.1% and 99.4% for Stable (S) and unstable (F) classes, respectively. For better visualization of the metric scores, Fig 4.39 is presented.

According to Fig. 4.39, it is clearly shown that the proposed technique has the highest performance
<figure><img src="image_76.png" /><figcaption>Figure 4.39: Score performance comparison [7].</figcaption></figure>

metrics. In order to show the computational timing, Table 4.25 resumes the training and testing time of the proposed SEC and some other benchmarks.

<table><caption>Table 4.25: Computational time (second) for PGS prediction [7].</caption><thead><tr><th>Time</th><th>MLP</th><th>SVC</th><th>XGB</th><th>LGBM</th><th>Proposed</th></tr></thead><tbody><tr><td>Training (s)</td><td>416.41</td><td>4.54</td><td>29.86</td><td>7.91</td><td>6485.39</td></tr><tr><td>Testing (s)</td><td>0.93</td><td>1.18</td><td>0.61</td><td>0.65</td><td>2.42</td></tr></tbody></table>

According to Table 4.25, the SEC requires a long time to be sufficiently trained (6485 seconds). However, the testing time is 2.42 seconds which is considered fast. Therefore, the most significant shortcoming of the proposed SEC is resumed in its computational burden. However, this burden is compensated by its high efficiency and effectiveness to predict the PGS, which can be helpful in various power system aspects.

## 4.4 Summary

Load forecasting and power grid stability play a major role in the energy management paradigm due to the bulk integration of PV energy. However, Load forecasting is challenging due to the volatile nature of the end-users. This chapter dives into the nitty-gritty of load patterns to propose efficient forecasting methods for condition monitoring and control management. Firstly, two hybrid methods have been proposed for STLF.
The first hybrid model employs the stacked generalization approach between XGB, LGBM, and MLP. The Stacked XGB-LGBM-MLP has outperformed 11 reference models. It worth noting that a comparative study with HO methods reveals that the Random Search method can achieve the optimal model configuration despite being computationally extensive. The second approach lies in the combination of a deep learning model, namely, PLS-CNN-BiLSTM. The numerical results demonstrate the performance capability of deep learning methods to achieve state-of-the-art performance for single and multi-step predictions. Next, the grid stability assessment has been conducted. An accurate classifier-based LGBM-XGB and Catboost have been proposed to detect and identify the stabilities in the grid. Moreover, an efficient regressor-based SA-BiGRU method has been introduced to predict the stability index accurately. It was shown through several case studies that on real datasets the supremacy and high efficiency of the proposed models compared to available techniques.

The main content of this chapter has been published in [6], [7], [5], and [112].
# Chapter 5

## Conclusion

This chapter concludes the dissertation and propose several directions for future research work.

### 5.1 Summary

The main objective of this dissertation is to provide a set of efficient forecasting tools to estimate the PV power variation. Beyond this, our ultimate goal consists of proposing innovative prediction tools for the load demand side to maintain power grid stability. These prediction paradigms work together for enhancing the energy management service between the supply side and load side under the SG umbrella.

To achieve these targets, the first chapter addresses energy management and SG's status while focusing on the recent progress of research on artificial intelligence techniques. As reported, SG lies at the core of energy management systems using advanced methodologies from the border spectrum of information and communication technologies. Several intelligent methods have been analyzed, including Fuzzy logic, Shallow networks, ensemble methods, DL methods, and Hybrid methods. It has been found that DL-based PVPF emerged nowadays as a promising research direction. With the enormous integration of smart meters in electrical grids, DL takes advantage of the large-scale and multi-source data representations to achieve a spectacular performance and high forecastability potential compared to the the-state-of-the-art.

In the second chapter, Two case studies have been depicted for PVPF using meteorological data. The first case proposed a DL model combining Long-Short Term Memory (LSTM), Nonlinear Auto-Regressive exogenous model (NARX), and Tabu Search algorithm, named NARX-LSTM. The NARX-LSTM achieved state-of-the-art performance for both point and interval forecasting for small and large-scale PV systems. In the second case study, an ensemble learning model has been proposed using Bayesian Regression, Continuous Wavelet Transform, and Categorical boosting. The ensemble model above analyses the feature correlations to predict the point PV power in the short-term range. These studies can pinpoint that the PVPF can be accurately performed using both ensemble models and DL models. However, the supply side variations from the PV generators could impact the demand side and end-consumers. Thus, demand management is
essential to use PV energy production wisely.

To bridge this knowledge gap, the third chapter proposes two innovative methods for load forecasting. The first proposed model makes use of a hybrid DL model combining convolutional Bidirectional LSTM, Savitzky-Golay filter, Partial Least square method, and evolution strategy optimizer. In comparison, the second model employs Light gradient boosting, Multilayer perceptron, and Extreme gradient boosting methods. The proposed hybrid models have achieved excellent performance results for one-day-ahead load forecasting. However, it can be mentioned that the HO step is primordial for their effectiveness. To achieve an optimal balance between the load demand and energy supply without compromising the grid stability prediction (GSP), two efficient GSP techniques were proposed assessment, classification, and prediction of the state of stability using ML techniques. The proposed methods' high performance was demonstrated using the electrical grid stability simulated dataset.

## 5.2 Future work

The future work of this thesis will embrace the following issues:

* Proposing efficient ML techniques for energy systems' reliability assessment and control.

* Big Data tools will be considered to foster the synergy between the large availability of data from smart meters and ML techniques to use these data for efficient energy management systems.

* Investigation of the implementation of advanced technologies with less training data requirement and more performance capabilities on embedded systems for commercialization purposes.
References

[1] M. Massaoudi, S. S. Refaat, and H. Abu-Rub, “On the pivotal role of artificial intelligence toward the evolution of smart grids: A review of advanced methodologies and applications,” *Smart Grid and Enabling Technologies*, pp. 359–420, 2021.

[2] M. Massaoudi, I. Chihi, L. Sidhom, M. Trabelsi, S. S. Refaat, H. Abu-Rub, and F. S. Oueslati, “An effective hybrid narx-lstm model for point and interval pv power forecasting,” *IEEE Access*, vol. 9, pp. 36571–36588, 2021.

[3] M. Massaoudi, S. S. Refaat, H. Abu-Rub, I. Chihi, and F. S. Wesleti, “A hybrid bayesian ridge regression-cwt-catboost model for pv power forecasting,” in *2020 IEEE Kansas Power and Energy Conference (KPEC)*, pp. 1–5, IEEE, 2020.

[4] M. Massaoudi, S. S Refaat, H. Abu-Rub, I. Chihi, and F. S. Oueslati, “Pls-cnn-bilstm: An end-to-end algorithm-based savitzky–golay smoothing and evolution strategy for load forecasting,” *Energies*, vol. 13, no. 20, p. 5464, 2020.

[5] M. Massaoudi, S. S. Refaat, I. Chihi, M. Trabelsi, F. Oueslati, and H. Abu-Rub, “A novel stacked generalization ensemble-based hybrid lgbm-xgb-mlp model for short-term load forecasting,” *Energy*, 2020.

[6] M. Massaoudi, H. Abu-Rub, S. S Refaat, I. Chihi, and F. S. Oueslati, “Accurate smart-grid stability forecasting based on deep learning: Point and interval estimation method,” in *IEEE Kansas Power and Energy Conference (KPEC) 2021*, IEEE, 2021.

[7] M. Massaoudi, H. Abu-Rub, S. S Refaat, I. Chihi, and F. S. Oueslati, “An effective ensemble learning approach-based grid stability assessment and classification,” in *IEEE Kansas Power and Energy Conference (KPEC) 2021*, IEEE, 2021.

[8] A. Shahsavari and M. Akbari, “Potential of solar energy in developing countries for reducing energy-related emissions,” *Renewable and Sustainable Energy Reviews*, vol. 90, pp. 275–291, 2018.

[9] Y. Geng, W. Chen, Z. Liu, A. S. Chiu, W. Han, Z. Liu, S. Zhong, Y. Qian, W. You, and X. Cui, “A bibliometric review: Energy consumption and greenhouse gas emissions in the residential sector,” *Journal of cleaner production*, vol. 159, pp. 301–316, 2017.

[10] A. Zervos, C. Lins, and J. Muth, *RE-thinking 2050: a 100% renewable energy vision for the European Union*. Erec, 2010.

[11] M. N. Akhter, S. Mekhilef, H. Mokhlis, and N. M. Shah, “Review on forecasting of photovoltaic power generation based on machine learning and metaheuristic techniques,” *IET Renewable Power Generation*, vol. 13, no. 7, pp. 1009–1023, 2019.

[12] L. Kumar, M. Hasanuzzaman, and N. Rahim, “Global advancement of solar thermal energy technologies for industrial process heat and its future prospects: A review,” *Energy Conversion and Management*, vol. 195, pp. 885–908, 2019.
[13] E. G. Cojocaru, J. M. Bravo, M. J. Vasallo, and D. M. Santos, "Optimal scheduling in concentrating solar power plants oriented to low generation cycling," *Renewable Energy*, vol. 135, pp. 789-799, 2019.

[14] K. M. Powell, K. Rashid, K. Ellingwood, J. Tuttle, and B. D. Iverson, "Hybrid concentrated solar thermal power systems: A review," *Renewable and Sustainable Energy Reviews*, vol. 80, pp. 215-237, 2017.

[15] U. K. Das, K. S. Tey, M. Seyedmahmoudian, S. Mekhilef, M. Y. I. Idris, W. Van Deventer, B. Horan, and A. Stojcevski, "Forecasting of photovoltaic power generation and model optimization: A review," *Renewable and Sustainable Energy Reviews*, vol. 81, pp. 912-928, 2018.

[16] R. Ahmed, V. Sreeram, Y. Mishra, and M. Arif, "A review and evaluation of the state-of-the-art in pv solar power forecasting: Techniques and optimization," *Renewable and Sustainable Energy Reviews*, vol. 124, p. 109792, 2020.

[17] R. Vinuesa, H. Azizpour, I. Leite, M. Balaam, V. Dignum, S. Domisch, A. Felländer, S. D. Langhans, M. Tegmark, and F. F. Nerini, "The role of artificial intelligence in achieving the sustainable development goals," *Nature communications*, vol. 11, no. 1, pp. 1-10, 2020.

[18] M. Q. Raza and A. Khosravi, "A review on artificial intelligence based load demand forecasting techniques for smart grid and buildings," *Renewable and Sustainable Energy Reviews*, vol. 50, pp. 1352-1372, 2015.

[19] M. Massaoudi, H. Abu-Rub, S. S Refaat, I. Chihi, and F. S. Oueslati, "Deep learning in smart grid technology: A review of recent advancements and future prospects," *IEEE Access*, vol. 9, 2021.

[20] S. S. Ali and B. J. Choi, "State-of-the-art artificial intelligence techniques for distributed smart grids: A review," *Electronics*, vol. 9, no. 6, p. 1030, 2020.

[21] H.-M. Chung, S. Maharjan, Y. Zhang, and F. Eliassen, "Distributed deep reinforcement learning for intelligent load scheduling in residential smart grid," *IEEE Transactions on Industrial Informatics*, 2020.

[22] K. Aurangzeb, M. Alhussein, K. Javaid, and S. I. Haider, "A pyramid-cnn based deep learning model for power load forecasting of similar-profile energy customers based on clustering," *IEEE Access*, vol. 9, pp. 14992-15003, 2021.

[23] M. Massaoudi, S. S. Refaat, I. Chihi, M. Trabelsi, H. Abu-Rub, and F. S. Oueslati, "Short-term electric load forecasting based on data-driven deep learning techniques," in *IECON 2020 The 46th Annual Conference of the IEEE Industrial Electronics Society*, pp. 2565-2570, IEEE, 2020.

[24] M. Khodayar, G. Liu, J. Wang, and M. E. Khodayar, "Deep learning in power systems research: A review," *CSEE Journal of Power and Energy Systems*, 2020.

[25] M. Massaoudi, I. Chihi, L. Sidhom, M. Trabelsi, and F. S. Oueslati, "Medium and long-term parametric temperature forecasting using real meteorological data," in *IECON 2019-45th Annual Conference of the IEEE Industrial Electronics Society*, vol. 1, pp. 2402-2407, IEEE, 2019.

[26] Q. Sun, N. Zhang, S. You, and J. Wang, "The dual control with consideration of security operation and economic efficiency for energy hub," *IEEE Transactions on Smart Grid*, vol. 10, no. 6, pp. 5930-5941, 2019.

[27] M. Albano, L. L. Ferreira, and L. M. Pinho, "Convergence of smart grid ict architectures for the last mile," *IEEE Transactions on Industrial Informatics*, vol. 11, no. 1, pp. 187-197, 2015.

[28] S. C. Müller, H. Georg, J. J. Nutaro, E. Widl, Y. Deng, P. Palensky, M. U. Awais, M. Chenine, M. Küch, M. Stifter, et al., "Interfacing power system and ict simulators: Challenges, state-of-the-art, and case studies," *IEEE Transactions on Smart Grid*, vol. 9, no. 1, pp. 14-24, 2016.
[29] M. You, X. Zhang, G. Zheng, J. Jiang, and H. Sun, "A versatile software defined smart grid testbed: Artificial intelligence enhanced real-time co-evaluation of ict systems and power systems," *IEEE Access*, vol. 8, pp. 88651-88663, 2020.

[30] Z. Ullah, F. Al-Turjman, L. Mostarda, and R. Gagliardi, "Applications of artificial intelligence and machine learning in smart cities," *Computer Communications*, 2020.

[31] E. S. Rigas, S. D. Ramchurn, and N. Bassiliades, "Managing Electric Vehicles in the Smart Grid Using Artificial Intelligence: A Survey," *IEEE Transactions on Intelligent Transportation Systems*, vol. 16, no. 4, pp. 1619-1635, 2015.

[32] Q. Sun and L. Yang, "From independence to interconnection—a review of ai technology applied in energy systems," *CSEE Journal of Power and Energy Systems*, vol. 5, no. 1, pp. 21-34, 2019.

[33] B. K. Bose, "Artificial intelligence techniques in smart grid and renewable energy systems—some example applications," *Proceedings of the IEEE*, vol. 105, no. 11, pp. 2262-2273, 2017.

[34] P. E. Brockway, A. Owen, L. I. Brand-Correa, and L. Hardt, "Estimation of global final-stage energy-return-on-investment for fossil fuels with comparison to renewable energy sources," *Nature Energy*, vol. 4, no. 7, pp. 612-621, 2019.

[35] D. Bogdanov, J. Farfan, K. Sadovskaia, A. Aghahosseini, M. Child, A. Gulagi, A. S. Oyewo, L. d. S. N. S. Barbosa, and C. Breyer, "Radical transformation pathway towards sustainable electricity via evolutionary steps," *Nature communications*, vol. 10, no. 1, pp. 1-16, 2019.

[36] U. Energy and I. Administration, "International Energy Outlook," 2019.

[37] M. Mohamed-Seghir, A. Krama, S. S. Refaat, M. Trabelsi, and H. Abu-Rub, "Artificial intelligence-based weighting factor autotuning for model predictive control of grid-tied packed u-cell inverter," *Energies*, vol. 13, no. 12, p. 3107, 2020.

[38] A. Al Mamun, M. Sohel, N. Mohammad, M. S. H. Sunny, D. R. Dipta, and E. Hossain, "A comprehensive review of the load forecasting techniques using single and hybrid predictive models," *IEEE Access*, vol. 8, pp. 134911-134939, 2020.

[39] D. Syed, S. S. Refaat, H. Abu-Rub, O. Bouhali, A. Zainab, and L. Xie, "Averaging ensembles model for forecasting of short-term load in smart grids," in *2019 IEEE International Conference on Big Data (Big Data)*, pp. 2931-2938, IEEE, 2019.

[40] T. Ahmad, H. Zhang, and B. Yan, "A review on renewable energy and electricity requirement forecasting models for smart grid and buildings," *Sustainable Cities and Society*, vol. 55, p. 102052, 2020.

[41] S. D'silva, M. Shadmand, S. Bayhan, and H. Abu-Rub, "Towards grid of microgrids: Seamless transition between grid-connected and islanded modes of operation," *IEEE Open Journal of the Industrial Electronics Society*, vol. 1, pp. 66-81, 2020.

[42] G. Liang, S. R. Weller, J. Zhao, F. Luo, and Z. Y. Dong, "The 2015 ukraine blackout: Implications for false data injection attacks," *IEEE Transactions on Power Systems*, vol. 32, no. 4, pp. 3317-3318, 2016.

[43] A. Mosavi, M. Salimi, S. Faizollahzadeh Ardabili, T. Rabczuk, S. Shamshirband, and A. R. Varkonyi-Koczy, "State of the art of machine learning models in energy systems, a systematic review," *Energies*, vol. 12, no. 7, p. 1301, 2019.

[44] D. Kobak and P. Berens, "The art of using t-sne for single-cell transcriptomics," *Nature communications*, vol. 10, no. 1, pp. 1-14, 2019.

[45] D. Huang, C.-D. Wang, and J.-H. Lai, "Locally weighted ensemble clustering," *IEEE transactions on cybernetics*, vol. 48, no. 5, pp. 1460-1473, 2017.
[46] C. Fahy, S. Yang, and M. Gongora, "Ant colony stream clustering: A fast density clustering algorithm for dynamic data streams," *IEEE transactions on cybernetics*, vol. 49, no. 6, pp. 2215-2228, 2018.

[47] J. Chen and X. Ran, "Deep learning with edge computing: A review," *Proceedings of the IEEE*, vol. 107, no. 8, pp. 1655-1674, 2019.

[48] M. Ourahou, W. Ayrir, B. E. Hassouni, and A. Haddi, "Review on smart grid control and reliability in presence of renewable energies: challenges and prospects," *Mathematics and Computers in Simulation*, vol. 167, pp. 19-31, 2020.

[49] B. W. Wirtz, J. C. Weyerer, and C. Geyer, "Artificial intelligence and the public sector—applications and challenges," *International Journal of Public Administration*, vol. 42, no. 7, pp. 596-615, 2019.

[50] H. S. Boudet, "Public perceptions of and responses to new energy technologies," *nature energy*, vol. 4, no. 6, pp. 446-455, 2019.

[51] S. N. Fallah, R. C. Deo, M. Shojafar, M. Conti, and S. Shamshirband, "Computational intelligence approaches for energy load forecasting in smart energy management grids: state of the art, future challenges, and research directions," *Energies*, vol. 11, no. 3, p. 596, 2018.

[52] F. Chollet, *Deep Learning mit Python und Keras: Das Praxis-Handbuch vom Entwickler der Keras-Bibliothek* . MITP-Verlags GmbH & Co. KG, 2018.

[53] U. Hasson, S. A. Nastase, and A. Goldstein, "Direct fit to nature: An evolutionary perspective on biological and artificial neural networks," *Neuron*, vol. 105, no. 3, pp. 416-434, 2020.

[54] N. Ayoub, F. Musharavati, S. Pokharel, and H. A. Gabbar, "Ann model for energy demand and supply forecasting in a hybrid energy supply system," in *2018 IEEE International Conference on Smart Energy Grid Engineering (SEGE)*, pp. 25-30, IEEE, 2018.

[55] G. Notton, C. Voyant, A. Fouilloy, J. L. Duchaud, and M. L. Nivet, "Some applications of ann to solar radiation estimation and forecasting for energy applications," *Applied Sciences*, vol. 9, no. 1, p. 209, 2019.

[56] D. Jayamaha, N. Lidula, and A. D. Rajapakse, "Wavelet-multi resolution analysis based ann architec- ture for fault detection and localization in dc microgrids," *IEEE Access*, vol. 7, pp. 145371-145384, 2019.

[57] M. A. Hannan, Z. A. Ghani, M. M. Hoque, P. J. Ker, A. Hussain, and A. Mohamed, "Fuzzy logic inverter controller in photovoltaic applications: Issues and recommendations," *IEEE Access*, vol. 7, pp. 24934-24955, 2019.

[58] M. Rizwan, M. Jamil, S. Kirmani, and D. Kothari, "Fuzzy logic based modeling and estimation of global solar energy using meteorological parameters," *Energy*, vol. 70, pp. 685-691, 2014.

[59] A. Chojecki, M. Rodak, A. Ambroziak, and P. Borkowski, "Energy management system for residential buildings based on fuzzy logic: design and implementation in smart-meter," *IET Smart Grid*, vol. 3, no. 2, pp. 254-266, 2020.

[60] S. S. Reza, T. N. Mahbub, M. M. Islam, M. M. Arifeen, S. R. H. Remu, and D. A. Hossain, "Assuring cyber security in smart grid networks by fuzzy-logic based trust management model," in *2019 IEEE International Conference on Robotics, Automation, Artificial-intelligence and Internet-of-Things (RAAICON)*, pp. 1-4, IEEE.

[61] Y.-Y. Hong and P. M. P. Buay, "Robust design of type-2 fuzzy logic-based maximum power point tracking for photovoltaics," *Sustainable Energy Technologies and Assessments*, vol. 38, p. 100669, 2020.

[62] J.-S. Jang and C.-T. Sun, "Neuro-fuzzy modeling and control," *Proceedings of the IEEE*, vol. 83, no. 3, pp. 378-406, 1995.
[63] Y. Ren, P. Suganthan, and N. Srikanth, "Ensemble methods for wind and solar power forecasting—a state-of-the-art review," *Renewable and Sustainable Energy Reviews*, vol. 50, pp. 82–91, 2015.

[64] X. Huang, J. Shi, B. Gao, Y. Tai, Z. Chen, and J. Zhang, "Forecasting hourly solar irradiance using hybrid wavelet transformation and elman model in smart grid," *IEEE Access*, vol. 7, pp. 139909–139923, 2019.

[65] J. Shi, J. Guo, and S. Zheng, "Evaluation of hybrid forecasting approaches for wind speed and power generation time series," *Renewable and Sustainable Energy Reviews*, vol. 16, no. 5, pp. 3471–3480, 2012.

[66] O. Sagi and L. Rokach, "Ensemble learning: A survey," *Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery*, vol. 8, no. 4, p. e1249, 2018.

[67] C. Persson, P. Bacher, T. Shiga, and H. Madsen, "Multi-site solar power forecasting using gradient boosted regression trees," *Solar Energy*, vol. 150, pp. 423–436, 2017.

[68] D. H. Wolpert, "Stacked generalization," *Neural networks*, vol. 5, no. 2, pp. 241–259, 1992.

[69] H. Zhou, Y. Zhang, L. Yang, and Q. Liu, "Short-term photovoltaic power forecasting based on stacking-svm," in *2018 9th International Conference on Information Technology in Medicine and Education (ITME)*, pp. 994–998, IEEE, 2018.

[70] S. Li, P. Wang, and L. Goel, "A novel wavelet-based ensemble method for short-term load forecasting with hybrid neural networks and feature selection," *IEEE Transactions on power systems*, vol. 31, no. 3, pp. 1788–1798, 2015.

[71] J. Reston Filho, C. Affonso, and R. de Oliveira, "Energy price classification in north brazilian market using decision tree," in *2015 12th International Conference on the European Energy Market (EEM)*, pp. 1–5, IEEE, 2015.

[72] D. Zhang, L. Qian, B. Mao, C. Huang, B. Huang, and Y. Si, "A data-driven design for fault detection of wind turbines using random forests and xgboost," *IEEE Access*, vol. 6, pp. 21020–21031, 2018.

[73] Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning," *nature*, vol. 521, no. 7553, pp. 436–444, 2015.

[74] H. Wang, D. Amagata, T. Makeawa, T. Hara, N. Hao, K. Yonekawa, and M. Kurokawa, "A dnn-based cross-domain recommender system for alleviating cold-start problem in e-commerce," *IEEE Open Journal of the Industrial Electronics Society*, vol. 1, pp. 194–206, 2020.

[75] H. Wang, Z. Lei, X. Zhang, B. Zhou, and J. Peng, "A review of deep learning for renewable energy forecasting," *Energy Conversion and Management*, vol. 198, p. 111799, 2019.

[76] M. Massaoudi, I. Chihi, L. Sidhom, M. Trabelsi, S. S. Refaat, and F. S. Oueslati, "Performance evaluation of deep recurrent neural networks architectures: Application to pv power forecasting," in *2019 2nd International Conference on Smart Grid and Renewable Energy (SGRE)*, pp. 1–6, IEEE, 2019.

[77] F. Wang, Z. Zhang, C. Liu, Y. Yu, S. Pang, N. Duić, M. Shafie-Khah, and J. P. Catalão, "Generative adversarial networks and convolutional neural networks based weather classification model for day ahead short-term photovoltaic power forecasting," *Energy conversion and management*, vol. 181, pp. 443–462, 2019.

[78] K. Wang, C. Gou, Y. Duan, Y. Lin, X. Zheng, and F.-Y. Wang, "Generative adversarial networks: introduction and outlook," *IEEE/CAA Journal of Automatica Sinica*, vol. 4, no. 4, pp. 588–598, 2017.

[79] M. Afrasiabi, M. Mohammadi, M. Rastegar, and S. Afrasiabi, "Advanced deep learning approach for probabilistic wind speed forecasting," *IEEE Transactions on Industrial Informatics*, 2020.
[80] M. Khodayar, J. Wang, and M. Manthouri, "Interval deep generative neural network for wind speed forecasting," *IEEE Transactions on Smart Grid*, vol. 10, no. 4, pp. 3974-3989, 2018.

[81] H. Wang, W. Xue, Y. Liu, J. Peng, and H. Jiang, "Probabilistic wind power forecasting based on spiking neural network," *Energy*, vol. 196, p. 117072, 2020.

[82] N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, "Dropout: a simple way to prevent neural networks from overfitting," *The journal of machine learning research*, vol. 15, no. 1, pp. 1929-1958, 2014.

[83] H. H. Çevik and M. Çunkaş, "Short-term load forecasting using fuzzy logic and anfis," *Neural Computing and Applications*, vol. 26, no. 6, pp. 1355-1367, 2015.

[84] A. Pourdaryaei, H. Mokhlis, H. A. Illias, S. H. A. Kaboli, and S. Ahmad, "Short-term electricity price forecasting via hybrid backtracking search algorithm and anfis approach," *IEEE Access*, vol. 7, pp. 77674-77691, 2019.

[85] Y. Tao and H. Chen, "A hybrid wind power prediction method," in *2016 IEEE Power and Energy Society General Meeting (PESGM)*, pp. 1-5, IEEE, 2016.

[86] M. Kim, W. Choi, Y. Jeon, and L. Liu, "A hybrid neural network model for power demand forecasting," *Energies*, vol. 12, no. 5, p. 931, 2019.

[87] H. Shao, X. Deng, and F. Cui, "Short-term wind speed forecasting using the wavelet decomposition and adaboost technique in wind farm of east china," *IET Generation, Transmission & Distribution*, vol. 10, no. 11, pp. 2585-2592, 2016.

[88] Q. Wen, J. Gao, X. Song, L. Sun, H. Xu, and S. Zhu, "Robuststl: A robust seasonal-trend decomposition algorithm for long time series," in *Proceedings of the AAAI Conference on Artificial Intelligence*, vol. 33, 2019.

[89] Z. Hu, B. Tan, R. R. Salakhutdinov, T. M. Mitchell, and E. P. Xing, "Learning data manipulation for augmentation and weighting," in *Advances in Neural Information Processing Systems*, pp. 15764-15775, 2019.

[90] A. Botchkarev, "Performance metrics (error measures) in machine learning regression, forecasting and prognostics: Properties and typology," *arXiv preprint arXiv:1809.03006*, 2018.

[91] R. J. Hyndman and A. B. Koehler, "Another look at measures of forecast accuracy," *International journal of forecasting*, vol. 22, no. 4, pp. 679-688, 2006.

[92] J. S. Armstrong and F. Collopy, "Error measures for generalizing about forecasting methods: Empirical comparisons," *International journal of forecasting*, vol. 8, no. 1, pp. 69-80, 1992.

[93] A. A. Syntetos and J. E. Boylan, "The accuracy of intermittent demand estimates," *International Journal of Forecasting*, vol. 21, no. 2, pp. 303-314, 2005.

[94] M. Massaoudi, I. Chihi, H. Abu-Rub, S. S. Refaat, and F. S. Oueslati, "Convergence of photovoltaic power forecasting and deep learning: State-of-art review," *IEEE Access*, 2021.

[95] S. Mohanty, P. K. Patra, S. S. Sahoo, and A. Mohanty, "Forecasting of solar energy with application for a growing economy like india: Survey and implication," *Renewable and Sustainable Energy Reviews*, vol. 78, pp. 539-553, 2017.

[96] T. Hong, J. Wilson, and J. Xie, "Long term probabilistic load forecasting and normalization with hourly information," *IEEE Transactions on Smart Grid*, vol. 5, no. 1, pp. 456-462, 2013.

[97] M. Abdel-Nasser and K. Mahmoud, "Accurate photovoltaic power forecasting models using deep lstm-rnn," *Neural Computing and Applications*, vol. 31, no. 7, pp. 2727-2740, 2019.
[98] C. Tao, D. Shanxu, and C. Changsong, "Forecasting power output for grid-connected photovoltaic power system without using solar radiation measurement," in *The 2nd International Symposium on Power Electronics for Distributed Generation Systems*, pp. 773-777, IEEE, 2010.

[99] M. Diagne, M. David, P. Lauret, J. Boland, and N. Schmutz, "Review of solar irradiance forecasting methods and a proposition for small-scale insular grids," *Renewable and Sustainable Energy Reviews*, vol. 27, pp. 65-76, 2013.

[100] I. Sansa, S. Missaoui, Z. Boussada, N. M. Bellaaj, E. M. Ahmed, and M. Orabi, "Pv power forecasting using different artificial neural networks strategies," in *2014 First International Conference on Green Energy ICGE 2014*, pp. 54-59, IEEE, 2014.

[101] G. Lobaccaro, S. Carlucci, and E. Löfström, "A review of systems and technologies for smart homes and smart grids," *Energies*, vol. 9, no. 5, p. 348, 2016.

[102] E. Rangel, E. Cadenas, R. Campos-Amezcua, and J. L. Tena, "Enhanced prediction of solar radiation using NARX models with corrected input vectors," *Energies*, vol. 13, no. 10, pp. 1-22, 2020.

[103] Tsung-Nan Lin, C. Giles, B. Horne, and Sun-Yuan Kung, "A delay damage model selection algorithm for NARX neural networks," *IEEE Transactions on Signal Processing*, vol. 45, no. 11, pp. 2719-2730, 1997.

[104] S. A. Khan, V. Thakore, A. Behal, L. Bölöni, and J. J. Hickman, "Comparative analysis of system identification techniques for nonlinear modeling of the neuron-microelectrode junction," *Journal of Computational and Theoretical Nanoscience*, vol. 10, no. 3, pp. 573-580, 2013.

[105] J. M. P. Menezes Jr and G. A. Barreto, "Long-term time series prediction with the narx network: An empirical evaluation," *Neurocomputing*, vol. 71, no. 16-18, pp. 3335-3343, 2008.

[106] S. J. Abdulkadir and S. P. Yong, "Empirical analysis of parallel-NARX recurrent network for long-term chaotic financial forecasting," *2014 International Conference on Computer and Information Sciences, ICCOINS 2014 - A Conference of World Engineering, Science and Technology Congress, ESTCON 2014 - Proceedings*, pp. 0-5, 2014.

[107] H. T. Siegelmann, B. G. Horne, and C. L. Giles, "Computational capabilities of recurrent narx neural networks," *IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics)*, vol. 27, no. 2, pp. 208-215, 1997.

[108] S. Hochreiter, Jürgen Schmidhuber, "Long Short-Term Memory," *Neural Computation*, vol. 9(8), pp. 1735-1780, 1997.

[109] A. M. Mohammed and S. O. Duffuaa, "A tabu search based algorithm for the optimal design of multi-objective multi-product supply chain networks," *Expert Systems with Applications*, vol. 140, p. 112808, 2020.

[110] N. Srinidhi, T. Datta, A. Chockalingam, and B. S. Rajan, "Layered tabu search algorithm for large-mimo detection and a lower bound on ml performance," *IEEE Transactions on Communications*, vol. 59, no. 11, pp. 2955-2963, 2011.

[111] H.-J. Glaeske and V. K. Tuan, "Some applications of the convolution theorem of the hilbert transform," *Integral Transforms and Special Functions*, vol. 3, no. 4, pp. 263-268, 1995.

[112] M. Massaoudi, S. S. Refaat, H. Abu-Rub, I. Chihi, and F. S. Oueslati, "Pls-cnn-bilstm: An end-to-end algorithm-based savitzky-golay smoothing and evolution strategy for load forecasting," *Energies*, vol. 13, no. 20, 2020.

[113] "Desert knowledge australia solar centre (dkasc)." http://dkasolarcentre.com, Accessed on: 2019-09-23.
[114] P. Ingenhoven, G. Belluardo, G. Makrides, G. E. Georghiou, P. Rodden, L. Frearson, B. Herteleer, D. Bertani, and D. Moser, "Analysis of Photovoltaic Performance Loss Rates of Six Module Types in Five Geographical Locations," *IEEE Journal of Photovoltaics*, vol. 9, no. 4, pp. 1091-1096, 2019.

[115] G. C. Adele Kuzmiakova and A. McKeehan, "Machine-learning-for-solar-energy-prediction," Accessed on: 2020-08-20.

[116] H. Zou and T. Hastie, "Regularization and variable selection via the elastic net," *Journal of the Royal Statistical Society. Series B: Statistical Methodology*, vol. 67, no. 2, pp. 301-320, 2005.

[117] F. Chollet, "keras-team/keras: Deep learning for humans," Accessed on: 2020-05-27.

[118] G. Varoquaux, L. Buitinck, G. Louppe, O. Grisel, F. Pedregosa, and A. Mueller, "Scikit-learn," *Get-Mobile: Mobile Computing and Communications*, vol. 19, no. 1, pp. 29-33, 2015.

[119] Simon Blanke, "Hyperactive: A hyperparameter optimization and meta-learning toolbox for machine-deep-learning models." https://github.com/SimonBlanke, since 2019.

[120] L. Gigoni, A. Betti, E. Crisostomi, A. Franco, M. Tucci, F. Bizzarri, and D. Mucci, "Day-ahead hourly forecasting of power generation from photovoltaic plants," *IEEE Transactions on Sustainable Energy*, vol. 9, no. 2, pp. 831-842, 2017.

[121] S. Theocharides, G. Makrides, A. Livera, M. Theristis, P. Kaimakis, and G. E. Georghiou, "Day-ahead photovoltaic power production forecasting methodology based on machine learning and statistical post-processing," *Applied Energy*, vol. 268, p. 115023, 2020.

[122] W. Yin, Y. Han, H. Zhou, M. Ma, L. Li, and H. Zhu, "A novel non-iterative correction method for short-term photovoltaic power forecasting," *Renewable Energy*, 2020.

[123] F. Wang, Z. Xuan, Z. Zhen, K. Li, T. Wang, and M. Shi, "A day-ahead pv power forecasting method based on lstm-rnn model and time correlation modification under partial daily pattern prediction framework," *Energy Conversion and Management*, vol. 212, p. 112766, 2020.

[124] L. Zjavka, "Photo-voltaic power daily predictions using expanding pde sum models of polynomial networks based on operational calculus," *Engineering Applications of Artificial Intelligence*, vol. 89, p. 103409, 2020.

[125] Y. Zhang, C. Qin, A. K. Srivastava, C. Jin, and R. Sharma, "Data-driven day-ahead pv estimation using autoencoder-lstm and persistence model," *IEEE Transactions on Industry Applications*, pp. 1-8, 2020.

[126] Y. Wen, D. AlHakeem, P. Mandal, S. Chakraborty, Y. Wu, T. Senjyu, S. Paudyal, and T. Tseng, "Performance evaluation of probabilistic methods based on bootstrap and quantile regression to quantify pv power point forecast uncertainty," *IEEE Transactions on Neural Networks and Learning Systems*, vol. 31, no. 4, pp. 1134-1144, 2020.

[127] P. Mandal, S. T. S. Madhira, J. Meng, R. L. Pineda, et al., "Forecasting power output of solar photovoltaic system using wavelet transform and artificial intelligence techniques," *Procedia Computer Science*, vol. 12, pp. 332-337, 2012.

[128] A. V. Dorogush, V. Ershov, and A. Gulin, "Catboost: gradient boosting with categorical features support," *arXiv preprint arXiv:1810.11363*, 2018.

[129] V. Madaan and A. Goyal, "Predicting ayurveda-based constituent balancing in human body using machine learning methods," *IEEE Access*, vol. 8, pp. 65060-65070, 2020.

[130] M. Massaoudi, I. Chihi, L. Sidhom, M. Trabelsi, S. S. Refaat, and F. S. Oueslati, "Enhanced random forest model for robust short-term photovoltaic power forecasting using weather measurements," *Energies*, vol. 14, no. 13, p. 3992, 2021.
[131] M. Massaoudi, H. Abu-Rub, S. S. Refaat, M. Trabelsi, I. Chihi, and F. S. Oueslati, "Enhanced deep belief network based on ensemble learning and tree-structured of parzen estimators: An optimal photovoltaic power forecasting method," *IEEE Access*, vol. 9, pp. 150330–150344, 2021.

[132] A. A. Mamun, M. Sohel, N. Mohammad, M. S. H. Sunny, D. R. Dipta, and E. Hossain, "A Comprehensive Review of the Load Forecasting Techniques Using Single and Hybrid Predictive Models," *IEEE Access*, pp. 1–1, 2020.

[133] P. Koponen, J. Ikäheimo, J. Koskela, C. Brester, and H. Niska, "Assessing and comparing short term load forecasting performance," *Energies*, vol. 13, no. 8, p. 2054, 2020.

[134] L. Hernandez, C. Baladron, J. M. Aguiar, B. Carro, A. J. Sanchez-Esguevillas, J. Lloret, and J. Mas-sana, "A survey on electric power demand forecasting: future trends in smart grids, microgrids and smart buildings," *IEEE Communications Surveys & Tutorials*, vol. 16, no. 3, pp. 1460–1495, 2014.

[135] N. Hatziargyriou, J. Milanovic, C. Rahmann, V. Ajjarapu, C. Canizares, I. Erlich, D. Hill, I. Hiskens, I. Kamwa, B. Pal, et al., "Definition and classification of power system stability revisited & extended," *IEEE Transactions on Power Systems*, 2020.

[136] T.-T. Wong and P.-Y. Yeh, "Reliable accuracy estimates from k-fold cross validation," *IEEE Transactions on Knowledge and Data Engineering*, 2019.

[137] "ISO New England - Pricing Reports." https://www.iso-ne.com/isoexpress/web/reports, 2020-05-24.

[138] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, et al., "Scikit-learn: Machine learning in python," *the Journal of machine Learning research*, vol. 12, pp. 2825–2830, 2011.

[139] F. Chollet, *Deep Learning mit Python und Keras: Das Praxis-Handbuch vom Entwickler der Keras-Bibliothek*. MITP-Verlags GmbH & Co. KG, 2018.

[140] H. Xie, L. Zhang, and C. P. Lim, "Evolving cnn-lstm models for time series prediction using enhanced grey wolf optimizer," *IEEE Access*, vol. 8, pp. 161519–161541, 2020.

[141] H. Shi, M. Xu, and R. Li, "Deep Learning for Household Load Forecasting-A Novel Pooling Deep RNN," *IEEE Transactions on Smart Grid*, vol. 9, no. 5, pp. 5271–5280, 2018.

[142] D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," *arXiv preprint arXiv:1412.6980*, 2014.

[143] D. H. Wolpert, "Stacked generalization," *Neural Networks*, vol. 5, no. 2, pp. 241–259, 1992.

[144] Z. Ma and Q. Dai, "Selected an Stacking ELMs for Time Series Prediction," *Neural Processing Letters*, vol. 44, no. 3, pp. 831–856, 2016.

[145] X. Luo, J. Sun, L. Wang, W. Wang, W. Zhao, J. Wu, J. H. Wang, and Z. Zhang, "Short-term wind speed forecasting via stacked extreme learning machine with generalized correntropy," *IEEE Transactions on Industrial Informatics*, vol. 14, no. 11, pp. 4963–4971, 2018.

[146] R. Efendi, Z. Ismail, and M. M. Deris, "A new linguistic out-sample approach of fuzzy time series for daily forecasting of Malaysian electricity load demand," 2015.

[147] T. Carneiro, R. V. Medeiros, D. A. Nobrega, T. Nepomuceno, G. Bian, and V. H. C. D. E. Albuquerque, "Performance Analysis of Google Colaboratory as a Tool for Accelerating Deep Learning Applications," *IEEE Access*, pp. 1–9, 2018.

[148] M. Ali, "Home - pycaret." https://pycaret.org/, Accessed: 2020-08-03.

[149] I. Ivanov, "vecstack: Python package for stacking (machine learning technique)."
[150] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay, "Scikit-learn: Machine Learning in Python Fabian," *Journal of Machine Learning Research*, vol. 12, pp. 128–154, 2011.

[151] S. Seabold and J. Perktold, "Statsmodels: Econometric and Statistical Modeling with Python," *Proceedings of the 9th python in science conference*, no. Scipy, p. 57, 2010.

[152] H. J. Sadaei, P. C. d. L. e Silva, F. G. Guimaraes, and M. H. Lee, "Short-term load forecasting by using a combined method of convolutional neural networks and fuzzy time series," *Energy*, vol. 175, pp. 365–377, 2019.

[153] V. Arzamasov, "Smart grid stability." https://archive.ics.uci.edu/ml/machine-learning-databases/00471/, Accessed: 05-03-2019.