# Photovoltaic power Forecasting and management of energy production based on meteorological data

Mohamed Sadok Massaoudi

► To cite this version:

Mohamed Sadok Massaoudi. Photovoltaic power Forecasting and management of energy production based on meteorological data. Systems and Control [cs.SY]. Institut National des Sciences Appliquées et de Technologie - Carthage (Tunisie), 2022. English. NNT: tel-03648301

HAL Id: tel-03648301

https://hal.science/tel-03648301

Submitted on 21 Apr 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Ministry of Higher Education and Scientific Research

* _ * _ * _ * _ *

UNIVERSITY OF CARTHAGE

* _ * _ * _ * _ *

NATIONAL INSTITUTE OF APPLIED SCIENCES AND TECHNOLOGY

Photovoltaic Power Forecasting and Management of
Energy Production Based on Meteorological Data

Submitted in partial fulfillment of the requirements for
the degree of Doctor of Philosophy in Electronics

Defended on Friday, March 4, 2022, in Tunis, Tunisia by:

Mohamed Sadok Massaoudi

Doctoral Committee Members:

<table><tr><td>Mr. Slim Kaddeche</td><td>INSAT, Tunisia</td><td>Chair</td></tr><tr><td>Mr. AmenAllah Guizani</td><td>CRTEn, Tunisia</td><td>Reviewer</td></tr><tr><td>Mrs. Afef Ben Abdelghani</td><td>INSAT, Tunisia</td><td>Reviewer</td></tr><tr><td>Mr. Khaled Nouri</td><td>ENICarthage, Tunisia</td><td>Examiner</td></tr><tr><td>Mr. Fakhreddine S. Oueslati</td><td>ENICarthage, Tunisia</td><td>Supervisor</td></tr><tr><td>Mr. Haitham Abu-Rub</td><td>Texas A&M University at Qatar, Qatar</td><td>Co-supervisor</td></tr><tr><td>Mrs. Ines Chihi</td><td>University of Luxembourg, Luxembourg</td><td>Invited</td></tr></table>
PHOTOVOLTAIC POWER FORECASTING AND MANAGEMENT OF
ENERGY PRODUCTION BASED ON METEOROLOGICAL DATA

BY

MOHAMED SADOK MASSAOUDI

National Institute of Applied Science and Technology

DISSERTATION

Submitted in partial fulfillment of the requirements
for the degree of Doctor of Philosophy in Electronics
in Carthage university, March 4, 2022

Tunis, Tunisia
# List of publications

## 0.1 Journal articles

1. **M. Massaoudi**, S. S. Refaat, I. Chihi, M. Trabelsi, F. S. Oueslati, H. Abu-Rub, A novel stacked generalization ensemble-based hybrid LGBM-XGB-MLP model for Short-Term Load Forecasting, Energy Vol.214, 2021. DOI: 10.1016/j.energy.2020.118874

**Indexed ISI Thomson & Scopus; Impact Factor: 7.147**

2. **M. Massaoudi**, I. Chihi, L. Sidhom, M. Trabelsi, S. S. Refaat, H. Abu-Rub, F. S. Oueslati, An Effective Hybrid NARX-LSTM Model for Point and Interval PV Power Forecasting, IEEE Access Vol.8, 2021. DOI: 10.1109/ACCESS.2021.3062776

**Indexed ISI Thomson & Scopus; Impact Factor: 3.367**

3. **M. Massaoudi**, H. Abu-Rub, S. S. Refaat, I. Chihi, F. S. Oueslati, Deep Learning in Smart Grid Technology: A Review of Recent Advancements and Future Prospects, IEEE Access, Vol.9, 2021. DOI: 10.1109/ACCESS.2021.3071269

**Indexed ISI Thomson & Scopus; Impact Factor: 3.367**

4. **M. Massaoudi**, S. S. Refaat, H. Abu-Rub, I. Chihi, F. S. Oueslati, PLS-CNN-BiLSTM: An End-to-end Algorithm-Based Savitzky-Golay Smoothing and Evolution Strategy for Load Forecasting, Energies Vol.13, 2020. DOI: 10.3390/en13205464

**Indexed ISI Thomson & Scopus; Impact Factor: 3.004**

5. **M. Massaoudi**, I. Chihi, L. Sidhom, M. Trabelsi, S. S. Refaat, F. S. Oueslati, Enhanced Random Forest Model for Robust Short-Term Photovoltaic Power Forecasting Using weather measurements, Energies Vol.13, 2021. DOI: 10.3390/en14133992

**Indexed ISI Thomson & Scopus; Impact Factor: 3.004**
6. **M. Massaoudi**, I. Chihi, H. Abu-Rub, S. S. Refaat, F. S. Oueslati, Convergence of Photovoltaic Power Forecasting and Deep Learning: State-of-Art Review, IEEE Access, Vol.9, 2021. DOI: 10.1109/ACCESS.2021.3117004

**Indexed ISI Thomson & Scopus; Impact Factor: 3.367**

7. **M. Massaoudi**, H. Abu-Rub, S. S. Refaat, M. Trabelsi, I. Chihi, F. S. Oueslati, Enhanced Deep Belief Network Based on Ensemble Learning and Tree-Structured of Parzen Estimators: An Optimal Data-Driven PV Generation Forecasting, IEEE Access, Vol.9, 2021. DOI: 10.1109/ACCESS.2021.3125895

**Indexed ISI Thomson & Scopus; Impact Factor: 3.367**

8. **M. Massaoudi**, I. Chihi, L. Sidhom, M. Trabelsi, S. S. Refaat, F. S. Oueslati, A Novel Hybrid Symbolic Regressor-Deep Multilayer Perceptron Approach for PV Power Forecasting, IEEE Canadian Journal of Electrical and Computer Engineering, Under review

## 0.2 Conference proceedings

1. **M. Massaoudi**, S. S. Refaat, H. Abu-Rub, I. Chihi, F. S. Oueslati, A Hybrid Bayesian Ridge Regression CWT-Catboost Model For PV Power Forecasting. IEEE Kansas Power and Energy Conference (KPEC) 2020 (pp. 1-5). DOI: 10.1109/KPEC47870.2020.9167596

2. **M. Massaoudi**, S. S. Refaat, H. Abu-Rub, I. Chihi, Mohamed Trabelsi, F. S. Oueslati, Short-Term Electric Load Forecasting Based on Data-Driven Deep Learning Techniques. IECON 2020-45th Annual Conference of the IEEE Industrial Electronics Society. DOI: 10.1109/IECON43393.2020.9255098

3. **M. Massaoudi**, I. Chihi, L. Sidhom, M. Trabelsi, S. S. Refaat, F. S. Oueslati, Medium and Long-Term Parametric Temperature Forecasting using Real Meteorological Data. IECON 2019-45th Annual Conference of the IEEE Industrial Electronics Society. Vol. 1, pp. 2402-2407. DOI: 10.1109/IECON.2019.8927778

4. **M. Massaoudi**, I. Chihi, L. Sidhom, M. Trabelsi, S. S. Refaat, F. S. Oueslati, Performance Evaluation of Deep Recurrent Neural Networks Architectures: Application to PV Power Forecasting. 2nd International Conference on Smart Grid and Renewable Energy (SGRE) 2019 (pp. 1-6). DOI: 10.1109/SGRE46976.2019.9020965

5. **M. Massaoudi**, H. Abu-Rub, S. S. Refaat, I. Chihi, F. S. Oueslati, Accurate Smart-Grid Stability Forecasting Based on Deep Learning: Point and Interval Estimation Method. IEEE Kansas Power
and Energy Conference (KPEC) 2021 (pp. 1-6). IEEE. DOI: 10.1109/KPEC51835.2021.9446196

6. **M. Massaoudi**, H. Abu-Rub, S. S. Refaat, I. Chihi, F. S. Oueslati, An Effective Ensemble Learning approach-Based Grid Stability Assessment and Classification, IEEE Kansas Power and Energy Conference (KPEC) 2021 (pp. 1-6). IEEE. DOI: 10.1109/KPEC51835.2021.9446197
Abstract

Energy management is an indispensable part of today's electrical systems and Smart Grid (SG) paradigms, especially with the high penetration of Renewable Energy Sources (RES). Thus, significant attention is paid from academia and industry to foster the synergy between these two paradigms as a means to accelerate the transition to a more diverse generation portfolio that includes an unprecedented amount of RES such as Photovoltaic (often shortened as PV) energy. Due to the ever-growing electricity consumption, state-of-the-art Artificial Intelligence (AI)-based techniques play a central role in providing necessary system flexibility to deal with the bulk integration of the PV energy for power-and-energy-efficient computing. AI lies at the core of forecasting methods to enhance the power delivery service between the grid-connected PV stations and end-consumers. In other words, the futuristic power grid infrastructure should rely on accurate PV Power Forecasting (PVPF) methods as a cornerstone of achieving unit commitment and stable energy supply. Nevertheless, designing effective energy management systems is complex because it involves designing components and hardware-software interfaces across the computing stack. So ubiquitous and complex are energy management mechanisms requiring a high level of scalability and generalization potential to gratify the load needs and cope with the meteorological factors' stochastic nature in tandem with optimal power grid stability. In an effort to break this stalemate, this research aims to explore the potential AI techniques for energy management between the energy-mix on the supply side and the load side. Consequently, the present work proposes efficient techniques to tackle the instability and intermittency of PV power production and its significant impact on the load demand.

First, a comprehensive overview of SG and energy management has been conducted. Next, various ML models-based PVPF have been introduced and applied to real-world scenarios to ensure an uninterrupted power supply. Afterward, innovative load forecasting methods have been proposed to cope with the volatile PV energy supply and accommodate the stochasticity of the customers' demand with efficient energy management strategies. Finally, SG stability methods have been proposed to predict the grid's state. This will aid in shaping the best strategies for preventive maintenance and risk hedging policies. This research thesis applies data science methods to the SG paradigm for effective dynamic control and management.
# Résumé

La gestion de l'énergie est un élément indispensable des systèmes électriques d'aujourd'hui et des paradigmes des réseaux intelligents, en particulier avec la forte pénétration des sources d'énergies renouvelables. Ainsi, les universités et l'industrie accordent une attention particulière à la synergie entre ces deux paradigmes afin d'accélérer la transition vers un portefeuille de production plus diversifié comprenant une quantité importante d'énergies renouvelables telles que l'énergie Photovoltaïque (souvent abrégée en PV).

En raison de l'augmentation de la consommation énergetique, les techniques de pointe basées sur l'intelligence artificielle jouent un rôle central en fournissant la flexibilité nécessaire au système pour faire face à l'intégration en masse de l'énergie photovoltaïque. L'intelligence artificielle est au cœur des méthodes de prévision pour améliorer le service de livraison d'électricité entre les stations photovoltaïques connectées au réseau et les consommateurs finaux. En outre, l'infrastructure futuriste du réseau électrique devrait s'appuyer sur des méthodes précises de prévision de la puissance photovoltaïque pour l'engagement des unités et de l'approvisionnement énergétique stable.

Néanmoins, la conception de systèmes de gestion de l'énergie efficace est complexe, car elle implique la conception de composants et d'interfaces matériel-logiciel sur l'ensemble de la pile informatique. Les mécanismes de gestion de l'énergie sont si omniprésents et complexes, nécessitant un haut niveau d'évolutivité et un potentiel de généralisation pour satisfaire les besoins de charge et faire face à la nature stochastique des facteurs météorologiques en tandem avec une stabilité optimale du réseau électrique. Dans un effort pour sortir de cette impasse, cette recherche vise à explorer les techniques potentielles d'intelligence artificielle pour la gestion de l'énergie entre le mix énergétique entre l'offre énergétique et la demande énergétique. Par conséquent, le présent travail propose des techniques efficaces pour lutter contre l'instabilité et l'intermittence de la production d'énergie photovoltaïque et son impact significatif sur la demande de charge.

Tout d'abord, un aperçu complet du réseau intelligent et de la gestion de l'énergie a été réalisé. Ensuite, diverses méthodes de prévision de la puissance photovoltaïque basées sur l'apprentissage automatique ont été introduites et appliquées à des scénarios du monde réel pour assurer une alimentation électrique ininterrompue. Par la suite, des méthodes innovantes de prévision de la charge ont été proposées pour faire
face à l'approvisionnement en énergie photovoltaïque volatile et s'adapter à la stochasticité de la demande des clients avec des stratégies de gestion de l'énergie efficaces. Enfin, des méthodes de stabilité de réseau intelligent ont été proposées pour prédire l'état du réseau. Cela aidera à définir les meilleures stratégies pour les politiques de maintenance préventive et de couverture des risques. Cette thèse de recherche applique des méthodes de science des données au paradigme de réseau intelligents pour un contrôle et une gestion dynamiques efficaces.
Acknowledgments

The research presented in this dissertation benefited from the valuable insights and support of many people. It is my pleasure to express my sincere gratitude to all of them. First and foremost, I would like to express my deepest gratitude to my advisor and co-advisor, Professor Fakhreddine S. Oueslati and Professor Haitham Abu-Rub. It has been my privilege to have completed my studies under their supervision, and they continue to be my role model in professionalism and class throughout my life.

I would like to acknowledge the following members of my committee for their valuable support and feedback: Professor Slim Kaddeche from the National Institute of Applied Science and Technology, Tunisia, Professor AmenAllah Guizani from the Center for Energy Research and Technology (CRTEn), Tunisia, Associate Professor Afef Ben Abdelghani from the National Institute of Applied Science and Technology, Tunisia, and Professor Khaled Nouri from the National Engineering School of Carthage, Tunisia.

I am also very thankful for visiting Qatar and joining Smart Grid Center (SGC) Laboratory. Special thanks go to my co-supervisors, Dr. Ines Chihi and Dr. Mohamed Trabelsi, for their extreme guidance, patience, and support throughout my Ph.D. studies. I would like to acknowledge Dr. Shadi S. Khalil and Dr. Lilia Sidhom, who significantly guided me and provided me with essential support in my doctoral study and helped me keep positive during tough times. I am always grateful for the support of my brother and my friends through the up and downs of these past few years and shared many precious moments: Eng. Maher Sadok Massaoudi, Dr. Abdelbasset Krama, Dr. Ahmad Darwish, Dr. Qasim Khan, Dr. Rayane Khalid, Dr. Dabeeruddin Syed, and Eng. Lamjed Essid. My gratitude is extended to Dr. Imed Romdhani for his endless support, motivation, and encouragement. This dissertation's accomplishment, my personal and intellectual growth throughout these years would be impossible without these people mentioned above.

I wish to acknowledge the University of Carthage, National Institute of Applied Sciences and Technology (INSAT), National Engineering School of Carthage (ENICarthage), and Texas A&M university of Qatar, Electrical and Computer Engineering Department (ECEN), for providing an excellent environment to carry out this research.
Funding resources

Graduate study was supported by NPRP grants NPRP10-0101-170082 and NPRP10-0101-170085 from the Qatar National Research Fund (a member of Qatar Foundation), the co-funding by IBERDROLA QSTP LLC, sponsorship by Texas A&M Energy Institute Fellowship, and ministry of higher education and scientific research in Tunisia.
# Table of Contents

<table><tr><td>List of publications</td><td>iii</td></tr><tr><td>0.1 Journal articles</td><td>iii</td></tr><tr><td>0.2 Conference proceedings</td><td>iv</td></tr><tr><td>Acknowledgments</td><td>ix</td></tr><tr><td>Funding resources</td><td>x</td></tr><tr><td>List of Tables</td><td>xiii</td></tr><tr><td>List of Figures</td><td>xv</td></tr><tr><td>List of Abbreviations</td><td>xvii</td></tr><tr><td>Chapter 1 Introduction</td><td>1</td></tr><tr><td>1.1 Background</td><td>1</td></tr><tr><td>1.2 Literature review</td><td>3</td></tr><tr><td>1.3 Research objectives and challenges</td><td>4</td></tr><tr><td>1.4 Outline of the dissertation</td><td>5</td></tr><tr><td>1.5 Contributions</td><td>6</td></tr><tr><td>Chapter 2 Overview of energy management and smart grid</td><td>7</td></tr><tr><td>2.1 Introduction</td><td>7</td></tr><tr><td>2.2 Primer of energy management</td><td>9</td></tr><tr><td>2.3 Smart grid technology</td><td>9</td></tr><tr><td>2.4 AI methods for energy management</td><td>15</td></tr><tr><td>2.4.1 Artificial neural networks-based</td><td>17</td></tr><tr><td>2.4.2 Fuzzy logic-based</td><td>17</td></tr><tr><td>2.4.3 Ensemble methods-based</td><td>19</td></tr><tr><td>2.4.4 Deep learning-based</td><td>20</td></tr><tr><td>2.4.5 Hybrid models-based</td><td>22</td></tr><tr><td>2.4.6 Evaluation procedure</td><td>26</td></tr><tr><td>2.5 Summary</td><td>29</td></tr><tr><td>Chapter 3 PV power forecasting</td><td>31</td></tr><tr><td>3.1 Introduction</td><td>31</td></tr><tr><td>3.2 Case study 1</td><td>32</td></tr><tr><td>3.2.1 Proposed methodology</td><td>32</td></tr><tr><td>3.2.2 Simulation results</td><td>41</td></tr><tr><td>3.3 Case study 2</td><td>64</td></tr><tr><td>3.3.1 Proposed methodology</td><td>64</td></tr><tr><td>3.3.2 Simulation results</td><td>67</td></tr><tr><td>3.4 Summary</td><td>71</td></tr></table>
<table>
  <tbody>
    <tr>
      <td><strong>Chapter 4 Control management and optimization</strong></td>
      <td><strong>72</strong></td>
    </tr>
    <tr>
      <td>4.1 Introduction</td>
      <td>72</td>
    </tr>
    <tr>
      <td>4.2 Load demand forecasting</td>
      <td>74</td>
    </tr>
    <tr>
      <td>4.2.1 Case study 1</td>
      <td>74</td>
    </tr>
    <tr>
      <td>4.2.2 Case study 2</td>
      <td>91</td>
    </tr>
    <tr>
      <td>4.3 Power grid stability assessment</td>
      <td>112</td>
    </tr>
    <tr>
      <td>4.3.1 Case study 1</td>
      <td>112</td>
    </tr>
    <tr>
      <td>4.3.2 Case study 2</td>
      <td>119</td>
    </tr>
    <tr>
      <td>4.4 Summary</td>
      <td>126</td>
    </tr>
    <tr>
      <td><strong>Chapter 5 Conclusion</strong></td>
      <td><strong>128</strong></td>
    </tr>
    <tr>
      <td>5.1 Summary</td>
      <td>128</td>
    </tr>
    <tr>
      <td>5.2 Future work</td>
      <td>129</td>
    </tr>
    <tr>
      <td><strong>References</strong></td>
      <td><strong>130</strong></td>
    </tr>
  </tbody>
</table>
List of Tables

<table>
  <tbody>
    <tr>
      <td>2.1</td>
      <td>Comparative study of the proposed techniques [1].</td>
      <td>26</td>
    </tr>
    <tr>
      <td>2.2</td>
      <td>Some exemplary score errors.</td>
      <td>28</td>
    </tr>
    <tr>
      <td>2.3</td>
      <td>Classes of score metrics [1].</td>
      <td>29</td>
    </tr>
    <tr>
      <td>3.1</td>
      <td>The related characteristics of the PV plant [2].</td>
      <td>43</td>
    </tr>
    <tr>
      <td>3.2</td>
      <td>Data types [2].</td>
      <td>43</td>
    </tr>
    <tr>
      <td>3.3</td>
      <td>Feature importance for system inputs on D<sub>1</sub> and D<sub>2</sub> [2].</td>
      <td>45</td>
    </tr>
    <tr>
      <td>3.4</td>
      <td>Development settings [2].</td>
      <td>46</td>
    </tr>
    <tr>
      <td>3.5</td>
      <td>Search space and TSA results of the proposed SG-CBiLSTM [2].</td>
      <td>47</td>
    </tr>
    <tr>
      <td>3.6</td>
      <td>Score performance comparison on D<sub>1</sub> [2].</td>
      <td>53</td>
    </tr>
    <tr>
      <td>3.7</td>
      <td>Score performance comparison on D<sub>2</sub> [2].</td>
      <td>54</td>
    </tr>
    <tr>
      <td>3.8</td>
      <td>Computational complexity of each model on the testing set [2].</td>
      <td>55</td>
    </tr>
    <tr>
      <td>3.9</td>
      <td>Comparison of 1-hour ahead PVPF results [2].</td>
      <td>56</td>
    </tr>
    <tr>
      <td>3.10</td>
      <td>Score performance comparison using D<sub>1</sub> [2].</td>
      <td>58</td>
    </tr>
    <tr>
      <td>3.11</td>
      <td>Score performance comparison using D<sub>2</sub> [2].</td>
      <td>58</td>
    </tr>
    <tr>
      <td>3.12</td>
      <td>Comparison of 3-hour ahead PVPF results [2].</td>
      <td>60</td>
    </tr>
    <tr>
      <td>3.13</td>
      <td>Comparison of 6-hour ahead PVPF results [2].</td>
      <td>60</td>
    </tr>
    <tr>
      <td>3.14</td>
      <td>Performance evaluation of probabilistic forecasting for case I-II [2].</td>
      <td>63</td>
    </tr>
    <tr>
      <td>3.15</td>
      <td>Results of multi-step probabilistic forecasting for 1hour(1-h), 2 hours (2-h) and 3 hours (3-h) forecasting horizons [2].</td>
      <td>64</td>
    </tr>
    <tr>
      <td>3.16</td>
      <td>Evaluation scores [3].</td>
      <td>68</td>
    </tr>
    <tr>
      <td>3.17</td>
      <td>Evaluation scores [3].</td>
      <td>68</td>
    </tr>
    <tr>
      <td>3.18</td>
      <td>Evaluation scores [3].</td>
      <td>69</td>
    </tr>
    <tr>
      <td>4.1</td>
      <td>Generated feature vectors [4].</td>
      <td>78</td>
    </tr>
    <tr>
      <td>4.2</td>
      <td>hyperparameter settings of the proposed model [4].</td>
      <td>81</td>
    </tr>
    <tr>
      <td>4.3</td>
      <td>Layer description of CBiLSTM [4].</td>
      <td>82</td>
    </tr>
    <tr>
      <td>4.4</td>
      <td>Performance analysis on the impact of different SG parameters on the forecasting accuracy [4].</td>
      <td>83</td>
    </tr>
    <tr>
      <td>4.5</td>
      <td>Forecasting accuracy of different models [4].</td>
      <td>85</td>
    </tr>
    <tr>
      <td>4.6</td>
      <td>Forecasting results of the proposed model [4].</td>
      <td>86</td>
    </tr>
    <tr>
      <td>4.7</td>
      <td>Forecasting results of different models on ISO-New England data set [4].</td>
      <td>90</td>
    </tr>
    <tr>
      <td>4.8</td>
      <td>Results of Multistep forecasting on ISO-NE dataset [4].</td>
      <td>91</td>
    </tr>
    <tr>
      <td>4.9</td>
      <td>hyperparameter settings of experimental models [5].</td>
      <td>98</td>
    </tr>
    <tr>
      <td>4.10</td>
      <td>Cross-validation results for case study 1 [5].</td>
      <td>100</td>
    </tr>
    <tr>
      <td>4.11</td>
      <td>Evaluation errors for case study 1 [5].</td>
      <td>101</td>
    </tr>
    <tr>
      <td>4.12</td>
      <td>Evaluation measures for the stacked model and other benchmarks for one day ahead [5].</td>
      <td>102</td>
    </tr>
    <tr>
      <td>4.13</td>
      <td>hyperparameters settings for reference models [5].</td>
      <td>105</td>
    </tr>
    <tr>
      <td>4.14</td>
      <td>Forecasts and actual load demand [5].</td>
      <td>107</td>
    </tr>
    <tr>
      <td>4.15</td>
      <td>Influence of the prediction range of the proposed model and benchmarks [5].</td>
      <td>108</td>
    </tr>
    <tr>
      <td>4.16</td>
      <td>Performance evaluation results for reference models with 10-CV [5].</td>
      <td>109</td>
    </tr>
    <tr>
      <td>4.17</td>
      <td>Performance evaluation results for THE proposed framework with 10-CV [5].</td>
      <td>110</td>
    </tr>
  </tbody>
</table>
<table>
  <tbody>
    <tr>
      <td>4.18</td>
      <td>hyperparameters settings for the simulated models [6].</td>
      <td>115</td>
    </tr>
    <tr>
      <td>4.19</td>
      <td>Comparative study of various prediction techniques [6].</td>
      <td>116</td>
    </tr>
    <tr>
      <td>4.20</td>
      <td>Performance evaluation of interval forecasting [6].</td>
      <td>118</td>
    </tr>
    <tr>
      <td>4.21</td>
      <td>Accuracy measures for SEC for 10-CV. FN, M, and SD are the fold number, mean, and standard derivation respectively [7].</td>
      <td>121</td>
    </tr>
    <tr>
      <td>4.22</td>
      <td>Confusion Matrix for the proposed model and benchmarks [7].</td>
      <td>122</td>
    </tr>
    <tr>
      <td>4.23</td>
      <td>Comparative study of various prediction techniques [7].</td>
      <td>123</td>
    </tr>
    <tr>
      <td>4.24</td>
      <td>Classification report for the proposed model and benchmarks [7].</td>
      <td>125</td>
    </tr>
    <tr>
      <td>4.25</td>
      <td>Computational time (second) for PGS prediction [7].</td>
      <td>126</td>
    </tr>
  </tbody>
</table>
List of Figures

<table>
  <tbody>
    <tr>
      <td>1.1 Artificial Intelligence timeline.</td>
      <td>4</td>
    </tr>
    <tr>
      <td>1.2 Proposed methodology of this dissertation.</td>
      <td>5</td>
    </tr>
    <tr>
      <td>1.3 Organization of the thesis presentation.</td>
      <td>6</td>
    </tr>
    <tr>
      <td>2.1 Existing power grid and futuristic smart grid paradigms.</td>
      <td>11</td>
    </tr>
    <tr>
      <td>2.2 Characteristics of SG paradigm.</td>
      <td>12</td>
    </tr>
    <tr>
      <td>2.3 AI applications for power systems.</td>
      <td>13</td>
    </tr>
    <tr>
      <td>2.4 Machine learning classes and usage.</td>
      <td>14</td>
    </tr>
    <tr>
      <td>2.5 Machine learning models with their open-source libraries [1].</td>
      <td>16</td>
    </tr>
    <tr>
      <td>2.6 Pictorial representation of the AI Models applied to SG in 2019-2020 [1].</td>
      <td>16</td>
    </tr>
    <tr>
      <td>2.7 Fuzzy inference system structure [1].</td>
      <td>18</td>
    </tr>
    <tr>
      <td>2.8 Representation of hybrid two-method models comprising GA, ensembles, ANN, FL, SVM, DL, and ARIMA [1].</td>
      <td>22</td>
    </tr>
    <tr>
      <td>2.9 Taxonomy of AI techniques applied to smart grids [1].</td>
      <td>24</td>
    </tr>
    <tr>
      <td>2.10 Commonly used score metrics for regression assessment [1].</td>
      <td>27</td>
    </tr>
    <tr>
      <td>3.1 PV power forecasting methodology.</td>
      <td>32</td>
    </tr>
    <tr>
      <td>3.2 Structure of NARX network [2].</td>
      <td>33</td>
    </tr>
    <tr>
      <td>3.3 LSTM memory block, including the input, output, and forget gates [2].</td>
      <td>35</td>
    </tr>
    <tr>
      <td>3.4 Framework of the proposed TSA algorithm [2].</td>
      <td>36</td>
    </tr>
    <tr>
      <td>3.5 Main framework of the proposed hybrid NARX-LSTM [2].</td>
      <td>40</td>
    </tr>
    <tr>
      <td>3.6 General flow chart of developed NARX-LSTM algorithm [2].</td>
      <td>41</td>
    </tr>
    <tr>
      <td>3.7 Optimal configuration of (a)NARXNN model (b) LSTM model [2].</td>
      <td>48</td>
    </tr>
    <tr>
      <td>3.8 Different possible scenario for D<sub>1</sub> dataset [2].</td>
      <td>48</td>
    </tr>
    <tr>
      <td>3.9 Different possible scenario for D<sub>2</sub> dataset [2].</td>
      <td>49</td>
    </tr>
    <tr>
      <td>3.10 Scatter plots ((a), (b)) and error distributions ((c), (d)) of actual and predicted PV output for D<sub>1</sub> and D<sub>2</sub> [2].</td>
      <td>50</td>
    </tr>
    <tr>
      <td>3.11 PV power forecast results in 5 min ahead daily forecasting using D<sub>1</sub> [2].</td>
      <td>51</td>
    </tr>
    <tr>
      <td>3.12 PV power forecast result in 1-hour ahead forecasting horizons using D<sub>2</sub> [2].</td>
      <td>52</td>
    </tr>
    <tr>
      <td>3.13 Variation of PV parameters [2].</td>
      <td>57</td>
    </tr>
    <tr>
      <td>3.14 Multistep forecasting tests of NARX-LSTM model using D<sub>1</sub> and D<sub>2</sub> datasets for different forecasting horizons [2].</td>
      <td>57</td>
    </tr>
    <tr>
      <td>3.15 Box-and-Whisker plots of nRMSE and nMAE errors for different forecasting horizons with D<sub>1</sub> (case I) and D<sub>2</sub> (case II) of NARX-LSTM model [2].</td>
      <td>59</td>
    </tr>
    <tr>
      <td>3.16 The PI forecasting curves of the proposed NARX-LSTM model with D<sub>1</sub> and D<sub>2</sub> [2].</td>
      <td>62</td>
    </tr>
    <tr>
      <td>3.17 90% Prediction interval results on D<sub>2</sub> and D<sub>2</sub> [2].</td>
      <td>62</td>
    </tr>
    <tr>
      <td>3.18 Proposed model architecture [3].</td>
      <td>67</td>
    </tr>
    <tr>
      <td>3.19 Assessment of variable contributions using BRR [3].</td>
      <td>68</td>
    </tr>
    <tr>
      <td>3.20 Real and predicted PV power output in steady weather conditions [3].</td>
      <td>69</td>
    </tr>
    <tr>
      <td>3.21 10-fold CV for the proposed model [3].</td>
      <td>70</td>
    </tr>
    <tr>
      <td>3.22 Real and predicted PV power output in unsteady weather conditions [3].</td>
      <td>70</td>
    </tr>
  </tbody>
</table>
<table>
  <tbody>
    <tr>
      <td>4.1</td>
      <td>AI applications for power grid stability.</td>
      <td>73</td>
    </tr>
    <tr>
      <td>4.2</td>
      <td>Load demand forecasting procedure.</td>
      <td>74</td>
    </tr>
    <tr>
      <td>4.3</td>
      <td>The adopted methodology for the proposed SG-CBiLSTM model [4].</td>
      <td>75</td>
    </tr>
    <tr>
      <td>4.4</td>
      <td>High-level flowchart of SG-CBiLSTM algorithm [4].</td>
      <td>75</td>
    </tr>
    <tr>
      <td>4.5</td>
      <td>10-fold cross-validation methodology [4].</td>
      <td>76</td>
    </tr>
    <tr>
      <td>4.6</td>
      <td>Load and temperature variation between 2003-2011 (103753 instances) for the ISO-NE data set [4].</td>
      <td>77</td>
    </tr>
    <tr>
      <td>4.7</td>
      <td>Comparative results of SG-CBiLSTM model with different polynomial functions [4].</td>
      <td>79</td>
    </tr>
    <tr>
      <td>4.8</td>
      <td>Variable importance of load patterns with a threshold value (red line) equal to VIP=0.5 [4].</td>
      <td>80</td>
    </tr>
    <tr>
      <td>4.9</td>
      <td>Performance comparison of SG-CBiLSTM model with a variety of polynomial filters p={3,5,7,10} [4].</td>
      <td>83</td>
    </tr>
    <tr>
      <td>4.10</td>
      <td>Loss trend of the SG-CBiLSTM model [4].</td>
      <td>84</td>
    </tr>
    <tr>
      <td>4.11</td>
      <td>Box plot of ten-fold CV of four models in terms of R<sup>2</sup>(%).</td>
      <td>87</td>
    </tr>
    <tr>
      <td>4.12</td>
      <td>Hourly predictions of the forecasting system [4].</td>
      <td>87</td>
    </tr>
    <tr>
      <td>4.13</td>
      <td>Simulation results of SG-CBiLSTM and reference models over multiple steps ahead [4].</td>
      <td>89</td>
    </tr>
    <tr>
      <td>4.14</td>
      <td>Multi-step STLF error variation with the Error (%)= 100*(ŷ<sub>norm</sub> - y<sub>norm</sub>) [4].</td>
      <td>91</td>
    </tr>
    <tr>
      <td>4.15</td>
      <td>Graphical representation of the stacking generalization approach [5].</td>
      <td>92</td>
    </tr>
    <tr>
      <td>4.16</td>
      <td>Schematic representation of the level-0 of stacked architecture framework built from base learners: XGB and LGBM [5].</td>
      <td>93</td>
    </tr>
    <tr>
      <td>4.17</td>
      <td>Main framework of the proposed stacking model [5].</td>
      <td>94</td>
    </tr>
    <tr>
      <td>4.18</td>
      <td>Flowchart of the proposed Stacked XGB-LGBM-MLP method [5].</td>
      <td>95</td>
    </tr>
    <tr>
      <td>4.19</td>
      <td>The time series data between 2009-2011 (17519 samples) [5].</td>
      <td>96</td>
    </tr>
    <tr>
      <td>4.20</td>
      <td>Actual and predicted demand for 31/12/09 [5].</td>
      <td>102</td>
    </tr>
    <tr>
      <td>4.21</td>
      <td>Scatter graph of the joint distribution of the real load demand and its forecast.</td>
      <td>103</td>
    </tr>
    <tr>
      <td>4.22</td>
      <td>Actual and predicted load demand with XGB, LGBM, MLP, and stacked model [5].</td>
      <td>103</td>
    </tr>
    <tr>
      <td>4.23</td>
      <td>Comparison of MAPE performance for ML models [5].</td>
      <td>106</td>
    </tr>
    <tr>
      <td>4.24</td>
      <td>RMSE and MAPE values for the reference models and the proposed technique [5].</td>
      <td>106</td>
    </tr>
    <tr>
      <td>4.25</td>
      <td>Actual and predicted demand for 31/12/09 and 01/01/10 [5].</td>
      <td>108</td>
    </tr>
    <tr>
      <td>4.26</td>
      <td>10-fold CV graph for Stacked XGB-LGBM-MLP [5].</td>
      <td>110</td>
    </tr>
    <tr>
      <td>4.27</td>
      <td>Box-and-Whisker plot of RMSE errors for STLF models [5].</td>
      <td>111</td>
    </tr>
    <tr>
      <td>4.28</td>
      <td>Methodology of grid stability prediction.</td>
      <td>112</td>
    </tr>
    <tr>
      <td>4.29</td>
      <td>Main framework of the proposed SA-BiGRU model [6].</td>
      <td>113</td>
    </tr>
    <tr>
      <td>4.30</td>
      <td>Flowchart of the proposed forecasting process [6].</td>
      <td>114</td>
    </tr>
    <tr>
      <td>4.31</td>
      <td>(a)Scatter plot and (b)error distribution of BiGRU model [6].</td>
      <td>115</td>
    </tr>
    <tr>
      <td>4.32</td>
      <td>Training and testing loss variation for epochs [6].</td>
      <td>116</td>
    </tr>
    <tr>
      <td>4.33</td>
      <td>Performance analysis of prediction models [6].</td>
      <td>117</td>
    </tr>
    <tr>
      <td>4.34</td>
      <td>Error variations [6].</td>
      <td>117</td>
    </tr>
    <tr>
      <td>4.35</td>
      <td>The constructed prediction intervals for grid stability index [6].</td>
      <td>118</td>
    </tr>
    <tr>
      <td>4.36</td>
      <td>Flowchart of the proposed ensemble method [7].</td>
      <td>119</td>
    </tr>
    <tr>
      <td>4.37</td>
      <td>Comparative results [7].</td>
      <td>123</td>
    </tr>
    <tr>
      <td>4.38</td>
      <td>Radar plot comparison for the proposed model with other benchmarks based on F1 score [7].</td>
      <td>124</td>
    </tr>
    <tr>
      <td>4.39</td>
      <td>Score performance comparison [7].</td>
      <td>126</td>
    </tr>
  </tbody>
</table>
List of Abbreviations

<table><tr><td>ABC</td><td>Ant bee colony</td></tr><tr><td>ACO</td><td>Ant Colony Optimization</td></tr><tr><td>Ada</td><td>Adaptive boosting</td></tr><tr><td>Adam</td><td>Adaptive Moment Estimation optimizer</td></tr><tr><td>Ant colony</td><td>Ant colony</td></tr><tr><td>AODE</td><td>Aggregating One-Dependence Estimators classifier</td></tr><tr><td>ARIMA</td><td>AutoRegressive Integrated Moving Average</td></tr><tr><td>ARIMA</td><td>Autoregressive integrated moving average</td></tr><tr><td>ARIMAX</td><td>AutoRegressive Integrated Moving Average process with eXogenous inputs</td></tr><tr><td>ARMA</td><td>AutoRegressive Moving Average</td></tr><tr><td>BAC</td><td>Bayesian Actor-Critic</td></tr><tr><td>BBN</td><td>Bayesian Belief Network</td></tr><tr><td>Bio</td><td>Biological swarm chasing algorithm</td></tr><tr><td>CNN-WT</td><td>Convolutional neural network-Wavelet transform</td></tr><tr><td>CRFS</td><td>Conditional random fields</td></tr><tr><td>CRO-SL</td><td>Coral Reefs Optimization algorithm with Substrate Layers</td></tr><tr><td>CV</td><td>Cross-Validation</td></tr><tr><td>DBN</td><td>Deep Belief Networks</td></tr><tr><td>DCNN</td><td>Deep convolutional neural network</td></tr><tr><td>DFO</td><td>Derivative-Free Optimization</td></tr><tr><td>DQL</td><td>Deep Q-Learning</td></tr><tr><td>DRN</td><td>Deep Residual Networks</td></tr><tr><td>DT</td><td>Decision Tree</td></tr><tr><td>DWT</td><td>Discrete Wavelet Transform</td></tr><tr><td>DWWC</td><td>Dynamically Weighted Wavelet Coefficients</td></tr></table>
<table>
   <tr>
    <td>ESN</td>
    <td>Echo State Networks</td>
   </tr>
   <tr>
    <td>FA</td>
    <td>Firefly Algorithm</td>
   </tr>
   <tr>
    <td>GANs</td>
    <td>Generative Adversarial Networks</td>
   </tr>
   <tr>
    <td>IoT</td>
    <td>Internet Of Things</td>
   </tr>
   <tr>
    <td>GASVM</td>
    <td>Genetic Functionality Support Vector Machine</td>
   </tr>
   <tr>
    <td>GBTD</td>
    <td>Gradient Boosting Theft Detector</td>
   </tr>
   <tr>
    <td>Glowworm</td>
    <td>Glowworm swarm based optimization</td>
   </tr>
   <tr>
    <td>GP</td>
    <td>Gaussian Process</td>
   </tr>
   <tr>
    <td>GRU</td>
    <td>Gated Recurrent Unit</td>
   </tr>
   <tr>
    <td>GS</td>
    <td>Grid Search</td>
   </tr>
   <tr>
    <td>GWO</td>
    <td>Grey Wolf Optimization</td>
   </tr>
   <tr>
    <td>HMM</td>
    <td>Hidden Markov models</td>
   </tr>
   <tr>
    <td>HNN</td>
    <td>Hybrid Neural Network</td>
   </tr>
   <tr>
    <td>IPSO</td>
    <td>Improved Particle Swarm Optimization</td>
   </tr>
   <tr>
    <td>KNN</td>
    <td>K-nearest neighbors</td>
   </tr>
   <tr>
    <td>KNN</td>
    <td>K-nearest Neighbors</td>
   </tr>
   <tr>
    <td>LES</td>
    <td>Linear exponential smoothing</td>
   </tr>
   <tr>
    <td>LES</td>
    <td>Least-Squares Estimation</td>
   </tr>
   <tr>
    <td>LF</td>
    <td>Load Forecasting</td>
   </tr>
   <tr>
    <td>MNB</td>
    <td>Multinomial Naïve Bayes</td>
   </tr>
   <tr>
    <td>MSLE</td>
    <td>Friction Index</td>
   </tr>
   <tr>
    <td>MSS</td>
    <td>Model Structure Selection</td>
   </tr>
   <tr>
    <td>NARX</td>
    <td>Nonlinear AutoRegressive eXogenous</td>
   </tr>
   <tr>
    <td>NB</td>
    <td>Naïve bayes</td>
   </tr>
   <tr>
    <td>Neuro-Fuzzy</td>
    <td>Artificial neural networks-Fuzzy logic</td>
   </tr>
   <tr>
    <td>NN</td>
    <td>Neural Networks</td>
   </tr>
   <tr>
    <td>OOF</td>
    <td>Out-Of-Fold method</td>
   </tr>
   <tr>
    <td>PCA</td>
    <td>Principal Component Analysis</td>
   </tr>
   <tr>
    <td>PSO</td>
    <td>Particle Swarm Optimization</td>
   </tr>
   <tr>
    <td>PVPF</td>
    <td>Photovoltaic power forecasting</td>
   </tr>
   <tr>
    <td>QR</td>
    <td>Quantile Regression</td>
   </tr>
   <tr>
    <td>RF</td>
    <td>Random Forest</td>
   </tr>
   <tr>
    <td>RFL</td>
    <td>Reinforcement learning</td>
   </tr>
  </table>
<table><tr><td>RMSE</td><td>Rooted Mean Squared Error</td></tr><tr><td>SELM</td><td>Stacked extreme learning machine</td></tr><tr><td>STLF</td><td>Short-Term Load forecasting</td></tr><tr><td>WT</td><td>Wavelet Transform</td></tr><tr><td>XGB</td><td>Extreme Gradient Boosting</td></tr></table>
# Chapter 1

# Introduction

## 1.1 Background

Energy is a core element for societal prosperity, technological revolution, and economic growth in the world [8]. Over the years, the exponential increase in global energy demand has become the leading cause of the rapid depletion of fossil fuels and increased Greenhouse Gas (GHG) emissions of conventional generators [9]. To effectively satisfy the meteoric growth in energy consumption, the world has taken serious initiatives to deploy RES on a larger scale. For example, the European Union targets achieving 100% in 2050 [10]. The harnessing of RERs, e.g., Solar Energy (SE), wind energy, biomass energy, geothermal energy, etc., provides an excellent opportunity to mitigate GHG emissions and reduce global warming for sustainable development. RESs are currently being implemented worldwide to meet the requirements of increased energy demand, mitigate environmental pollutants, and achieve socio-economic benefits for sustainable development.

SE hold out the greatest promise for mankind among all RES, being free, clean, and abundantly available [11]. For these reasons, it keeps increasing its share in the energy mix in the face of diminishing conventional fossil fuel energy sources and rising environmental protection concerns [11]. For electricity production, SE can be mainly harnessed using two different technologies: solar PV and solar thermal systems. Concentrating Solar-thermal Power (CSP) technology concentrates the sun's light energy to power a turbine by converting the focused sunlight into high-temperature heat, which produces steam [12]. Mirrors use steam to drive steam turbines for large-scale electricity generation. Thus, countries such as the United States have been implementing CSP plants for energy generation [13]. Nevertheless, CSP ultimately needs large installations to be efficient [14].

In contrast, PV stations use incident light to excite free electrons in embedded semiconductors, called the photovoltaic effect. [15]. To increase the energy generation of PV cells, they are connected in chains to build larger units named modules or panels. These panels are operational at various scales and are often used on rooftops or open spaces, integrated into buildings or vehicle designs, or arranged in massive arrays in solar power plants. Over the past decades, solar panels have been embraced use globally, and their demand has
increased as a direct consequence of their mass public appeal and reduction in tariff charge systems. However, the major hindrance of the deepening penetration of PV stations is the insufficient energy generation, and intermittent nature of solar energy [16].

To provide a secure and stable energy supply, the public grid needs an advanced coordination framework. Thus, nations worldwide heavily invest in the integration of power management systems to efficiently accomplish the possible looming challenges in terms of instability and volatility of RES. According to the US energy department, the annual investment in SGs technology will reach more than $13.8 billion in 2024 [17]. The motivation for adopting this next-generation power system comes essentially from the flourishing sector of RES. Meanwhile, the foremost challenge in a SG with distributed energy resources is managing the intermittent nature of RESs. SGs are digitizing the grid elements with Information and Communication Technologies (ICT) as a fragment of the IoT framework conformable to the energy sector.

The structure of the SGs denotes four prominent folds, specifically, load, traditional energy, public grid, and RES. All the mentioned parts are connected with AI, which denotes the heart of the SG paradigm. The operational process pass by the AI hub. AI contribute to the system by an autonomous and fast distribution. The communication of the system elements provides flexibility to the grid and ensures an optimal load supply balance [18]. The grid integrity via AI ensures the required protection from cyber threats. The data generated in the generation process provide a reliable source with Advanced Metering Infrastructure (AMI). The information received is used to tune and enhance AI techniques. The data management and control platforms are carried out through AI. Great importance has been addressed to AI techniques due to their crucial role in the overall energy sector. The information-sharing framework assigns core importance to AI since it presents critical success with edge computing support [19].

These radical transformations include distribution, transmission, and electricity generation [20]. The improved version of the traditional grid covers the implementation of potential computing, advanced sensors, and meters with self-monitoring of variant energy suppliers and loads interconnected with high-level automation. Regarding the International Energy Agency (IEA) report, SGs are a strategic element for sustainable energy goals [20]. The significant potential of SGs comes from implementing an embedded layer of ICT of bulk generation between the components of the system. This two-way communication paradigm uses smart meters and sensors to take decisional actions during the generation of transmission and distribution lines. The aim is to cover the weak points of energy heterogeneity and sensibility to environmental and operational conditions [21]. This coordination platform provides a high-performance power supply due to the contentious supervision of detailed parts in the system.

Recently, the SG framework has experienced a vivid revival and has become a sign of countries' de-
velopment since it provides a sustainable and stable electricity supply [19]. These latter include generating electricity from traditional and alternative energy sources, transmitting this energy from the centralized generators to the local grid, and distributing the flow of energy to homes and cities. For each of the mentioned elements, SGs control the operating process via the developed mechanism of supervision and decision-making. The need for full automation in the system operation requires integrating learning-based modeling frameworks for each stage. It can be cited fault detection, load and power forecasting, cybersecurity, big data analysis, etc. Each factor should clearly define the elements' priorities in terms of environmental challenges, customer and market needs, infrastructure challenges, and innovative technologies such as AMI devices.

For achieving these items, the system adopts some fundamental aspects. Those aspects consist of digitalization, flexibility, resiliency, and intelligence. Such a system's efficiency will improve the penetration of alternative energy sources since the stability of the outdated grid is threatened by the internment change of weather. With SGs, precautions are taken for any slight perturbation to select the subsequent suitable decision-making. However, the transition to future grids cannot be achieved without the high connectivity of advanced power electronics and devices through high-performance computing and monitoring techniques. With daily operations, a new massive amount of data is generated and stored in the system. An intelligent grid system's role is to manage this data and make it useful for monitoring and control. The static hierarchical grid needs require advanced methodologies for a safe and stable transition.

## 1.2 Literature review

In light of the bulk PV capacity expansion, the employment of PVPF for an efficient energy management system has been considered as the core element of the modern SG. AI-based PVPF emerged nowadays as a promising research direction. With the enormous integration of smart meters in electrical grids, machine learning methods take advantage of the large-scale and multi-source data representations to achieve a spectacular performance and high forecastability potential.

Unlike PV power generation, which is almost only determined by the weather conditions, the load is also dependant on other factors related to top particular microgrid patterns: building occupancy, seasonality, or consumers' behavior. As reported in the literature, electricity can not be stored in large amounts, which ultimately requires that PV generation and load demand must have a fair balance [22]. Thus, Load forecasting (LF) is undoubtedly the essence of strategic management operations in SG as it offers an optimization solution for operation costs, optimum load supply tradeoff, and power quality. LF is the analysis of historical data related to environmental conditions, charge graph behavior, and historical consumption.
LF output is an accurate prediction of microgrid consumption to determine preventive actions for stable operating conditions. Based on the time horizon, load demand forecasting can be classified into three ranges, short-term load forecasting (STLF) for hours to weeks, medium-term load forecasting for weeks to months, and long-term load forecasting starting from five years [23]. The estimated quantity of supply the load needs is ensured using LF techniques. An accurate LF prediction facilitates the coordination of SG and guarantees the budget-saving commitment and grid stability.

With the exponential growth of electricity operations and measurements in terms of size and complexity, generation schedules in the context of real-time operations are programmed for short-term, mid-term, and long-term power system operations. Therefore, it is mandatory to provide accurate predictions of PV generation output and load demand for ensuring uninterruptable power supply to the end-users. The rampant use of AI in power systems is an emerging trend to cover the knowledge gaps for better flexibility and reliability of the supply service [24]. Fig. 1.1 presents the timeline evolution of AI from its birth until the uptake of DL deployments.

<figure><img src="image_2.png" /><figcaption>Figure 1.1: Artificial Intelligence timeline.</figcaption></figure>

## 1.3 Research objectives and challenges

This thesis' contributions broadly consist of tailoring intelligent computational methods for PV energy production and managing the electric power in the grid from both demand and supply sides, as shown in Fig. 1.2. This is mainly done by implementing state-of-the-art machine learning techniques to cope with the
intermittent weather conditions and predict the PV power with high exactitude. The time series prediction and analysis tackle single-step forecasting, multistep forecasting, point forecasting, and interval forecasting.

<figure><img src="image_3.png" /><figcaption>Figure 1.2: Proposed methodology of this dissertation.</figcaption></figure>

Although multiple methods have been proposed to solve the energy management paradigm using AI, there is a lack of comprehensive review and comparative analysis of the proposed methods. This dissertation first provides a comprehensive review of existing methods for energy management tools for SG. Moreover, this dissertation proposes two innovative PVPF methods using the virtue of prepossessing units and ML/DL models. Next, two efficient methods for load forecasting were comprehensively presented to secure adequate management of the demand side from the intermittent supply of PV power. Finally, a grid stability prediction system is designed to guarantee the efficient management of both supply and load sides. Using machine learning and DL technique methods, multiple simulation results are generated using Python and Matlab softwares on real-world datasets.

## 1.4 Outline of the dissertation

The remainder of this dissertation is as follows: Chapter 2 provides a systematic literature review of energy management, smart grid, and the available AI methods for energy management systems. Chapter 3 proposes two efficient PVPF methods and studies their performance via multiple simulation scenarios. Chapter 4 sheds light on an essential side of control management and optimization strategies for efficient use of PV power production. In other words, the load demand prediction has been addressed by proposing innovative tools on real datasets with case studies. Meanwhile, the grid stability prediction and assessment have been tackled to enhance the proposed energy management paradigm by optimal balancing between PV generation and demand. The grid stability paradigm emerges nowadays as a central element of energy systems due to the high integration of PV systems, importing many uncertainties on the generated power flow. Section 5 concludes this thesis by giving the main findings and future work. This study's organization is illustrated in Fig. 1.3.
<figure><img src="image_4.png" /><figcaption>Figure 1.3: Organization of the thesis presentation.</figcaption></figure>

## 1.5 Contributions

The main contributions of this thesis can be summarized in terms of the following salient points:

* A bird's eye review of energy management, SG, and AI methods applied to the energy sector has been conducted.

* Efficient ML models have been proposed for PV power forecasting.

* Novel computational learning models have been introduced for load demand forecasting.

* To find optimal equilibrium between the supply and load, effective ML techniques have been proposed for grid stability prediction and assessment.

The main contents of this thesis have been published in [1–4, 6, 7, 19, 25]
# Chapter 2

## Overview of energy management and
smart grid

### 2.1 Introduction

Due to the continuous industrial revolution and world population growth, the electrical grid faces many operational challenges in terms of power scalability, reliability, and autonomous control [26]. For the time being, the power grid transformation for higher efficiency and flexibility enhancement reveals a strong consensus. Meanwhile, the SG paradigm enables the dominant grid integration of volatile and intermittent RES [17]. The SG concept revolutionizes the traditional utility grid with high reliability, eco-friendly impact, human safety, and infrastructure protection. This interactive power system paves the way for customers and suppliers to exchange their benefits from a bidirectional coordination platform for electricity and information. The SG framework provides a performance toolkit based on Information and Communication Technologies (ICT) [27, 28]. The SG paradigm supervises and promotes grid operations at a high level of expertise [29].

In the present-day scenario, PV energy becomes the largest RES of carbon-free energy owing to fossil fuel resource depletion in the near future, and climate change concerns [30]. To incorporate this immense PV potential, the SG requires experiencing a radical transformation to provide high-quality and highly reliable energy delivery to end-users. However, century-old control techniques and legacy mechanical meters can not meet the technical requirements of this renewable transition [17]. Moreover, the ICT uptake has entailed significant achievements in the evolution of smart meters, sensors, and data center development. This leads to more data that can be generated. Fortunately, AI techniques efficiently make use of the provided data to reap multiple socio-economic and financial benefits. AI methods drastically take the lead to manage all parts of the SG with high-efficiency [17].

AI systems have contributed to the realization of Sustainable Development Goals (SDGs), including the vital integration of RES. Furthermore, the decarbonization of fossil fuel systems via the deployment of Electric Vehicles (EVs) using Vehicle to Grid (V2G) and Grid to Vehicle (G2V) concepts requires smart infrastructure. This infrastructure supports a real-time demand response, charge scheduling, fair energy provision, congestion management, and market bidding optimization in which AI methodologies are mandatory
for their implementation [31]. Therefore, power systems-based AI gained tremendous interest from engi-
neers and scientists to use their vast potential for the SG concept. The SG enormously requires intelligent
management to allow diverse RES for bulk penetration to grid-connected systems and ensure well-organized
scheduling generation capacity and effective management control. AI-based technologies work under the
SG umbrella to provide the correct decision-making and guarantee self-controlled, pervasive networks with
broadband ICT. The fusion of AI and SG presents a powerful tool for handling sophisticated problems. Com-
puter reasoning competently carries out complex tasks with high expertise when automation is inefficient
in interpreting nonlinear and narrow behavior. Cognitive scientists define AI as the fusion of science and
engineering to benefit intelligent computational programming [17]. The AI applications include learning,
reasoning, and taking actions to face complicated problems such as electric operations.

Nowadays, Computational Intelligence (CI) illustrates the core factor for the digital automation of SG
systems taking advantage of the existing infrastructure in the level of hardware, software, and data avail-
ability [32]. In energy systems, AI applications include prediction, system modeling and control, and energy
management. Predictive models investigate the power system behavior to generate forecasts about the sys-
tem variability for a specific time horizon. Predictive Modelling (PM) techniques are applied to demand,
renewable power prediction, fault detection prediction, etc. The ML concept involves Supervised Learning
(SL) models such as Neural Networks, and Genetic Algorithms [33]. In contrast, system modeling consists
of defining a symbolic function for a transformation process. For example, ES efficiently interferes with the
model simulation using knowledge structure, and logic control [33]. Furthermore, system control and energy
management employ robust AI algorithms to secure normal operations of power systems.

Doubtless, SG and AI are aroused substantial attention at a fast pace in the last decade. Furthermore,
technology evolution creates a significant emergence for SG. For example, 5G network technology has become
available on many continents around the globe. Many countries take advantage of this technology to increase
the size of the data collected and use it for behavioral understanding and predictive applications. On
another side, the computational work's potential is enormous, particularly the recent waves of quantum
computing that deeply integrate the Internet Of Things (IoT) concept in electric systems that allow AI
to be upgraded to AI2.0. Therefore, the improvement of AI techniques is accelerated to gratify the high-
performance requirements and promote the usage of smart meters and controllers to improve and monitor
the power quality in the SG.

The chapter is organized as follows: Section 2.2 depicts the pivotal role of energy management for
the electrical system and utility grids. In section 2.3, the SG paradigm is comprehensively presented to
give insightful breakthroughs of the ample opportunities it can give to the actual century-old utility grids.
In section 2.4, the standard AI classes for energy systems are reported, discussing their strengths and weaknesses. Finally, the chapter summary is provided in Section 3.4.

## 2.2 Primer of energy management

The scarcity of conventional energy resources in the near future and their increasing threats to the environment extremely require the transition towards RES [34]. Several milestones have been reached that are indicative of a vital need for RES in the modern economy due to industrial expansion and technological development. Thus, nations around the globe are promoting the use of RES deployment [35].

The ongoing increase of renewable plants reveals technical issues on their deployment in the current power grid for their high sensitivity to weather parameters such as temperature, wind speed, and irradiation. Traditionally, the utility grid concept is not perfectly adapted to emerge such sources in unidirectional energy flow due to partial observability and uncertainty issues. Furthermore, the high disruption of RES and their sensibility to weather conditions made controlling the mixed flow of energy a problematic task. The outdated grid faces a major collapse due to RES reliance. The bidirectional energy flow requires advanced technology to meet renewable incorporation requirements.

Similarly, with the global modernization and development of nations worldwide, domestic electricity consumption has dramatically increased from 14 TWh in one year to reach 997 TWh in 2018 [36]. Yearly, the load demand is soaring by 3.5%, with expectations to reach a 50% increase in 2050 [36]. The residual, industrial, and transportation sectors are the most dominant sectors for the current heavy consumption due to a surge in population and electrified technologies. Nonetheless, electricity production primarily uses gas turbines and vapor turbines as sources of energy. Following the ongoing increase in electricity demand, the extinction of these sources is significantly accelerated. The traditional utility grid has four phases, namely, generation, distribution, transmission, and supply. The transmission lines have a unidirectional energy flow where the electricity is delivered to customers via transportation lines. The high voltage in this phase over long distances is stepped down via transformers in the distribution lines. The distribution phase supplies the energy provided to customers in the final step. The four phases are linked sequentially to build an archaic system.

## 2.3 Smart grid technology

Although the SG paradigm presents a reliable and secure electricity supply, the complexity of SG requires advanced automation tools to control power systems. The high integration of RES into the electrical grid
leads to intermittent power generation. This inherent intermittency requires AI techniques to cope with the stochastic meteorological variations and efficiently deal with the unprecedented amount of information from ubiquitous sensors and smart devices [37]. Prediction techniques present a suitable alternative for the PV owners and the grid operators leading to optimum generation-distribution-transmission power flow.

Taking as an example demand forecasting, stakeholders must provide an effective solution to carry out the load variations at different levels and meet the market demands [38]. However, generating accurate load forecasting is a challenging task due to the existence of several uncertain factors. These factors include social activities, economic indicators, and seasonal effects. The accuracy of load forecasting from AI techniques has a strong effect on the operation cost and the control strategies for power systems [39]. Furthermore, the AI technique ensures the flexibility of electricity prices in deregulated markets to optimize the real-time pricing and billing in usage time. The structure of the SGs shown in Fig.2.1, denotes four prominent folds, specifically, load, traditional energy, public grid, and RES. All the mentioned parts are connected with AI, which denotes the heart of the SGs paradigm.
<figure><img src="image_5.png" /><figcaption>Figure 2.1: Existing power grid and futuristic smart grid paradigms.</figcaption></figure>

Researchers have long sought to investigate the different elements that increase the profitability of grid utility. AI methods provide the essential techniques to identify the power consumption behavior by users [40]. Using this identification ability, the consumers could ensure the reasonable pricing of the energy system and optimize their profitability from the electrical grid. AI tools make use of the power consumption analysis to prevent consumers from abnormal behavior detection. One of the fundamental goals for a sustainable SG system is to smooth the integration of RES in the electrical grid. Several challenges should be taken into consideration to prevent grid failure. Fig. 2.2 presents the main features of the SG paradigm. These features essentially lie in intelligence, market-enabling, efficiency, flexibility, sustainability, and reliability.
<figure><img src="image_6.png" /><figcaption>Figure 2.2: Characteristics of SG paradigm.</figcaption></figure>

AI paradigm could provide fault tolerance and self-healing system to control electrical systems. The safety of equipment against every environmental threat could be conducted using AI. Fault diagnosis and AI techniques could enlarge the life cycle of the equipment by the early treatment of the operating system before the complete failure. Furthermore, the power network security protection is assured by AI techniques to prevent the system from any potential threats. Outages caused by the electric Distribution and Transmission (DT) failures frequently occur these last years [41]. Blackouts can occur due to cybersecurity attacks and device failures from environmental conditions and sudden accidents where a branch of a tree falling in the ground causes a blackout of all the neighboring houses around [42]. In an uncertain and complex environment, the SG system continuously deals with deep information flow leads to possible attacks on the physical power grid. AI ensures quick detection of malware and intrusion. Furthermore, the AI methods draw the best strategies for cybersecurity protection to protect energy systems. Some of the AI methods and their corresponding power grid applications are illustrated in Fig. 2.3.
<figure><img src="image_7.png" /><figcaption>Figure 2.3: AI applications for power systems.</figcaption></figure>

AI deployment contributes to the energy systems through budget optimization, self- automation, and efficient management in the SG era. More specifically, the most noticeable impact of AI deployment in power systems lies in greater improvement in terms of prediction accuracy, powerful security, and high performance. AI models include different aspects based on the type of information used for data representation mapping. These aspects are applied to different classes according to the data provided and the problem complexity. ML classes include Supervised Learning (SL), Semi-Supervised Learning (SSL), Unsupervised Learning (UL), and Reinforcement Learning (RL), as illustrated in Fig. 2.4.
<figure><img src="image_8.png" /><figcaption>Figure 2.4: Machine learning classes and usage.</figcaption></figure>

According to Fig. 2.4, the application of ML for SG includes classification, regression, clustering, dimensionality reduction, and control applications. The classification techniques are regarded as identifying different correlations for feature vectors to distinct output classes. In comparison, the regression techniques determine the exact value of labeled data from the data representation [43]. The regression accuracy is measured through error metrics [43]. Dimensionality reduction is employed to optimize feature numbers for the multi-labeled system. This optimization is conducted to identify the most suitable data representations for a specific task and avoid The Curse Of Dimensionality (COD) [44]. The COD occurs when the prediction system is overfitted [44]. The clustering defines the grouping of database samples to classes [45]. It could be used as a preprocessing stage for the enhancement of classification and prediction accuracy results. For UL, the clustering consists of grouping the inputs to sub-classes according to specific criteria [46].

AI contributes to the power grid by autonomous and fast equipment control. The communication of the system elements provides flexibility to the grid and ensures an optimal load supply balance. The grid integrity via AI ensures the required protection from cyber threats. The generated data provide a reliable source with AMI systems. The received information is used to tune and enhance AI techniques. The data management and control platform are carried out through AI. Great importance has been addressed to AI techniques due to their crucial role in grid transformation. The information-sharing framework assigns core importance to intelligence since it presents the key success with the support of edge computing [47].

In light of the inherent necessity of advanced tools, the system adopts some fundamental aspects. Those aspects consist of digitalization, flexibility, resiliency, and intelligence [48]. The efficiency of such a system will improve the penetration of alternative energy sources since the stability of the outdated grid is threatened by the internment change of weather. With AI, precautions are taken for any slight perturbation to select the
next suitable decision-making. However, the transition to future grids cannot be achieved without the high connectivity of advanced power electronics and devices through high-performance computing and monitoring techniques. With daily operations, a tremendous amount of data is generated and stored in the system. The role of an intelligent grid system is to manage this data and make it useful for monitoring and control. The static hierarchical grid needs require AI techniques for a safe and stable transition.

## 2.4 AI methods for energy management

Recently, AI receives major attention from stakeholders and energy managers to boost their financial benefits in a highly competitive market [49]. In the energy field, AI interferes in modeling, simulation, and prediction to cope with energy sources heterogeneity and operation complexity [43]. CI contributes to power systems with intelligent programming and advanced methodologies that emerge the SG-enabling strategies. Fault diagnosis, maintenance scheduling, restoration, remedial control, substation monitoring, and management excessively rely on AI to work efficiently and gratify the requirements for safe and secured power system operations [49]. Similarly, the deployment of AMI increases exponentially the amount of data collected from the system operations. AMI architecture incorporates ICT technologies, smart meters, and data management platforms [50]. Smart meter data analytics deployment overtakes consumption measurement to provide a massive database for further usage perspective. Using AI techniques, the collection of this rich information allows the SG paradigm to cope with people's lifestyles and decrease their electric bills. The research community takes advantage of the widespread popularity of AI techniques to resolve the operational problems of power systems. In the AI domain, the hybridization approach offers insights to improve the existing models. From a quantitative investigation, hybrid models are commonly implemented in energy systems as a clue for higher robustness and precision according to a fair assessment basis [51]. Fig. 2.5 illustrates the frequently used ML models for SG linked to their most suitable platforms for deployment according to the number of citations from the Scopus engine.

According to Fig. 2.5, The most overwhelming models in real-world applications include Support Vector Machines (SVM), Principal Component Analysis (PCA), Gradient Boosted Machines (GBM), K-Nearest Neighbors (KNN), Decision Trees (DT), DL (DL), Random Forest (RF), Genetic Algorithm (GA), Ensemble Models (EM), Convolutional Neural Network (CNN), Auto-Regressive Integrated Moving Average (ARIMA), and Generative Adversarial Networks (GAN). To follow the share of ML algorithms deployment for SG paradigm in the year 2019-2020, a pictorial flowchart of the AI models and their frequency of use is presented in Fig. 2.6.
<figure><img src="image_9.png" /><figcaption>Figure 2.5: Machine learning models with their open-source libraries [1].</figcaption></figure>

According to Fig. 2.6, ANN and Swarm intelligence are still hot topics for scientists due to their mature development and high efficiency. The frequent use of ML models is explained by the increased popularity of open-sourced platforms dedicated to facilitating their implementation, such as Keras [52]. On the other side, every method is preferred for a specific application of the SG area to perform a custom task better than the rest of the models due to a special inner mechanism. For example, DL takes advantage of its well-designed architectures to do feature engineering by default. This part is devoted to tackling the most relevant AI techniques used in SG operations for a flow and cohesion section. These techniques consist of ANN, Fuzzy Logic, Ensemble methods (EM), and hybrid models.

<figure><img src="image_10.png" /><figcaption>Figure 2.6: Pictorial representation of the AI Models applied to SG in 2019-2020 [1].</figcaption></figure>
### 2.4.1 Artificial neural networks-based

Conceptually, ANN model is an interconnection of interconnected processing units called neurons to mimics the present understanding of the human brain and its associated nervous system [53]. The information processing passes through different computational layers, namely, input, hidden, and output layers. The computational stages are forward propagation in the input layers, followed by numerically weighted values optimization and backpropagation in the output layers. The *N* vector inputs pass through the activation function $\varphi(\cdot)$ with the initial weights $w$ and bias $b$ values of each node. The weights $w_k$ and the bias $b_k$ of elementary processors called perceptrons are adjusted through the loss function between the ground truth and the prediction values in every back-propagation.

ANN provides a universal approximator to learn from the unsupervised environments [54]. By numerical weighting, ANN solves the issue of exclusive and nonlinear relationships. Therefore, the ANN is denoted as one of the most promising solutions in energy management. There are numerous other advantages of this concept, particularly in higher-dimensional search space; still, the dilemma of ANN deployment consists of high model complexity, long training time for deep ANN, and higher computational effort [54].

Some authors in [54] implement the ANN model for the short-term demand prediction of a microgrid supply system using one-year historical data. The Backpropagation neural network (BP) is proposed to generate MAPE=2.12% [54]. The simulation results yield that the complexity of ANN does not necessarily produce a high forecasting accuracy [54]. The authors in [55] demonstrate that ANN can forecast Global Horizontal Irradiance and Global Temporal Irradiance with a high exactitude. In the authors' work [55], the forecasting horizon was set to 5 minutes ahead while the historical database was built from seven meteorological data centers. The authors in [56] use ANN classifier for fault detection diagnostics in DC microgrids. Their work aims to stabilize the current, the frequency, and the phasor information to better integrate RES. The use of Wavelet Transform (WT) processing stands for capturing the dynamic changes during the faults. The ANN classifier determines the localization of the faults for fast detection and isolation of the faults in the system.

### 2.4.2 Fuzzy logic-based

Fuzzy Logic Control (FLC) is the development of the fuzzy set theory and its associated techniques. This supervisory technique mimics human cognition, and approximate reasoning [57]. FLC involves uncertainty in engineering by attaching degrees of certainty contrary to binary logic. The Fuzzy control system involves four elements, particularly, fuzzification module, knowledge base, inference engine, and defuzzification module as shown in Fig.2.7 [57]. Due to the imprisonment robustness of FLC model and its great generalization
<figure><img src="image_11.png" /><figcaption>Figure 2.7: Fuzzy inference system structure [1].</figcaption></figure>

capabilities, it has been widely integrated with energy systems [58]. Paper [59] proposed a smart meter with an autonomous energy management system using FL. The implementation of the FL system enables saving 34% of energy consumption during the peak period. However, the proposed controller is very sensitive to weather variations and the user's stochastic behavior.

Authors in [60] propose a trust management model-based fuzzy system. The proposed model is designed to secure the SG network against compromising lightweight devices by measuring the trustworthiness of devices. The trustworthiness of smart devices is FLC system has been deployed to facilitate the RES integration and monitoring of electricity distribution and production between different elements in the grid. The simulation results lead to a classification accuracy of 80%. In [61], the authors introduce an Interval Type-2 Fuzzy Logic System (IT2FLS) based MPPT to maximize the PV generation. The proposed model not only tracks the maximum PV power but also pays attention to results uncertainty. The IT2FLS model contributes to the stability of the MPPT model while optimizing the PV power generation compared to Perturb and Observe (P&O) algorithm.

According to the reviewed articles, It has been demonstrated that Fuzzy systems have a flexible and suitable architecture where the fuzzy rules could be changed to serve any logical system. Therefore, FLC model provides high efficiency in modeling and controlling nonlinear dynamical systems that require a high level of expertise in deriving mathematical parametric equations. Furthermore, FLC exhibits several features, including the ease of implementation and the ability to cope with imprecise, incomplete, or uncertain information. However, the shortcomings of FL include its dependence on the problem formulation because it does not always provide a satisfactory performance due to misleading fuzzy rules and membership function parameters. Thus, careful attention must be put on fuzzy rules selection with all membership combinations. Moreover, since FLC is widely applied to control systems, the evaluation process is costly due to the fees of hardware implementation [62].
### 2.4.3 Ensemble methods-based

Model assembling targets building a strong model from base learners with higher generalization ability, over-fitting avoidance, fault tolerance, and adaptability to solve complex problems. Ensemble learning techniques include two major classes based on the way of building base learners: competitive ensembles and cooperative ensembles [63]. Competitive ensembles use multiple base predictors to constrict a powerful ensemble by averaging single base predictors' prediction. The prediction output could take into account purring the base predictors. The weak predictors' diversity is maintained by employing different parameters, initial conditions, kernel diversity, or data diversity [63]. On the other side, for cooperative ensembles, the prediction task is partitioned into distinguished sub-folds. Then, the cooperative model computes each sub-fold individually. The prediction results are averaged to generate the final forecast output.

Cooperative ensemble forecasting includes two classes: Pre-processing and post-processing. In the pre-processing, the subsets of the cooperating ensembles are divided without considering the data characteristics. Wavelet Decomposition and Empirical Mode Decomposition (EMD) are two popular time-series examples for pre-processing [64]. While in the post-processing, the data linearity is taken into account. The cooperative ensembles attribute the most suitable subsets for each base model according to their potential capabilities to carry out specific data characteristics. For example, ARIMA is used as a linear model and associated with a nonlinear model such as Generalized Auto-Regressive Conditional Heteroscedasticity (GARCH) model, SVM, and ANN [65]. Ensemble learning techniques present efficient meta-algorithms for forecasting problems related to SG systems [64, 65].

The goal of using EM in energy systems consists of avoiding class imbalance, COD, and concept drift. This goal is conducted using two frameworks according to the dependency of inducers in the training stage [66]. The dependent training uses transfer learning to correlate the actual inducer and the following inducers and acquire knowledge. On the other side, independent training separates the base learners to be trained individually. For example, Bootstrap Aggregation (Bagging) such as RF and DT, stacking, and blending approaches are the most well-known independent frameworks. While Boosting is one of the commonly used dependent frameworks in practical applications for its high efficiency, including Adaptive Boosting (AdaBoost), Categorical boosting (Catboost), and Gradient Boosting Machines (GBM) [67]. Bagging is an intelligent algorithm that employs a parallel generation of individual estimations to optimize the variance. Although the Bagging approach demonstrates its prediction ability in high-variance data, studies approve that the risk of overfitting is high. Therefore, it is recommended to define a threshold when the learning rate stops improving. While using the Boosting approach, the accuracy is increasingly improved via sequential generation and weighted average.
Boosting is advantageous because the optimization accords more importance to the bias rather than the variance. The stacking approach consists of creating multiple prediction levels from heterogeneous models [68]. The output result is generated through a combination of meta-learner training folds and using an out-of-fold learning technique. Those folds are utilized as a base learner, and the information is transferred via the folds training. The boosting method prevents the system from overfitting and generates a high-quality prediction [69]. However, The Stacking restraint lies in time-consuming with a substantial computational burden requirement. Blending is quite similar to Stacking because it uses the fold-training technique, but the validation set is made only from the training folds. The holdout set accelerates the training process to simplify the learning process [70].

A novel predictive method for energy price classification was proposed in [71]. The proposed method does not identify the exact value of electricity price, but it classifies the values to a specific threshold. Their work was extended to include an investigation of Classification and Regression Trees (CART) and C5.0 models. The combination of EM provides high accuracy of 99.3%. Zhang et al. used a combination of two EM; specifically, RF and XGB, to provide generalization capabilities for fault detection of wind turbines [72]. RF model is implemented to rank the features according to their contribution to the domain knowledge, while XGB trains the model from three features.

### 2.4.4 Deep learning-based

DL is a promising branch of ML with breathtaking potential and widespread adoption from the research community and big tech companies. As a novel wave of ANN research, DL takes advantage of multiple processing layers, improved parallel calculation, and excellent representation ability to boost the performance of computational models using massive high dimensional data [73, 74]. Neural networks architectures with different configurations and structures such as RNN, CNN, GAN, DBN, and auto-encoders are the typical DL models to solve complex problems in the area of SG and power electronics [75, 76]. For instance, LSTM has been found very suitable for data-driven supervised learning with its three thresholds, specifically, input gate, output gate, and forget gate [76]. These gates tend to reduce the vanishing gradient from RNN, where the information is lost in-depth when it back-propagates. However, RNN-based structures, including LSTM and GRU, have limited potential in learning spatial features. Unlike LSTM networks, CNNs can solve the spatial dependencies using three sections, specifically, convolutional layers, pooling layers, and dense layers. Thus, the CNN method is well-known for its inherent ability to carry out data with different dimensionalities [4]. However, CNN algorithm faces serious learning problems with temporal data representations [4]. GAN has gained tremendous interest as a mainstream super-resolution model. GAN uses a generator and discriminator
neural network to create synthetic data that follows the same distribution from the original data set [77]. More specifically, the generator network mimics the data distribution using noise vectors to confuse the discriminator in differentiating between fake and real images. The role of the discriminator is trained to distinguish generated fake image created by the generator from the original image following the two-player zero-sum game [78]. Despite the large popularity of GANs, the assessment of GANs is quite challenging and often limited to computer vision and image recognition applications.

In [77], the authors used GAN and CNN for day ahead PVPF. Wasserstein Generative Adversarial Networks with Gradient Penalty (WGANGP) is proposed to classify 33 meteorological weather types for generating synthetic training data. From the simulation results, it has been found that weather classification plays a measure role in determining the most suitable PVPF technique. In [79], the authors proposed a DL approach for interval wind forecasting. The proposed model employs CNN, and GRU models to learn spatio-temporal representations for probabilistic forecasting. From the loss versus epoch variation, it is worth saying that the convergence of the model was reached in the first ten iterations, which may lead to serious over-fitting problems. In [80], the authors proposed an interval probability distribution learning model for wind forecasting. The proposed model, namely, deep mixture density neural network, employs restricted Boltzmann machines, rough set theory, and fuzzy sets to capture the intrinsic uncertainties from the wind data. It has been found that fuzzy type II inference system could supplement the DL models' robustness leading to an RMSE=0.419 (m/s) for 10-min ahead forecasting. Despite the intense competitiveness of the proposed forecasting system, serious Hyperparameter Optimization (HO) and high computational power are highly required in the proposed model application in real operational conditions. To overcome the mentioned problems for wind power forecasting, Spiking neural network (SNN) was introduced as one of the third generations ANN [81].

Despite the inherent characteristics of DL, overfitting persists as a complex problem. Overfitting happens with inadequate data size leading to poor generalization potential [82]. Regularization techniques such as dropout and batch normalization tackle the overfitting problem by using shared weights or network capacity and internal co-variate shift reduction. Data augmentation can be mentioned as an efficient technique to reduce overfitting caused by data scarcity. Furthermore, DL models require extensive work on HO and model structure. Although DL is deeply investigated in power electronics, the reported limitations impede the latter concept from becoming a canonical approach in the industry.
### 2.4.5 Hybrid models-based

Previous studies, to date, have been primarily focused on AI model' hybridization as a potentially viable approach since it gives individual models a massive upgrade in the majority of cases. Conceptually, hybrid techniques include two or more heterogeneous predictors. The rest of the models compensates for the weakness of a single predictor in capturing individual and group behavior from data. The evolution of AI passes through the fusion of techniques is a key solution to overcome narrow tasks in which conventional ML techniques generate poor accuracy. A variety of hybrid methods uses optimizers and predictors in the forecasting engine to minimize errors. The hybridization of ML techniques illustrates a promising approach, especially with narrow datasets. Fig. 2.8 presents a variety of hybrid techniques dedicated to SG applications.

<figure><img src="image_12.png" /><figcaption>Figure 2.8: Representation of hybrid two-method models comprising GA, ensembles, ANN, FL, SVM, DL, and ARIMA [1].</figcaption></figure>

These techniques outperform conventional models and significantly ameliorate model uncertainty. There is extensive interest in hybrid models regarding the number of papers that dramatically increased during the last ten years. The authors in [83] depicted the load forecasting problem using an Adaptive Neuro-Fuzzy Inference System (ANFIS) and compared from FL model. A yearly dataset is widely used for the training part. ANFIS is a supervised adaptive MLP. It combines the self-learning capability from NN with the linguistic expression function of FL. The mechanism of this architecture consists of converting the series data
into fuzzy inputs. In the inference system, these fuzzified data build the IF-THEN rules, the implication and defuzzification take place to indicate output weighting factors from the preceding part and then convert the fuzzy rules into crisp values. The proposed method in [83] provides accurate hourly load forecasts for the electric consumption data of Turkey using a monthly three years dataset from 2009 to 2011. Although both algorithms are very efficient in load prediction, ANFIS model outperforms FL [83]. However, the high complexity results in a long computational time. Some authors in [84] integrated an ANFIS model but with a two-stage feature selection with a hybrid Backtracking Search Algorithm (BSA). The proposed model deals with an electricity price prediction. BSA presents an efficient method for membership function tuning. The feature selection is held through a multi-objective binary-valued backtracking search algorithm to create various subsets from the dataset. Then, ANFIS model assesses the importance of each dataset.

Paper [85] proposed a hybrid method for wind energy forecasting. The proposed method combines a Deep Belief Network (DBN) and Support Vector Regression (SVR). DBN is a supervised DL technique having 1-layers and parabolic hidden cells [85]. The construction of layers is made by Restricted Boltzmann Machine (RBM). The idea of this fusion is resumed in the fact that the wind consists of two levels: hourly average wind power level and turbulent residual error level. Thus, each predictor is concerned by a single level. Despite the transient stochastic variation, DBN-SVR model generates quite accurate results with an error MAE equal to 0.014. Furthermore, the proposed model outstrips individual models' accuracy (SVR and DBN). However, the proposed method is time-consuming [85]. In a similar vein, the authors in [86] combine Long Short-Term Memory (LSTM) and CNN for power demand forecasting.

The hybrid approach, namely (c,1)-LSTM-CNN proved their high performance in various areas. For the energy field, this hybrid model uses dual-stage attention, and the inputs are passed through LSTM cells for features extraction with two hidden layers. Then, CNN model uses these data processed to make predictions. Compared with S2S LST, ARIMA, and (c,1)-LSTM. The model shows a better performance according to MAPE and RMSE scores. However, the suggested model is high complexity for just a short horizon in which more sampler techniques can generate for the same range of accuracy. Paper [87] combines WT and ensemble learning types illustrated in AdaBoost model. This paper tackles the problem of short-term wind speed forecasting. WT is deployed to transform the data into numerous scales and derive a divided subset. Overall, AI techniques are divided into seven learning techniques with different degrees of complexity according to their distinctive architectures, as shown in Fig.2.9.
<figure><img src="image_13.png" /><figcaption>Figure 2.9: Taxonomy of AI techniques applied to smart grids [1].</figcaption></figure>

According to Fig. 2.9, The AI learning techniques include Statistical learning, Ensemble learning, Swarm intelligence, ML and RL, Bayesian learning, DL, and Hybrid models. The learning diversity refers essentially to the nature of rules deployed, parametric functions, the information processing stages, learning styles, and application domains. A clear understanding of the AI taxonomy mapping enables the users to select the most suitable algorithm for their applications and thus, improving the prediction system performance. It must be noted that all the algorithms mentioned in the flowchart are taken from real applications on power systems and SG. On the other side, the short-term prediction is investigated rather intensively. The efficiency of
these methods is limited for further steps ahead in medium- and long-term predictions.

However, ES and meta-heuristic algorithms in AI have lost their popularity due to their high complexity and difficult implementation compared to ML models. Forecasting and prediction are the dominant sorts of using CI due to the availability and abundance of high-resolution data in most cases. Therefore, the prediction system is able to learn solely from concept examples. For particular problems, scare data limits the profitable use of ML due to high acquisition cost or low data storage efficiency. These problems include anomaly detection, predictive control, diagnosis, time series classification, etc. Data Augmentation (DA) is an effective approach to overcome poor data quality. DA uses basic approaches such as Fast Fourier transform and WT to convert the data into the time domain, frequency domain, or time-frequency domain [88]. Moreover, several advanced methods were proposed in the literature to increase the limited labeled data using decomposition methods, model methods, and learning methods [89]. Table 2.1 present the advantages and limitations of the presented models.
<table><caption>Table 2.1: Comparative study of the proposed techniques [1].</caption><thead><tr><td>Model</td><td>Strengths</td><td>Limitations</td></tr></thead><tbody><tr><td>ANN</td><td>-Ability to approximate continuously differentiable functions<br/>- Fault tolerance ability</td><td>- Easy to overfitting<br/>- Local minima can reduce the model performance<br/>- The complex structure of a large-scale</td></tr><tr><td>FL</td><td>- High Flexibility and consistency<br/>- Efficient computation<br/>- Universal approximation capability<br/>- Accurate and practical</td><td>- Hard to define the optimal fuzzy rule base<br/>- A large number of parameters to tune<br/>- Can be affected by the curse of dimensionality</td></tr><tr><td>EM</td><td>- Easy to understand<br/>- Non-parametric<br/>- Better generalization capability of multiple models</td><td>-Previous knowledge of system/ May overfit data<br/>It can get stuck in local minima<br/>- The training cost is too high</td></tr><tr><td>DL</td><td>-offer unsupervised feature engineering<br/>- High robustness and reliability</td><td>- Requires a large amount of training data labels<br/>- Previous knowledge of system<br/>- Results can be incomprehensible</td></tr><tr><td>HM</td><td>- Better generalization capabilities<br/>- High accuracy<br/>- Overcoming the limitations of each method used alone</td><td>- More complex structure<br/>- High computational burden<br/>- Difficult to implement</td></tr></tbody></table>

All these models hide some limitations in their configuration, data exploration of the heavy computational complexities. Therefore, meta-heuristic optimization techniques were adopted due to their efficiency and scalability for various complex applications, including designing the perfect architectures of DL approaches.

### 2.4.6 Evaluation procedure

In SG systems, the AI techniques' evaluation is highly important due to the intrinsic uncertainty of the heterogeneous parameters to the grid. The volatility of the SG environment may cause a considerable loss of accuracy, which threatens the whole SG system operations. The assessment of ML models is conducted using
a diversity of score metrics [90], etc. Score metrics are devoted to revealing a key idea on the model perfor-
mance for unseen data in the testing phase. The growing interest in accuracy measures is demonstrated by a
large number of ML techniques proposed. However, adopting the most convenient score function according
to data characteristics and model proprieties is non-trivial for ML techniques. For the classification type,
confusion matrix and classification accuracy are commonly used due to their simplicity and generalization
capabilities for such tasks. However, for regression problems, the score functions diversity requires a better
understanding of the forecasting system. Table 2.2 presents the mathematical equations of the most popular
accuracy metrics from regression and classification problems [90]. These methods were found to be the most
suited indicators to truly reflect the performance of ML techniques [91].

$y_i$, $\hat{y}_i$ and $\bar{y}$ are the actual truth, the forecasted value, and the normalization factor, respectively. $R^2$, $n$, and $k$ are the determination coefficient, the forecast horizon, and the $k^{th}$ sample, respectively. PINC denotes Prediction Interval Nominal Confidence. $TP$ denotes True Positive, $TN$ denotes true negative, $FN$ denotes False Negative, and $FN$ denotes the False Positive. Performance metrics are employed for the model selection stage to judge the suitability of the model face to a specific task, training stage to follow the learning structure, or in the tuning and adjustment stage to enhance the model efficiency. Authors in [90] classified the score metrics into seven-folds according to the nature of the data. These folds include measures based on percentage errors, absolute forecasting errors, scaled errors, symmetric errors, measures based on relative errors, relative measures, and other error measures as shown in Table 2.3 [91,92].

The purpose of suggesting a new type of score metrics is needed for preventing some mathematical errors
such as the zero-denominator error. The widely used ML score metrics for regression evaluation in the recent
research articles are ranked according to their use for SG applications in Fig. 2.10.

<figure><img src="image_14.png" /><figcaption>Figure 2.10: Commonly used score metrics for regression assessment [1].</figcaption></figure>

From Fig. 2.10, it can be concluded that the RMSE is the most reliable and widely used score metric
followed by MAPE and MAE. Regardless of percentage errors, generally, the score metrics, the smaller,
the better. Many researchers aim to make standard criteria for effectiveness measures. Nevertheless, the
<table><caption>Table 2.2: Some exemplary score errors.</caption><thead><tr><th>Name</th><th>Abbreviation</th><th>equation</th></tr></thead><tbody><tr><td colspan="3"><strong>Regression</strong></td></tr><tr><td>Mean Absolute Error</td><td>MAE</td><td>MAE = <sup>1</sup>&frasl;<sub>N</sub> &sum;<sub>i=0</sub><sup>N-1</sup> |y<sub>i</sub> - ŷ<sub>i</sub>|</td></tr><tr><td>Root Mean Squared Error</td><td>RMSE</td><td>RMSE = &radic;<span style="text-decoration:overline;"><sup>1</sup>&frasl;<sub>N</sub>(&sum;<sub>N=0</sub><sup>N-1</sup> y<sub>i</sub> - ŷ<sub>i</sub>)<sup>2</sup></span></td></tr><tr><td>Mean Absolute Percentage Error</td><td>MAPE</td><td>MAPE = <sup>100(%)</sup>&frasl;<sub>N</sub> &sum;<sub>N=0</sub><sup>N-1</sup> |ŷ<sub>i</sub> - y<sub>i</sub>|</td></tr><tr><td>Coefficient of determination</td><td>R<sup>2</sup></td><td>R<sup>2</sup> = 1 - <sup>&sum;<sub>n=0</sub><sup>n-1</sup>(ŷ<sub>i</sub> - y<sub>i</sub>)<sup>2</sup></sup>&frasl;<sub>&sum;<sub>n=0</sub><sup>n-1</sup>(ŷ<sub>i</sub> - y<sub>i</sub>)<sup>2</sup></sub>, ŷ = <sup>&sum;<sub>i=0</sub><sup>n-1</sup> y<sub>i</sub></sup>&frasl;<sub>&sum;<sub>i=0</sub><sup>n-1</sup> y<sub>i</sub></sub></td></tr><tr><td>Median absolute error</td><td>MdAE</td><td>MdAE = Md(|y<sub>i</sub> - ŷ<sub>i</sub>|)</td></tr><tr><td>Standard deviation</td><td>σ</td><td>σ = &radic;<span style="text-decoration:overline;"><sup>&sum;<sub>i=0</sub><sup>N-1</sup>(y<sub>i</sub> - ŷ<sub>i</sub>)</sup>&frasl;<sub>N</sub></span></td></tr><tr><td>Normalized Root Mean Squared Error</td><td>nRMSE</td><td>nRMSE = 100(%) <sup>&radic;<span style="text-decoration:overline;"><sup>1</sup>&frasl;<sub>N</sub>(&sum;<sub>N=0</sub><sup>N-1</sup>(y<sub>i</sub> - ŷ<sub>i</sub>)<sup>2</sup></span>&frasl;<sub>ŷ<sub>i</sub></sub></sup></sup>&frasl;<sub>100(%)</sub></td></tr><tr><td>Normalized Mean Absolute Error</td><td>nMAE</td><td>nMAE = 100(%) <sup>1</sup>&frasl;<sub>n</sub> <sup>&sum;<sub>i=0</sub><sup>n-1</sup> |y<sub>i</sub> - ŷ<sub>i</sub>|</sup>&frasl;<sub>100(%)</sub></td></tr><tr><td>Mean Bias Error</td><td>MBE</td><td>MBE = &radic;<span style="text-decoration:overline;"><sup>&sum;<sub>i=1</sub><sup>N</sup>(ŷ<sub>i</sub> - y<sub>i</sub>)</sup>&frasl;<sub>N</sub></span></td></tr><tr><td>Prediction Interval Coverage Probability</td><td>PICP</td><td>PICP = <sup>1</sup>&frasl;<sub>n</sub> &sum;<sub>i=0</sub><sup>n-1</sup> ζ, ζ = { 1, T<sub>i</sub> &isin; [L<sub>i</sub>, U<sub>i</sub>] 0, T<sub>i</sub> &notin; [L<sub>i</sub>, U<sub>i</sub>]</td></tr><tr><td>Prediction Interval Normalized Average Width</td><td>PINAW</td><td>PINAW = <sup>1</sup>&frasl;<sub>nR</sub> &sum;<sub>i=0</sub><sup>n-1</sup> (U<sub>i</sub> - L<sub>i</sub>)</td></tr><tr><td>Average Coverage Error</td><td>ACE</td><td>ACE = PICP - PINC</td></tr><tr><td colspan="3"><strong>Classification</strong></td></tr><tr><td>Accuracy</td><td>Accu-<br>racy</td><td>Accuracy = <sup>TP+TN</sup>&frasl;<sub>TP+FP+TN+FN</sub></td></tr><tr><td>Precision</td><td>Precision</td><td>Precision = <sup>TP</sup>&frasl;<sub>TP+FP</sub></td></tr><tr><td>Recall</td><td>Recall</td><td>Recall = <sup>TP</sup>&frasl;<sub>TP+FN</sub></td></tr><tr><td>F1-score</td><td>F<sub>1</sub></td><td>F<sub>1</sub> = 2 &times; <sup>Precision &times; Recall</sup>&frasl;<sub>Precision + Recall</sub></td></tr></tbody></table>

selection of error metrics depends essentially on the nature of the data and the type of application deployed. If the task estimated is classification, then it is preferable to use the ratio of the correct estimated samples by the forecasting horizon. On the other side, for regression, data exploration lies at the core of the goodness of prediction models. Therefore, the choice of which one is computed should be first given higher consideration
<table><caption>Table 2.3: Classes of score metrics [1].</caption><thead><tr><th>Class</th><th>Advantages</th><th>Shortcomings</th></tr></thead><tbody><tr><td>Absolute errors</td><td>-Popularity<br>-Simplicity</td><td>-Scale dependencies.<br>-High sensibility to outliers.<br>-Low reliability with different datasets.</td></tr><tr><td>Percentage errors</td><td>-High Reliability</td><td>-Biased error measures<br>-High sensibility to outliers.<br>-Possible errors due the division by zero<br>-Asymmetry problems</td></tr><tr><td>Scaled errors</td><td>-Symmetrical<br>-Not effected by outliers</td><td>-Possible errors due the division by 0</td></tr><tr><td>Symmetric errors</td><td>-Less calculation errors<br>-Reliability</td><td>-Non-symmetric values<br>-Possible errors due the division by 0<br>-Less usage</td></tr><tr><td>Relative errors</td><td>-High Reliability</td><td>-Possible errors due the division by 0</td></tr><tr><td>Relative measures</td><td>-Ease of Interpretability</td><td>-Possible errors due the division by 0<br>-Less reliability</td></tr></tbody></table>

for the zero-denominator error (RMSE, MAPE, MASE). Cross-Validation (CV) method is also an effective method for avoiding biased data.

This method a trained data from a part of the dataset to make predictions of another fold. Despite the efficiency of cross-validation, it is time-consuming. For multiple types of regression techniques, the best method for assessment use the determination coefficient (R-square) with automatic model selection. In [93], the authors introduce two categories for score metrics selection which are Absolute Accuracy Measures (AAM) and Accuracy measures relative to another Method (AMRAM). AAM use their own mathematical function for calculation, such as RMSE and MAE. They are commonly used because they are simple and straightforward interpreted. AMRAM models involve the percentage of times better. This includes the percentage of time that one method is more accurate than one other. Despite all the work done, it must be mentioned that it is necessary to find a conventional method to assess the bench of prediction techniques.

## 2.5 Summary

AI-as a service provides potential benefits to the energy sector, including full automation of decision-making. This chapter outlines the commonly emerging AI-driven edge computing for SG operations. A vast majority of case studies from the literature demonstrated that hybridization and assembling straighten AI techniques lead to better accuracy and robustness. Towards that end, power systems require logical reasoning and meta-heuristic search methods to solve narrow problems effortlessly. On the other side, the forecasting horizon
plays an essential role in assigning the ML method to its suitable application. However, AI techniques
are often oriented toward the Short-term horizon. Regarding the reviewed work from the previous studies,
it can be observed that DL has acclaimed significant attention in the recent few years due to the high
availability of IoT devices and smart meters. However, complex ANN architectures are computationally
expensive and time-consuming. Therefore, parallel and distributed computing presents a highly active area
of research to reduce the training time and computational complexity in order to have better adaptability
to industrial applications of SG. From the reviewed articles, it has been found that the current development
of AI deployment paves the way for SG's significant expansion. However, the AI progress faces several
challenging points, including significant computational burden, higher complexity, and scalability, which
requires further investigation.

The main content of this chapter has been published in [1], [19], and [94].
# Chapter 3

## PV power forecasting

### 3.1 Introduction

SE is the largest RES with an overall capacity of 1.5×1018 kW h/year reaching the earth's surface [95]. This enormous energy potential represents nearly 10,000 times the actual yearly world energy consumption [95]. However, the vagaries of weather lead to high intermittency and volatility of the generated PV power for energy practitioners [96]. PV generation's intrinsic nonstationary and uncertain nature is mainly caused by erratic and time-varying solar irradiation behavior. Sudden fluctuations in PV generation can lead to reverse power flow, poor power quality, and voltage regulation problems.

Despite being a new concept, PVPF gained significant economic value, and the methodologies and best practices are progressively enhancing. Sound and realistic forecasts of the PV output can bring viable advantages for power systems such as fast dispatchability, grid balancing, high efficiency, PV plant operation monitoring, and secure ride-through unanticipated failures [94]. The dire necessity for accurate PVPF techniques shad light on the ample opportunities for integrating more large-scale distributed PV generators. Thus, significant efforts from both industry and academia have been conducted to improve forecast quality. However, forecasting distributed PV generation is quite challenging due to the discontinuous appearance of solar irradiation.

PV generation forecast methodologies can be taxonomized according to forecasting periods into three classes, specifically short-term forecasting (STF) required to be valid for seconds to one day, medium-term forecasting for one day to weeks, and Long-Term Forecasting (LTF) that may be valid for years [97]. Broadly speaking, the STF of the PV generation output tends to be more precise than LTF. Furthermore, the forecasting methodologies can be classified as physical methods, statistical methods, AI methods, or a mix of them (hybrid models) [98]. The physical models use numerical weather prediction (NWP) models or satellite imagery alongside physical considerations such as meteorological or topological data. However, physical models are restricted to tedious mathematical approaches for specific PV plants, leading to poor generalization potential [98]. Statistical models employ prediction models such as Moving Average (MA)
and Autoregressive (AR) [99]. AI methods employ computational intelligence to predict the PV output accurately, taking advantage of the evolved enhancement in hardware and software. The optimal models are often a combination of physical and statistical models [100]. According to the literature, it has been found that the combination of different forecasting models could enhance the performance and efficiency of the overall prediction paradigm [95, 101]. In order to tackle the above problems, PVPF has been emphasized using two case studies and multiple testing scenarios as shown in fig3.1. The proposed models are dedicated to both small and large data sizes. By incorporating Nonlinear Auto-Regressive with exogenous input Neural Network (NARXNN) into deep LSTM, we introduce an accurate interval model for point prediction of unstable solar power datasets. The first proposed model takes advantage of the enormous capabilities of DL by implementing NARX-LSTM model and Tabu-Search algorithm for HO. The second model uses wavelet transform, Catboost, and ridge regression to build an effective prediction model for PVPF.

<figure><img src="image_15.png" /><figcaption>Figure 3.1: PV power forecasting methodology.</figcaption></figure>

The chapter is organized as follows: Two efficient computing frameworks are first depicted to provide a high-fidelity power system generation. First, section 3.3 proposes a novel short-term PVPF technique based on NARX-LSTM model. Second, section 3.4 proposes an efficient machine learning model which takes advantage of wavelet transform to enhance the feature representation of the overall system. Extensive simulation results and comparison analysis are conducted to validate the performance superiority of the proposed approaches. Finally, the chapter summary is provided in Section 3.4.

## 3.2 Case study 1

### 3.2.1 Proposed methodology

Forecasting is a complex and narrow problem. Due to the volatility in weather states, short term PVPF are well suited to generate multiple forecasts based on various case scenarios. In this section, we focus on
the proposed framework with more details. At the core of our method lies the Tabu search algorithm that identifies the optimal configuration for the hybrid NARX-LSTM algorithm.

## NARX Neural Network

NARXNN is a specific type of dynamic neural network based on the linear Auto-Regressive exogenous (ARX) model. In dynamic networks, the output is affected by both previous input and output layers of the model. Thus, this NARXNN model has been praised for its competitive potential to handle nonlinear problems due to its fast convergence and versatility, especially with time series data. The NARXNN architecture essentially comprises an input, feedback, and output layers. The NARXNN acquires the information from a feedback layer connected to the input layers. NARNN uses two tapped delay lines in the input and output layers. Alternatively, NARXNN employs a couple of tapped-delay links from the input/output samples [102]. The parametric function of the network output is expressed as follows [103]:

$$a(k) = f[b(k); \bar{u}(k); \bar{a}(k-1)] \qquad (3.1)$$

$$a(k) = f[b(k), \dots, b(k-d_E+1); \hat{b}(k), \dots, \hat{b}(k-d_a+1)] \qquad (3.2)$$

where $\bar{u}(k) \in \mathbb{R}$ and $\bar{a}(k) \in \mathbb{R}$ are the model input and output at $k$ time-step. $d_E \ge 1$ and $d_a \ge 1$ denote the input and output memory orders respectively with $\{d_E, d_a\} \in \mathbb{N}^*$. The feedback loop enhances the ability of the forecasting model in recognizing hidden time-series patterns. Fig. 3.2 presents the NARXNN structure.

<figure><img src="image_16.png" /><figcaption>Figure 3.2: Structure of NARX network [2].</figcaption></figure>

Regarding Fig.3.2, the NARXNN model can be described as a two-layer feedforward architecture, with a linear transfer function in the output layer and a sigmoid activation function ($\sigma$) in the hidden layer
computed as [104]:

$$ \sigma(x) = 1 + \exp(-\sigma)^{-1} \qquad (3.3) $$

From the literature, two configurations were proposed for training the NARXNN: Series-Parallel (SP) mode and Parallel (P) modes [105]. The SP mode provides an embedded output memory using the information of the true output values. In the P mode, previous forecasting outputs were considered inputs to estimate future information by linking the feedback loop between input and output samples. The key favor of NARX-SP mode lies in the resulting network has a pure feed-forward network and static back-propagation network for better performance.

Paper [104] demonstrated the high accuracy of the NARXNN model for efficient pattern recognition of the sequential data compared to the state-of-the-art reference models [106]. Important to note, the NARXNN efficiency is essentially related to two tapped-delay lines from the input-output layers. In contrast to the standard RNN model, the NARXNN exclusively acquires the feedback signals to the output layers rather than including the hidden layers' information. Nonetheless, it has been demonstrated that this dynamic architecture proves its performance supremacy compared to the basic RNN models [107].

However, the significant limitation of the NARXNN lies in the large additional number of parameters coming from higher polynomial orders. Moreover, similarly to vanilla RNN, the exploding gradients are still the major weakness of the series-parallel identification mode [103]. This limitation makes this dynamic model prone to over-fitting when processing extremely long temporal data. This vanishing gradient problem encounter when the gradient descent retracts in highly separated long-term regularities, leading to fading memory dilemma [103].

## Long Short-Term Memory

The LSTM model is a time RNN with a control gate structure to exhibit dynamic temporal behavior from time series related tasks. This specialized RNN model tends to avoid gradient disappearance and gradient explosion by filtering the feedback information. The LSTM prevents the learning system from the vanishing gradient problem. Fig. 3.3 illustrates the detailed structure of the LSTM cell.
<figure><img src="image_17.png" /><figcaption>Figure 3.3: LSTM memory block, including the input, output, and forget gates [2].</figcaption></figure>

From Fig.3.3, the LSTM cell includes the input gates $I_t$ to which define how much ratio of actual input could be added to the cell state, the forgetting gates $f_t$, which calculates the forget rate of the memory unit given current input, and output gates $o_t$ responsible of controlling how the memory unit will affect the node output. This configuration is designed to handle sequential data using memory units effectively. The LSTM cells are devoted to guiding the RNN model to learn when to neglect inconsequential information and when to update memory units through new input samples. The LSTM gating structure is calculated with the following formulas [108]:

$$I_t = \sigma(W_I x_t + U_i h_{t-1} + b_I) \qquad (3.4)$$

$$f_t = \sigma(W_f x_t + U_f h_{t-1} + b_f) \qquad (3.5)$$

$$o_t = \sigma(W_o x_t + U_o h_{t-1} + b_o) \qquad (3.6)$$

$$\tilde{c} = f_t \odot c_{t-1} \qquad (3.7)$$

$$h_t = o_t \odot \tanh(c_t) \qquad (3.8)$$

$x_t$ and $c_t$ are the inputs and the memory cells. ($b_f, b_I, b_o$), ($W_f, W_I, W_o$), and ($U_f, U_I, U_o$) denote the biases, recurrent weights, and input weights for each gate, respectively. The parameter $\odot$ denotes the related multiplication operation. $h_{t-1}$ denotes the hidden states for the corresponding gates $x$ in the actual time-step. Assuming $x = [x_1, \dots, x_t]$ denotes the input features for LSTM unit with $x_t \in \mathbb{R}^k$. $K$ presents the dimensional vector of the real data set at the $t^{th}$ time-step.

First, the $h_{t-1}$, $c_{t-1}$, and $x_t$ process the input sequences to the LSTM cells. The LSTM gating structure influences the entered sequence to evaluate the learned information's importance. After connecting with $f_t$, a new cell state $c_t$ is built. $x_t$ and $h_{t-1}$ move to the forget layer to estimate the significance of the information. Here, the $f_t$ gate duty is to scrap or add temporal information allowing data to pass selectively. Next, the forgetting gate actualizes the cell state $c_t$ with the beneficial information. The output layer uses
the important information from the memory unit [108].

## Tabu Search Algorithm

Meta-heuristic algorithms have largely fueled exponential interest in HO. Broadly speaking, the performance of ML models depends heavily on their hyperparameter configuration. Due to its simplicity, fast-convergence ability, and generalization potential, meta-heuristics have been broadly applied to HO. Proposed by F. Glover in 1986, meta-heuristic local search algorithm. The TSA is a local search approach with a flexible memory structure to avoid trapping the solution in local optima. This is done by forbidden recorded movements in the tabu list [109]. The best candidate solution of the combinatorial optimization task in the neighborhood is selected. Fig. 3.4 presents the overall structure of TSA:

<figure><img src="image_18.png" /><figcaption>Figure 3.4: Framework of the proposed TSA algorithm [2].</figcaption></figure>

Regarding Fig. 3.4, let $A = \{a_1, ..., a_M\}$ with $M$ is the cardinality of $A$. Let a set $S(a_q), q \in \{1, ..., M\}$, a fixed subset $A \setminus a_q$. $a_q$ present the symbol-neighborhood of $a_q$. define $w_v(a_q), v = 1, ..., N$ as the $v$th symbol-neighbor of $a_q$. Assuming $x^m = [x_1^m, x_{n_t}^m]$ with $x^m \in A$ denote a proposed solution in the $m$th iteration. The $(u, v)$th neighborhood vector $z^m(u, v)$ of $x^m, u = 1, n_t, v = 1, ..., N$ based on Euclidean distance can be written as [110]:

$$z^m(u, v) = [z_1^m(u, v), z_2^m(u, v), ..., z_{n_t}^m(u, v)] \qquad (3.9)$$
The $(u, v)$th neighbor follows the condition [110]:

$$z_i^m(u, v) = \begin{cases} x_i^m & \text{for } i \neq u \\ w_v(x_u^m) & \text{for } i = u \end{cases} \qquad (3.10)$$

In the solution space, $n_t N$ vectors are generated from a defined vector in a specific coordinate. Then, the TSA moves to the best vector among the area while fulfilling the non-move back towards the past solution space that has already been traced. The Tabu matrix $(T)$ of size $n_t M \times N$ and $t$ coordinates representing the tabu values of moves is computed as:

$$T = \begin{pmatrix} t_{1,1} & t_{1,(n_t-1)M+1} & \cdots & t_{1,n_tM} \\ \vdots & \vdots & \ddots & \vdots \\ t_{N,1} & t_{N,(n_t-1)M+1} & \cdots & t_{N,n_tM} \end{pmatrix} \qquad (3.11)$$

Define $g^m$ the vector with the maximum-likelihood performance cost generate until the $m$th operation of TSA. The shift $(u_1, v_1)$ is accepted if one of the following conditions is respected [110]:

$$\varphi(z^m(u_1, v_1)) \leq \varphi(g^m) \qquad (3.12)$$

$$T(u_1 - 1)M + q, v_1 = 0 \qquad (3.13)$$

where the term $q$ is defined as $a_q = x_{u_1 m, a_q \in A}$. The next move is calculated as [110]:

$$(u_2, v_2) = \underset{u,v; u \neq u_1, v \neq v_1}{\arg \varphi(z^m(u, v))} \qquad (3.14)$$

The iteration process is carried out until the optimal solution is met. TSA has many merits in terms of adaptability, robustness, and accuracy. The solid and super-quick neighborhood search capacity made the TSA well-known and widely used for complex problems [109]. Therefore, TSA stands tied very closely to HO for PV stakeholders to enable unbiased ML method development with high scalability potential in real systems.

## Proposed NARX-LSTM model

The proposed model (NARX-LSTM) denotes the fusion of NARXNN and LSTM algorithms. The proposed NARX-LSTM algorithm combines the key characteristics of LSTM and NARX to improve the forecasting
system. For this current special architecture, NARXNN is linked with embedded memory cells that make jump-ahead connections in the time-unfolded network. The NARXNN is employed hereinafter to quantify the residual error correction values. The residual error correction is used to reduce the error values of the final output. Assuming that the additional tapped time delays from NARXNN enhance the efficiency of the forecasting system using the error Hankel matrix. Assuming $E_n = [e_1, ..., e_n]$ be the error vector between the real values $Y_t = [y_1, ..., y_n]$ and the forecasted values of NARXNN $\hat{Y}_t = [\hat{y}_1, ..., \hat{y}_n]$. The core calculation formula of the residual error is as follows:

$$E_n = Y_t - \hat{Y}_t = y_t t - \sum_{i=1}^{n} w_i F_{it}, \quad (t = 1, \dots, n) \qquad (3.15)$$

where $F$ and $w$ denote the nonlinear mapping function of NARXNN and the corresponding weight value. A Hankel matrix is build from the expansion of the error vector into a multi-dimensional data matrix in order to capture the dynamic change of the model variation as:

$$E_n^* = [e_1^*, \dots, e_L^*] = \begin{pmatrix} e_1 & e_2 & \cdots & e_L \\ \vdots & \vdots & \ddots & \vdots \\ e_K & e_{K+1} & \cdots & e_{K+L-1} \end{pmatrix} \qquad (3.16)$$

where $L \in [2 \le L \le L(n/2)]$ is the window length and $K = n - L + 1$ is the number of overlapping segments. At this stage, two types of transformations are computed for the Hankel error matrix: Min-Max transformation and Hilbert transformation (HT) [111, 112]. Let $f(x)$ be the error function and $H()$ the Hilbert transform operator. The HT $g(x)$ of $f(x)$ is calculated as [111]:

$$g(y) = H(f(x)) = \frac{1}{\pi} \int_{-\infty}^{\infty} \frac{f(x)dx}{x-y} = \frac{1}{\pi} \int_{-\infty}^{\infty} \frac{f(x-y)dx}{x} \qquad (3.17)$$

The HT can be reformulated as a convolution by:

$$g(y) = \frac{1}{\pi X} f(x) \qquad (3.18)$$

A fast algorithm based on convolution theorem is employed to develop the HT as [111]:

$$g(y) = ifft(fft(\frac{1}{\pi x})fft(f(x))) \qquad (3.19)$$

Where `fft()` and `ifft()` denote the fast Fourier transform and the inverse fast Fourier transform respectively
satisfying [111]: .

$$ \text{fft}(\frac{1}{\pi x}) = -j\operatorname{sgn}(\text{freq}) \qquad (3.20) $$

where freq and sgn indicate the frequency and the sign function respectively. Referring to Eq.3.19, the product `fft(1/πx)` `fft(f(x))` denotes special filtering of Fourier transform and the inputs. The computation of the inverse Fourier transform on the product leads to produce the HT of the error vector. HT is employed for its effectiveness for nonstationary and nonlinear time series data. The proposed framework comprises two essential steps: In the first step, the NARXNN acquire the meteorological indicators data to generate initial PV generation' predictions as $\hat{Y} = [\hat{y}_1, \hat{y}_2, \dots, \hat{y}_n]$. The former estimations generated from NARXNN are employed to compute the error values and generate the vector error correction. The vector error correction is computed by assembling the Min-Max transform and HT of the error values vector produced by NARXNN. The vector error correction function (V) is linked with the original database as entries to the LSTM model optimized by TSA, and the calculation formula is:

$$ X(t) = x_i(t) + V[y(t); x_i(t)] \qquad (3.21) $$

By using this approach, the LSTM model procures the learning capability of NARXNN to enhance the performance potential of the overall prediction paradigm. The LSTM gating structure utilizes the tangible information as:

$$ i'_{t} = \sigma(W_{I}(x_{t} + V[y_{t}; x_{t}]) + U_{I}h_{t-1} + b_{I}) \qquad (3.22) $$

$$ f'_{t} = \sigma(W_{f}(x_{t} + V[y_{t}; x_{t}]) + U_{f}h_{t-1} + b_{f}) \qquad (3.23) $$

$$ o'_{t} = \sigma(W_{o}(x_{t} + V[y_{t}; x_{t}]) + U_{o}h_{t-1} + b_{o}) \qquad (3.24) $$

$$ \tilde{c}' = f_t \odot c'_{t-1} \qquad (3.25) $$

$$ h'_{t} = o'_{t} \odot \tanh(c'_{t}) \qquad (3.26) $$

where $(i'_t, f'_t, o'_t)$ present the enhanced input, forgetting, and output gates, respectively. $\tilde{c}'$ and $c'$ denote the cell state and the memory unit for NARX-LSTM, respectively.

The explanation for the use of NARXNN lies in its fruitful execution in dealing with the latching issue and nonlinear pattern recognition. Then again, LSTM cells decline the exploding gradient. For a superior comprehension of the proposed model, the data preparation is continued in four phases as demonstrated in
<figure><img src="image_19.png" /><figcaption>Figure 3.5: Main framework of the proposed hybrid NARX-LSTM [2].</figcaption></figure>

In the primary step, data preprocessing and feature engineering is carried out beginning from data acquisition to collect valuable data about the condition of the PV framework. At that point, data cleaning from absent and invalid instances registration and feature significance assessment is conducted to provide more reliable data representation for the forecasting system. At long last, the information is standardized with a greatness scope of [0, 1] utilizing Min-Max technique [112]. In the subsequent step, a common information detachment method between inputs and outputs from the training and testing folds is conducted in the object determination step. Next, the training information is acquired to NARXNN model to produce the PV generation estimations. Then, the NARXNN outcomes are employed to calculate the vector error correction to be fed to the LSTM model optimized by TSA. The output of the model construction step is the PV power point predictions and their related vulnerabilities.

From this clarification, NARX-LSTM could be portrayed as a combination of two modules classes: (1) NARXNN that gets the succession of outer inputs and the recurrent output layer state; (2) LSTM cells that
get restorative data with the first highlights to produce the last yields. Consequently, the residual data took
care of the NARXNN to be reutilized to enhance the LSTM model with an extra wellspring of data. The
stream diagram of the proposed model appears in Fig. 4.30.

<figure><img src="image_20.png" /><figcaption>Figure 3.6: General flow chart of developed NARX-LSTM algorithm [2].</figcaption></figure>

3.2.2 Simulation results

The feature engineering technique is carried out based on the Australian meteorological data. At that point,
the experimental results are computed to demonstrate the accuracy of the proposed method fully. At last,
a performance comparison is handled to assess the potential capabilities of the underlying technique. The
core goal is to show the capacities of NARX-LSTM in following temporal patterns with high exactness.

Feature Engineering

To assess the capability of the proposed framework utilizing an actual data set, the simulation results employ
two PV power systems from different locations to validate the speculation abilities of the NARX-LSTM
model.
1. $\mathcal{D}_1$: the available database from Desert Knowledge Alice Springs Center (DKASC) in Central Australia (23.7618° S, 133.8749° E) [113]. Beginning in September 2008, the DKASC comprises an exhibition facility of 38 destinations to construct a high certainty level of PV plants with various brands from competitive companies [114]. These destinations consist of real-life monitoring of PV technologies from various sorts, models, and setups as introduced in Tables 3.1-4.2. The specific characteristics of the two years from DKASC's public-facing repository were profoundly investigated to focus on the PV power yield from DKASC online portal (DKASC 2019). The data representation of our framework includes ambient temperature Celsius (T in (°C)), Wind direction (Wd in Å), Horizontal Radiation (Ir in (W/m²)), Diffuse horizontal radiation (DIr in (W/m²)), and Relative humidity (Rh in (%)) while the output is the active power (kW).

2. $\mathcal{D}_2$: Urbana-Champaign solar farm-Flyover (UCF) dataset from USA (40°06′07.3″N 88°13′37.5″W) [115]. The climate information is gathered from the National Oceanographic and Atmospheric Administration (NOAA). Beginning from its first year of activity, The UCF plant creates around 7.28 megawatt-hours (MWh). This last is viewed as the biggest solar array installation for Urbana campus. The pre-owned information comprises T, Rh, Cloud coverage (Cc), visibility, Wind speed (Ws), station Pressure (P), Altimeter indication (A), and PV power. More insights regarding the attributes of UCF is found in Tables 3.1-4.2.
<table><caption>Table 3.1: The related characteristics of the PV plant [2].</caption><thead><tr><th>System specification</th><th>Characteristics</th></tr></thead><tbody><tr><td colspan="2">D<sub>1</sub></td></tr><tr><td>Array rating</td><td>191.74 kW</td></tr><tr><td>Average of Powering</td><td>141house</td></tr><tr><td>Location</td><td>475 South Stuart Highway, Alice Springs, Australia</td></tr><tr><td>PV technology</td><td>(Mono/Poly)crystalline Silicon Thin Film (CdTe/CIGS)</td></tr><tr><td>Array area</td><td>4 * 38.37m<sup>2</sup></td></tr><tr><td>Type of tracker</td><td>Fixed: Ground Mount, Single Axis, Dual Axis</td></tr><tr><td>Inverter size/type</td><td>46 kW, SMA/Sunny Mini Central 6000A</td></tr><tr><td colspan="2">D<sub>2</sub></td></tr><tr><td>Array rating</td><td>5.87 MWp</td></tr><tr><td>Location</td><td>South of Windsor Road between First Street and the railroad tracks</td></tr><tr><td>Array area</td><td>20.8 acres</td></tr><tr><td>Average of powering</td><td>2% of the electrical demand for the Urbana campus</td></tr><tr><td>Number of panels</td><td>18867 panels</td></tr><tr><td>Carbon reduction</td><td>6000 metric tons/year</td></tr></tbody></table>

<table><caption>Table 3.2: Data types [2].</caption><thead><tr><th>PV station</th><th>Data</th><th>timestep</th><th>Training year</th><th>Testing year</th><th>Simples</th><th>Time window</th></tr></thead><tbody><tr><td>Case study I</td><td>D<sub>1</sub></td><td>5min</td><td>2017</td><td>2018</td><td>183680</td><td>3/31/2017 23:55 - 01/01/2019 00:00</td></tr><tr><td>Case study II</td><td>D<sub>2</sub></td><td>1hour</td><td>2016</td><td>2017</td><td>15068</td><td>01/02/2016 04:00 - 28/10/2017 23:00</td></tr></tbody></table>

It can be said that the data set is cleaned from absent values and smoothed from sensors breakdown'
measurements. The input factors have an immediate connection with the anticipated PV power yield. However, this relationship isn't similarly divided. Different techniques have been proposed to gauge the variety of feature importance. In this investigation, the idea is to permit one parameter and gauge the increment of estimating blunder for each situation by the likelihood value (P-value) computation.

Elastic Net and Extreme Gradient Boosting (XGB) are employed to investigate the trustworthiness of the data representation [116]. These models rank the features according to their relative Probability value (P-value) following the feature inequality C(s):

$$C(s) = \begin{cases} Keep \ if, IXGB(s) + ElasticNet(s) \ge t \\ Keep \ if, IXGB(s) + ElasticNet(s) < t \end{cases} \qquad (3.27)$$

where IXGB(s), ElasticNet(s) and t denote the calculated P-value by XGB and Elastic Net, and feature selection threshold respectively. Table 4.1 presents the numerical measures of P values for the collected meteorological features.
<table><caption>Table 3.3: Feature importance for system inputs on $\mathcal{D}_1$ and $\mathcal{D}_2$ [2].</caption><thead><tr><th>Features</th><th>XGB weights</th><th>Elastic Net weights</th></tr></thead><tbody><tr><td colspan="3"><strong>$\mathcal{D}_1$</strong></td></tr><tr><td>Horizontal radiation</td><td>1.9460 ± 0.0070</td><td>1.9917 ± 0.0075</td></tr><tr><td>Temperature</td><td>0.0274 ± 0.0005</td><td>0.0175 ± 0.0002</td></tr><tr><td>Lagged PV power</td><td>0.0212 ± 0.0002</td><td>0.0080 ± 0.0001</td></tr><tr><td>Relative humidity</td><td>0.0071 ± 0.0003</td><td>0.0017 ± 0.0001</td></tr><tr><td>Diffuse horizontal radiation</td><td>0.0064 ± 0.0000</td><td>0.0048 ± 0.0001</td></tr><tr><td>Wind direction</td><td>0.0046 ± 0.0001</td><td>0.0014 ± 0.0000</td></tr><tr><td colspan="3"><strong>$\mathcal{D}_2$</strong></td></tr><tr><td>Relative humidity</td><td>0.4104 ± 0.0127</td><td>0.1641 ± 0.0054</td></tr><tr><td>Temperature</td><td>0.3739 ± 0.0074</td><td>1.6558 ± 0.0229</td></tr><tr><td>Station pressure</td><td>0.0910 ± 0.0043</td><td>0.0545 ± 0.0033</td></tr><tr><td>Wind speed</td><td>0.0785 ± 0.0042</td><td>0.0243 ± 0.0027</td></tr><tr><td>Altimeter indication</td><td>0.0348 ± 0.0019</td><td>0.0013 ± 0.0001</td></tr><tr><td>Cloud coverage</td><td>0.0342 ± 0.0029</td><td>0.0000 ± 0.0000</td></tr><tr><td>Dew point</td><td>0.0274 ± 0.0021</td><td>0.9074 ± 0.0091</td></tr><tr><td>Visibility</td><td>0.0030 ± 0.0004</td><td>0.0054 ± 0.0015</td></tr></tbody></table>

As per Table 4.1, the irradiation from $\mathcal{D}_1$ and the temperature and relative humidity from $\mathcal{D}_2$ significantly affects the PV power prediction. For effortlessness, the time series data representation incorporates all the collected meteorological features as inputs associated with the date features (minute/hour/day/month/year) to predict the future PV power. The date features are numerically transformed using a one-hot encoder to fit the model prerequisites. The overall database should be divided into two data sets, which are training and testing. A complete year $\{2017_{\mathcal{D}_1}, 2016_{\mathcal{D}_2}\}$ is committed to model training and the remainder of $\{2018_{\mathcal{D}_1}, 2017_{\mathcal{D}_2}\}$ is utilized for testing purposes.
## Point PV Power Forecasting

This part handles the numerical validation of the model performance utilizing 2 datasets from various countries (USA and Australia) with different timesteps of information procurement. The feasibility of NARX-LSTM model is carried out using Python coding. Keras and Sckit-learn libraries are employed to facilitate the model implementation [117,118]. The model is assessed on single-step and multiple steps ahead respectively. The simulation environment setup is depicted in Table 3.4.

<table><caption>Table 3.4: Development settings [2].</caption><thead><tr><td>Experimental Environment</td><td>Properties</td></tr></thead><tbody><tr><td>Operating system</td><td>Windows 10</td></tr><tr><td>Processor</td><td>Intel Core i7 9th Gen</td></tr><tr><td>Graphics card</td><td>NVIDIA GeForce GTX 1650</td></tr><tr><td>Programming language</td><td>Python 3.6.7</td></tr><tr><td>Deep learning framework</td><td>Keras 2.4.3</td></tr><tr><td>Platform</td><td>Tensorflow 2.3.0</td></tr><tr><td>Supplementary tools</td><td>Sklearn 0.23.2</td></tr></tbody></table>

The TSA is executed utilizing Hyperactive library [119]. This framework uses a defined search space to distinguish the optimal hyperparameters for a particular database. To avoid the extensive computational requirement, Trial and Error (T&E) experiments tests were directed to limit the search space. It can be noted that the configuration of twenty NARXNN neurons, a four delay value, a Levenberg-Marquardt optimization algorithm, and three LSTM layers create the best outcomes. Then, LSTM cells, dropout ranges, activation, and optimizer functions are calculated utilizing TSA optimization. The search space for TSA and the optimization results are resumed in Table 3.5. To confirm the framework universality for a PV plant with a higher scale, the automatic search system is led uniquely for $\mathcal{D}_1$.
<table><caption>Table 3.5: Search space and TSA results of the proposed SG-CBiLSTM [2].</caption><thead><tr><th>Model</th><th>Configuration</th><th>Search Space</th><th>TSA</th></tr></thead><tbody><tr><td rowspan="10">LSTM</td><td>LSTM layer 1</td><td>[64, 512]</td><td>512</td></tr><tr><td>LSTM layer 2</td><td>[36, 256]</td><td>256</td></tr><tr><td>LSTM layer 3</td><td>[36, 256]</td><td>36</td></tr><tr><td>Dropout 1</td><td>[0.1, 0.5]</td><td>0.1</td></tr><tr><td>Dropout 2</td><td>[0.1,0.5]</td><td>0.1</td></tr><tr><td>Dropout 2</td><td>[0.1,0.5]</td><td>0.1</td></tr><tr><td>activation</td><td>[Sigmoid,Softplus,Selu,Elu,Softmax]</td><td>Softplus</td></tr><tr><td>optimizer</td><td>[adam,SGD, RMSprop]</td><td>adam</td></tr><tr><td>Time (min)</td><td></td><td>189.52</td></tr><tr><td>Time/iteration (min)</td><td></td><td>18.95</td></tr></tbody></table>

According to Table 3.5, it worth mentioning that the LSTM layers are fixed at 512, 256, and 36 units for the three LSTM levels, respectively. The dropout values are fixed at 0.1 for all the previously mentioned levels. The optimizer function is adam. The most reasonable activation function, Softplus, is determined as follows:

$$f(x) = \log(1 + \exp(x)) \qquad (3.28)$$

It bears noting that the TSA is computationally demanding with a conversion speed of 18.95 min per iteration. Although intensive, the TSA usage could be demonstrated by the fact that the high-resolution dataset for a small time-step of 5 minutes causes an exhaustive calculation due to a large number of examined patterns. The final architecture of the proposed model is presented in Fig. 3.7.
<figure><img src="image_21.png" /><figcaption>Figure 3.7: Optimal configuration of (a) NARXNN model (b) LSTM model [2].</figcaption></figure>

The LSTM deep learning model employs 100 iterations with an early stopping operation to prevent over-fitting issues. Fig. 3.8 and Fig. 3.9 show, for five minutes and one hour ahead, the actual/predicted daily PV power for various meteorological states comprising sunny, rainy, foggy, partially cloudy, and cloudy hours of the day).

<figure><img src="image_22.png" /><figcaption>Figure 3.8: Different possible scenario for $\mathcal{D}_1$ dataset [2].</figcaption></figure>
<figure><img src="image_23.png" /><figcaption>Figure 3.9: Different possible scenario for $\mathcal{D}_2$ dataset [2].</figcaption></figure>

Regarding Fig. 3.8-3.9, the predicted power tracks the true PV power with high accuracy. It can be inferred that the proposed method is highly efficient for PVPF. It is worth saying that the model performance is very prominent, particularly during sunny days. Notwithstanding, the model precision is debased in quick changes of the climate conditions (Fig. 3.8.(e)). Nevertheless, even with quick vacillations of the meteorological factors, the NARX-LSTM network can follow the overall state of the actual PV power. To present the model exhibition, the scatter plots and the error distributions are outlined in Fig. 3.10.
<figure><img src="image_24.png" /><figcaption>Figure 3.10: Scatter plots ((a), (b)) and error distributions ((c), (d)) of actual and predicted PV output for $\mathcal{D}_1$ and $\mathcal{D}_2$ [2].</figcaption></figure>

As illustrated in the scatter plots with $\mathcal{D}_1$ and $\mathcal{D}_2$, the proposed model follows the real PV power with high exactness. In spite of the low error measures, the 1 hour ahead daily PV expectations on $\mathcal{D}_2$ generates better outcomes. For a quantitative evaluation with score measurements and contrast models, Table 4.18 presents the benchmarks' configuration such as LSTM, NARX, XGB, ANN, and LGBM. It is important to say that Random search method is adopted to train XGB, LGBM ANN. While LSTM and NARXNN use the original HO from the construction of the proposed NARX-LSTM model.

The point forecasting measurements are used to compare different environmental conditions (sunny, partially cloudy, cloudy, and rainy). The simulation results are illustrated in Fig. 3.11 and 3.12.
<figure><img src="image_25.png" /><figcaption>Figure 3.11: PV power forecast results in 5 min ahead daily forecasting using $\mathcal{D}_1$ [2].</figcaption></figure>
<figure><img src="image_26.png" /><figcaption>Figure 3.12: PV power forecast result in 1-hour ahead forecasting horizons using $\mathcal{D}_2$ [2].</figcaption></figure>

As indicated to Fig. 3.11, it can be clearly seen that the forecasting execution profoundly relies upon a specific period of the year. In particular, the prediction performance of NARX model decreases significantly during the winter days (Fig. 3.11. (d) and Fig. 3.12(d)). In comparison, NARX-LSTM provide a better accuracy among reference models. The performance superiority is observed on winter and spring days. With more detailed information comparing the original LSTM with the proposed model, it can be said that the error correction vector has essentially upgraded its exhibition in catching the pattern of the real PV power. From Fig. 3.12, it has been found that the performance supremacy has been referred to NARX-LSTM. Tables 3.6-3.7 resume the error values on $\mathcal{D}_1$ and $\mathcal{D}_2$ respectively.
<table><caption>Table 3.6: Score performance comparison on $\mathcal{D}_1$ [2].</caption><thead><tr><th>Weather</th><th>Model</th><th>R<sup>2</sup> (%)±SD</th><th>nRMSE (%)±SD</th><th>nMAE (%)±SD</th></tr></thead><tbody><tr><td rowspan="5">Sunny</td><td>NARX-LSTM</td><td>99.4</td><td>2.39</td><td>1.46</td></tr><tr><td>LSTM</td><td>95.04</td><td>7.98</td><td>6.38</td></tr><tr><td>XGB</td><td>96.91</td><td>6.05</td><td>4.8</td></tr><tr><td>LGBM</td><td>97.52</td><td>3.98</td><td>4.35</td></tr><tr><td>NARX</td><td>97.86</td><td>4.67</td><td>3.26</td></tr><tr><td rowspan="5">Partially cloudy</td><td>NARX-LSTM</td><td>99.27</td><td>2.14</td><td>1.23</td></tr><tr><td>LSTM</td><td>95.88</td><td>5.6</td><td>3.82</td></tr><tr><td>XGB</td><td>94.53</td><td>6.58</td><td>4.31</td></tr><tr><td>LGBM</td><td>96.51</td><td>3.92</td><td>4.23</td></tr><tr><td>NARX</td><td>77.85</td><td>11.47</td><td>6.15</td></tr><tr><td rowspan="5">Cloudy/foggy</td><td>NARX-LSTM</td><td>99.29</td><td>2.33</td><td>1.5</td></tr><tr><td>LSTM</td><td>95.59</td><td>6.59</td><td>4.93</td></tr><tr><td>XGB</td><td>95.82</td><td>6.15</td><td>4.56</td></tr><tr><td>LGBM</td><td>96.77</td><td>3.95</td><td>4.44</td></tr><tr><td>NARX</td><td>78.24</td><td>13.14</td><td>8.04</td></tr><tr><td rowspan="5">Rainy</td><td>NARX-LSTM</td><td>97.4</td><td>1.07</td><td>0.62</td></tr><tr><td>LSTM</td><td>95.28</td><td>1.62</td><td>1.12</td></tr><tr><td>XGB</td><td>92.2</td><td>1.9</td><td>1.14</td></tr><tr><td>LGBM</td><td>36.12</td><td>4.14</td><td>3.92</td></tr><tr><td>NARX</td><td>68.62</td><td>4.15</td><td>2.27</td></tr><tr><td rowspan="5">Overall</td><td>NARX-LSTM</td><td>98.84±0.83</td><td>1.98±0.53</td><td>1.20±0.35</td></tr><tr><td>LSTM</td><td>95.45±0.32</td><td>5.45±2.37</td><td>4.06±1.93</td></tr><tr><td>XGB</td><td>94.86±1.75</td><td>5.17±1.90</td><td>3.99±1.76</td></tr><tr><td>LGBM</td><td>81.73±26.34</td><td>4.62±0.49</td><td>3.94±0.02</td></tr><tr><td>NARX</td><td>80.64±10.66</td><td>8.36±4.01</td><td>4.93±2.29</td></tr></tbody></table>
<table><caption>Table 3.7: Score performance comparison on $\mathcal{D}_2$ [2].</caption><thead><tr><th>Weather</th><th>Model</th><th>R<sup>2</sup> (%)±SD</th><th>nRMSE (%)±SD</th><th>nMAE (%)±SD</th></tr></thead><tbody><tr><td rowspan="4">Sunny</td><td>NARX-LSTM</td><td>99.83</td><td>1.54</td><td>1.39</td></tr><tr><td>LSTM</td><td>95.72</td><td>6.74</td><td>4.3</td></tr><tr><td>XGB</td><td>90.25</td><td>9.76</td><td>5.65</td></tr><tr><td>LGBM</td><td>98.42</td><td>4.47</td><td>3.04</td></tr><tr><td rowspan="4">Partially cloudy</td><td>NARX</td><td>92.81</td><td>7.92</td><td>8.65</td></tr><tr><td>NARX-LSTM</td><td>99.91</td><td>1.09</td><td>0.96</td></tr><tr><td>LSTM</td><td>95.11</td><td>7.14</td><td>4.64</td></tr><tr><td>XGB</td><td>81.5</td><td>12.56</td><td>8.45</td></tr><tr><td rowspan="4">Cloudy/foggy</td><td>NARX</td><td>90.13</td><td>10.11</td><td>9.01</td></tr><tr><td>NARX-LSTM</td><td>99.62</td><td>1.88</td><td>1.63</td></tr><tr><td>LSTM</td><td>88.67</td><td>10.34</td><td>6.84</td></tr><tr><td>XGB</td><td>83.94</td><td>12.6</td><td>7.49</td></tr><tr><td rowspan="4">Rainy</td><td>NARX</td><td>80.38</td><td>12.36</td><td>10.59</td></tr><tr><td>NARX-LSTM</td><td>99.41</td><td>0.82</td><td>0.67</td></tr><tr><td>LSTM</td><td>57.89</td><td>5.09</td><td>2.88</td></tr><tr><td>XGB</td><td>54.42</td><td>5.49</td><td>3.16</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.67</td><td>7.46</td><td>4.25</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr><tr><td rowspan="4">Overall</td><td>NARX</td><td>87.75±4.63</td><td>9.65±1.83</td><td>8.37±2.26</td></tr><tr><td>NARX-LSTM</td><td>99.69±0.19</td><td>1.33±0.41</td><td>1.16±0.37</td></tr><tr><td>LSTM</td><td>84.35±15.52</td><td>7.33±1.90</td><td>4.67±1.42</td></tr><tr><td>XGB</td><td>77.53±13.7</td><td>10.10±2.90</td><td>6.19±2.02</td></tr></tbody></table>

Regarding the numerical results introduced in Tables 3.6-3.7, it can be concluded that the NARX-LSTM
provides a better accuracy other single methods. The NARX-LSTM generate a mean nRMSE=1.98% and nRMSE=1.33% on $\mathcal{D}_1$ and $\mathcal{D}_2$ respectively. The model performance is The simulated results affirm the outstanding potential of the proposed NARX-LSTM in temporal dependencies prediction. Table 3.8 illustrate the execution time for the proposed model and benchmarks using the testing set.

<table><caption>Table 3.8: Computational complexity of each model on the testing set [2].</caption><thead><tr><th>model</th><th>D<sub>1</sub></th><th>D<sub>2</sub></th></tr></thead><tbody><tr><td>NARX-LSTM</td><td>3.37</td><td>0.904</td></tr><tr><td>LSTM</td><td>2.821</td><td>1.773</td></tr><tr><td>XGB</td><td>0.806</td><td>0.613</td></tr><tr><td>LGBM</td><td>0.682</td><td>0.636</td></tr><tr><td>ANN</td><td>0.637</td><td>0.626</td></tr><tr><td>NARX</td><td>1.578</td><td>0.84</td></tr></tbody></table>

From Table 3.8, the basic ANN offers the least calculation time. The proposed model generally more computationally requesting than the individual LSTM (case I), while on $\mathcal{D}_2$, the proposed model took less time than the original LSTM. Notwithstanding the more extended testing computation of the proposed NARX-LSTM model, it is effectively pertinent to true case scenarios (particularly with the advancement of distributed computing). As the investigation endeavors to offer a fair evaluation of NARX-LSTM model, a reasonable examination is led with the recently provided benchmark models, explicitly, Ensemble of Methods (ENS), improved ANN, Grey-Box model (GB), Correlation model, Extreme Learning Machines (ELM), modified LSTM, Differential Polynomial Neural Network (D-PNN), Auto-encoders LSTM (AE-LSTM), and Wavelet transform-Radial Basis function neural network- Particle Swarm Optimization (WT-RBFNN-PSO). The comparison of the proposed NARX-LSTM model with the cited models is conducted based on hourly ahead daily PVPF. Accordingly, the NARX-LSTM execution is solely included with $\mathcal{D}_2$ in this comparative experiment. Table 3.9 illustrates the prediction results of reference prediction models according to their mean nRMSE scores.
<table><caption>Table 3.9: Comparison of 1-hour ahead PVPF results [2].</caption><thead><tr><td>Model</td><td>nRMSE %</td></tr></thead><tbody><tr><td>ENS [120]</td><td>7.16</td></tr><tr><td>Improved ANN [121]</td><td>3.76</td></tr><tr><td>GB [120]</td><td>7.36</td></tr><tr><td>Correlation model [122]</td><td>5.68</td></tr><tr><td>ELM [122]</td><td>10.14</td></tr><tr><td>Modified LSTM [123]</td><td>8.83</td></tr><tr><td>D-PNN [124]</td><td>13.42</td></tr><tr><td>AE-LSTM [125]</td><td>8.39</td></tr><tr><td>WT-RBFNN-PSO [126]</td><td>1.85</td></tr><tr><td>NARX-LSTM</td><td>1.33</td></tr></tbody></table>

As per Table 3.9, the NARX-LSTM model could accomplish the best outcomes based on the overall forecasting performance in point forecasting with 1h forecasting range. The mean nRMSE of NARX-LSTM is nRMSE=1.33%, while the nearest model (WT-RBFNN-PSO) achieves a mean nRMSE=1.85%. For better visibility of the proposed model superiority among all top-of-the-line models based on hourly ahead daily PVPF, Fig. 3.13 illustrates a comparison results of all models based on nRMSE score.
<figure><img src="image_27.png" /><figcaption>Figure 3.13: Variation of PV parameters [2].</figcaption></figure>

As seen in Fig. 3.13, the NARX-LSTM model is favorable for short-term PVPF for its high precision among the existing models. To confirm the model effectiveness for multi-step forecasting, multiple forecasting ranges were computed. Due to the characteristic diversity between $D_1$ and $D_2$, $D_1$ will carry out one hour, six hours and twelve hours ahead while $D_2$ will predict the PV power for three hours, six hours and twenty-four hours ahead. Fig. 3.14 visualizes the generated forecasts, where four random days from the testing set were selected to conduct these experiments.

<figure><img src="image_28.png" /><figcaption>Figure 3.14: Multistep forecasting tests of NARX-LSTM model using $D_1$ and $D_2$ datasets for different forecasting horizons [2].</figcaption></figure>
As shown in Fig. 3.14, it can be seen that of the PV forecasts are surprisingly similar trends with the real values for 1 hour 3 hour and 6-hours ahead on $D_1$ and $D_2$. Nevertheless, the proposed algorithm generates a slightly larger error measures when the prediction range exceeds the six-hour timestamp. Tables 3.10 and 3.11 quantify the forecasting performance on 10-CV for $D_1$ and $D_2$.

<table><caption>Table 3.10: Score performance comparison using $D_1$ [2].</caption><thead><tr><th>Horizon</th><th>Fold</th><th>R<sup>2</sup> (%)</th><th colspan="2">nRMSE (%)</th><th colspan="2">nMAE (%)</th></tr><tr><th></th><th></th><th>train</th><th>test</th><th>train</th><th>test</th><th>train</th><th>test</th></tr></thead><tbody><tr><td rowspan="2">1 hour</td><td>Mean</td><td>99.89</td><td>99.88</td><td>0.97</td><td>0.99</td><td>0.49</td><td>0.49</td></tr><tr><td>SD</td><td>0.02</td><td>0.05</td><td>0.1</td><td>0.19</td><td>0.09</td><td>0.09</td></tr><tr><td rowspan="2">6 hours</td><td>Mean</td><td>99.6</td><td>99.57</td><td>1.85</td><td>1.93</td><td>0.79</td><td>0.8</td></tr><tr><td>SD</td><td>0.05</td><td>0.06</td><td>0.11</td><td>0.13</td><td>0.1</td><td>0.1</td></tr><tr><td rowspan="2">12 hours</td><td>Mean</td><td>98.86</td><td>98.73</td><td>3.13</td><td>3.31</td><td>1.17</td><td>1.21</td></tr><tr><td>SD</td><td>0.07</td><td>0.12</td><td>0.1</td><td>0.15</td><td>0.08</td><td>0.1</td></tr></tbody></table>

<table><caption>Table 3.11: Score performance comparison using $D_2$ [2].</caption><thead><tr><th>Horizon</th><th>Fold</th><th>R<sup>2</sup> (%)</th><th colspan="2">nRMSE (%)</th><th colspan="2">nMAE (%)</th></tr><tr><th></th><th></th><th>train</th><th>test</th><th>train</th><th>test</th><th>train</th><th>test</th></tr></thead><tbody><tr><td rowspan="2">3 hours</td><td>Mean</td><td>93.64</td><td>93.56</td><td>7.22</td><td>7.26</td><td>3.68</td><td>3.7</td></tr><tr><td>SD</td><td>0.28</td><td>0.59</td><td>0.15</td><td>0.33</td><td>0.14</td><td>0.11</td></tr><tr><td rowspan="2">6 hours</td><td>Mean</td><td>93.73</td><td>93.53</td><td>7.17</td><td>7.27</td><td>3.72</td><td>3.77</td></tr><tr><td>SD</td><td>0.27</td><td>0.58</td><td>0.15</td><td>0.26</td><td>0.17</td><td>0.17</td></tr><tr><td rowspan="2">24 hours</td><td>Mean</td><td>84.31</td><td>83.23</td><td>11.34</td><td>11.69</td><td>6.14</td><td>6.33</td></tr><tr><td>SD</td><td>1.11</td><td>1.41</td><td>0.42</td><td>0.43</td><td>0.42</td><td>0.33</td></tr></tbody></table>

According to 3.10, the performance of NARX-LSTM decreases with the increase of the forecasting horizon. A clear conclusion can be drawn that the proposed technique delivers more accurate single-step predictions than multiple steps predictions. However, from the simulation results on hourly daily ahead pre-
dictions, it is concretely evident that the proposed NARX-LSTM could generate accurate results with an $R^2 = 99.89 \pm 0.02$ and $R^2 = 99.69 \pm 0.19$ for $\mathcal{D}_1$ and $\mathcal{D}_2$ respectively. While the worse results are achieved for 12-hours and 24-hours ahead with $R^2 = 98.86 \pm 0.07$ and $R^2 = 84.31 \pm 1.11$ for $\mathcal{D}_1$ and $\mathcal{D}_2$ respectively. Fig. 3.15 shows the variation of the performance for a multistep analysis based on nMAE and nRMSE for ten fold cross-validation on $\mathcal{D}_1$ and $\mathcal{D}_2$.

<figure><img src="image_29.png" /><figcaption>Figure 3.15: Box-and-Whisker plots of nRMSE and nMAE errors for different forecasting horizons with $\mathcal{D}_1$ (case I) and $\mathcal{D}_2$ (case II) of NARX-LSTM model [2].</figcaption></figure>

As illustrated in Fig. 3.15, the forecasting skill of the proposed NARX-LSTM dramatically decreases after 6 hours ahead forecasting threshold. Therefore, the validity of the proposed model on 24-ahead forecasting is not considered in what follows. For the sake of comparing the model forecasting performance with benchmarks based on 3 hours and 6 hours ahead forecasting, several benchmarks were employed including Wavelet Transform-Generalized Regression Neural Network (WT-GRNN), Wavelet Transform-Generalized
Regression Neural Network-Particle Swarm Optimization (WT-GRNN-PSO), Wavelet transform-Radial Basis function neural network (WT-RBFNN), and WT-RBFNN-PSO models [126]. Table 3.12 resumes the forecasting performance of NARX-LSTM ($\mathcal{D}_2$) with the cited benchmarks based on 3-hour ahead multistep forecasting. While Table 3.13 presents a 6-hour ahead forecasting performance comparison for NARX-LSTM model.

<table><caption>Table 3.12: Comparison of 3-hour ahead PVPF results [2].</caption><thead><tr><th>Model</th><th>nRMSE %</th><th>nMAE %</th></tr></thead><tbody><tr><td>WT-GRNN [126]</td><td>12.88</td><td>7.65</td></tr><tr><td>WT-GRNN-PSO [126]</td><td>9.89</td><td>7.38</td></tr><tr><td>WT-RBFNN [126]</td><td>12.75</td><td>8.12</td></tr><tr><td>WT-RBFNN-PSO [126]</td><td><strong>6.85</strong></td><td>5.44</td></tr><tr><td>NARX-LSTM ($\mathcal{D}_2$)</td><td>7.26</td><td><strong>3.70</strong></td></tr></tbody></table>

Note: The best information criterion is in boldface.

<table><caption>Table 3.13: Comparison of 6-hour ahead PVPF results [2].</caption><thead><tr><th>Model</th><th>nRMSE %</th><th>nMAE %</th></tr></thead><tbody><tr><td>WT-GRNN [126]</td><td>12.38</td><td>9.78</td></tr><tr><td>WT-GRNN-PSO [126]</td><td>12.39</td><td>10.02</td></tr><tr><td>WT-RBFNN [126]</td><td>12.77</td><td>10.29</td></tr><tr><td>WT-RBFNN-PSO [126]</td><td>10.95</td><td>9.03</td></tr><tr><td>NARX-LSTM ($\mathcal{D}_1$)</td><td><strong>1.93</strong></td><td><strong>0.80</strong></td></tr><tr><td>NARX-LSTM ($\mathcal{D}_2$)</td><td>7.27</td><td>3.77</td></tr></tbody></table>

Note: The best information criterion is in boldface.

Table 3.12 presents the performance comparison of the proposed model with other benchmarks. The performance improvement of reconciled forecasts over reference models is validated with nMAE=3.70%. while WT-RBFNN-PSO (nRMSE=6.85%) is outperforming the NARX-LSTM (nRMSE=7.26%) in terms of
lower nRMSE values. On the other side, according to Table 3.13, the performance of the proposed model is best with the dominance of NARX-LSTM on $\mathcal{D}_1$ data set. The NARX-LSTM generates an nRMSE=1.93 % and nMAE=0.80% on $\mathcal{D}_1$, while the reported performance errors on $\mathcal{D}_2$ are equal to nRMSE=7.27% and nMAE=3.77% for 6-hour daily PVPF. The proposed model is found highly accurate with good competitiveness skills for a single step and multistep point forecasting. Nonetheless, a longer time horizon requires further investigations.

## Probabilistic forecasting of PV power

Due to the bulk integration of RES, grid utilities draw near the boundaries of a controllable uncertainty. With the increasing volatility from both of power supply and demand sides, probabilistic forecasting is essentially used to evaluate the intrinsic uncertainties of the PVPF accompanied with deterministic forecasting. However, probabilistic forecasting is a problematic task due to the intermittent variability of the meteorological conditions during the hours of the day. More importantly, the inevitableness of prediction errors shed light on the importance of probabilistic forecasting. Therefore, the inherent and prominent uncertainty of PV power generation is tackled in this work using interval PVPF method. The loss function $\ell$ of individual point prediction in quantile regression is applied by the following formula:

$$ \ell(\xi|\alpha) = \begin{cases} \alpha\xi_i & \xi \ge 0 \\ (\alpha - 1)\xi_i, & \xi < 0 \end{cases} \qquad (3.29) $$

where $\xi$ denotes the quantile value which lies between 0 and 1 and $\xi$ is defined as:

$$ \xi = y_i - f(x_i) \qquad (3.30) $$

$f(x)$ denotes the estimated quantile model. The average function for a given data set can be described as:

$$ \ell(y_i|\alpha) = \frac{1}{N} \sum_{i=1}^{N} \ell(y_i - f(x_i)|\alpha) \qquad (3.31) $$

The experimental results are conducted for two data sets to validate the not problem-specific of the forecasting model. The 1st case scenario is simulated for 5-min daily prediction while the 2nd case scenario is curried out for hourly daily prediction. The experiment outputs on $\mathcal{D}_1$ and $\mathcal{D}_2$ are shown in Fig. 3.16.
<figure><img src="image_30.png" /><figcaption>(a) Prediction intervals for case I using $\mathcal{D}_1$.</figcaption></figure>

<figure><img src="image_31.png" /><figcaption>(b) Prediction intervals for case II using $\mathcal{D}_2$.</figcaption></figure>

<figure><img src="image_32.png" /><figcaption>(a) NARX-LSTM at 5 min ahead with PINC=90% on $\mathcal{D}_1$.</figcaption></figure>

<caption>(b) NARX-LSTM at 1-hour ahead with PINC=90% on $\mathcal{D}_2$.</caption>

<caption>Figure 3.17: 90% Prediction interval results on $\mathcal{D}_2$ and $\mathcal{D}_2$ [2].</caption>

As can be seen from Fig. 3.16, it has been found that for case I-II that the prediction intervals enclose most of the actual values. The interval forecasting yields accurate uncertainty results for 5 min and 1 hour ahead predictions. In Fig.3.17, 90% prediction intervals are illustrated where they enclose the actual
PV power for different weather conditions. In order to quantify the probabilistic results, the interval error measures for case I-II are resumed in Table 3.14.

<table><caption>Table 3.14: Performance evaluation of probabilistic forecasting for case I-II [2].</caption><thead><tr><th>PINC</th><th>PICP</th><th>PINAW</th><th>ACE</th><th>Pinball Loss(%)</th></tr></thead><tbody><tr><td colspan="5" style="text-align: center; font-style: italic;">D<sub>1</sub></td></tr><tr><td>90%</td><td>96.07%</td><td>0.3071</td><td>6.08%</td><td>0.16</td></tr><tr><td>80%</td><td>95.09%</td><td>0.0437</td><td>1.51%</td><td>0.98</td></tr><tr><td>70%</td><td>73.52%</td><td>0.0142</td><td>3.52%</td><td>1.08</td></tr><tr><td colspan="5" style="text-align: center; font-style: italic;">D<sub>2</sub></td></tr><tr><td>90%</td><td>100%</td><td>0.2634</td><td>3.02%</td><td>0.12</td></tr><tr><td>80%</td><td>98.02%</td><td>0.0387</td><td>18.02%</td><td>0.32</td></tr><tr><td>70%</td><td>96.04%</td><td>0.039</td><td>26.04%</td><td>0.43</td></tr></tbody></table>

From an interval perspective, the error measures for $D_1$ and $D_2$ are computed in Table 3.14. More specifically, for PINC=90%, the NARX-LSTM generates a PICP=96% for 5 min ahead of daily forecasting. While PICP=100% for hourly daily forecasting using $D_2$ data set. Regarding the error measures, a general conclusion can be drawn that the probabilistic forecasting with NARX-LSTM is high performing. However, it can be said that 5 min ahead probabilistic forecasting produces lower results (PICP=96.07% for PINC=90%) than hourly forecasting from (PICP=100% for PINC=90%). This variation is explained by the high volatility of the actual PV power on $D_1$ data set. In this study, the assessment procedure includes the multistep analysis. The proposed framework is validated for 1 hour, 2-hours, and 3-hours ahead. The results are shown in Table 3.15.

According to Table 3.15, it should be reported that the proposed NARX-LSTM model is firmly effective in interval prediction where all the true values are encased by the produced prediction intervals, particularly in one hour ahead and two-hour time range. The enlargement of the forecasting range to three hours ahead prompts losing sight of exactness in terms that some of the actual results surpass the forecasting intervals. For instance, at 17:00, from the second case study, the real PV energy is 1193.5kW, while the prediction interval is [865.867,1175.649]. Notwithstanding, the increment of the forecasting horizon diminishes the prediction precision. As an end, the NARX-LSTM model demonstrates its appropriateness for single step,
one hour and two hour ahead interval prediction.

<table><caption>Table 3.15: Results of multi-step probabilistic forecasting for 1hour(1-h), 2 hours (2-h) and 3 hours (3-h) forecasting horizons [2].</caption><thead><tr><th>Time (hh:mm)</th><th>Actual value(kW)</th><th>1-h ahead</th><th>2-h ahead</th><th>3-h ahead</th></tr></thead><tbody><tr><td colspan="5">D<sub>1</sub></td></tr><tr><td>07:00</td><td>5.224</td><td>[3.733,9.727]</td><td>[3.406,11.228]</td><td>[5.643,17.995]</td></tr><tr><td>08:00</td><td>8.101</td><td>[6.879,11.327]</td><td>[4.578,13.646]</td><td>[6.229,17.796]</td></tr><tr><td>09:00</td><td>10.328</td><td>[9.574,14.003]</td><td>[5.814,16.221]</td><td>[7.602,17.533]</td></tr><tr><td colspan="5">...</td></tr><tr><td>13:00</td><td>83.072</td><td>[75.757,93.114]</td><td>[70.670,88.299]</td><td>[72.931,82.184]</td></tr><tr><td>14:00</td><td>87.347</td><td>[80.687,93.388]</td><td>[74.637,91.911]</td><td>[79.381,86.549]</td></tr><tr><td>15:00</td><td>91.136</td><td>[85.214,96.983]</td><td>[82.648,97.055]</td><td>[84.173,91.728]</td></tr><tr><td colspan="5">D<sub>2</sub></td></tr><tr><td>08:00</td><td>57.88</td><td>[13.798,334.950]</td><td>[0.002,415.508]</td><td>[13.798,334.950]</td></tr><tr><td>09:00</td><td>593.25</td><td>[362.070,891.557]</td><td>[228.263,1089.999]</td><td>[362.070,891.557]</td></tr><tr><td>10:00</td><td>1214</td><td>[920.289,1936.505]</td><td>[945.482,2044.318]</td><td>[920.289,1936.505]</td></tr><tr><td colspan="5">...</td></tr><tr><td>17:00</td><td>1193.5</td><td>[398.825,1204.242]</td><td>[317.962,1568.137]</td><td>[865.867,1175.649]</td></tr><tr><td>18:00</td><td>311.5</td><td>[285.082,1072.712]</td><td>[626.824,1146.699]</td><td>[409.353,995.631]</td></tr><tr><td>19:00</td><td>17.34</td><td>[16.044,151.493]</td><td>[0.085,100.161]</td><td>[2.045,163.101]</td></tr></tbody></table>

## 3.3 Case study 2

### 3.3.1 Proposed methodology

The purpose of this second case study is to present a new methodology for day ahead PVPF, where the Categorical boosting model is fused with Continuous Wavelet Transform and Bayesian Ridge Regression.
## Bayesian Regression

Bayesian modeling framework is based on Bayesian theorem [127]. It permits to receive a posterior distribution of model parameters using conditional likelihood and prior distribution [127]. Bayesian Ridge Regression (BRR) is found well suited to handle hierarchical data structure and multicollinearity. The BRR is seen as a derivative of the old posterior Bayes regression with variable prior distribution on the structure of the regression vector. It has been reported that Catboost has a better accuracy than XGB model in terms of execution time and memory limitations. For datasets with a large number of features, XGB can not run due to memory limitations, and Catboost converges to a good solution in the shortest time

## Continuous Wavelet Transform

Many solid theoretical and empirical results indicate that Wavelet Transform is considered one of the most efficient solutions for time-series analytics. Wavelet transform tools are proposed to overcome the limitations of Fourier Transform and short-time Fourier transform for signal processing. The WT empowers the multiresolution analysis in both the time and frequency domain. WT provides a superior grant time and frequency resolution for the whole signal. To manage non-stationary and nonlinear signals, The wavelet transformation is categorized into two families; specifically, Continuous WT (CWT) and Discrete WT (DWT) [128]. In CWT, the scale and transition are selected randomly. The continuous Wavelet $W(a, b)$ formulate of a wavelet dilation $a$ and time shifts $b$ is calculated as:

$$W(x, y) = \frac{1}{\sqrt{a}} \int_{-\infty}^{+\infty} s(t) \phi^*\left(\frac{t-b}{a}\right) dt \qquad (3.32)$$

With $\phi$ and $\phi^*$ indicate the mother wavelet and its intricate conjugate, respectively. $s(t)$ denotes the signal at a time step $t$ which is the PV generation in this situation. It merits referencing that the wavelet dilation adjusts the spread of wavelet and the time-shift wavelet computes the central position. There are various sorts of mother wavelets such as the Morlet Wavelet, Meyer Wavelet, and Mexican Hat Wavelet) [128]. According to DWT, the scales and transitions are controllable in terms the mother wavelet can be transformed conveniently. The CWT' mother wavelet $W(x, y)$ can be calculated as:

$$W(x, y) = 2^{-(x/2)} \sum_{t=0}^{T-1} s_t \phi^* \left( \frac{t - y^2 x}{2^x} \right) \qquad (3.33)$$

Where $x$ and $y$ denote the position and resolution level parameters, respectively. $T$ is the series length. CWT is frequently used either with Haar Wavelet or Daubechies Wavelet for a specific application. In our
case, the computational work is conducted through the PYWAVELETS library [128]. For the PV power application with CWT, Morlet Wavelet $\psi_{a,b}(t)$ for t timestep has been chosen with its parametric equation can be gotten as:

$$ \psi_{a,b}(t) = e^{-\frac{t^2}{2}} \cos(5t) \qquad (3.34) $$

Since the accuracy of the CTW and DWT is nearly similar, the choice of CWT has been made based on its superior simplicity [128]. The CWT utilizes three levels of decomposition for the PV power output. The number of levels is concluded based on trial and error simulations.

## Catboost

classical gradient boosting also suffers from overfitting caused by the prediction shift in the learned model, also known as a special kind of target leakage [88]. Furthermore, categorical features with discrete sets of values that are not necessarily comparable to each other cannot be directly handled by binary trees [2]. A popular method to incorporate categorical features in GBDT method consists of transforming them to numerical values. However, this conversion process often leads to target leakage and prediction shifts. To remedy this issue, CatBoost was introduced as a fusion of OGB and ordered target statics. The CatBoost model uses a more efficient strategy that reduces over-fitting and allows the use of the whole dataset for training. Namely, they perform a random permutation of the dataset, and the average label value for each example with the same category value placed before the given one in the permutation is computed.

Several studies prove the high efficiency compared to the Gradient Boosted Decision Trees (GBDT) models in terms of computational intensity and complexity restraints [129]. For high dimensional data representation, Catboost provides a fast convergence to the optimal candidate solution compared to the reference models [129].

## Proposed model

The arbitrary power outputs in PV power generation reflects the an urgent demand for precise PVPF for day-ahead production scheduling. The proposed framework consists of multiple prepossessing blocks with different functionalities. The loop body of the proposed architecture is presented in Fig.3.18.

According to Fig. 3.18, the proposed framework is conducted in three basic stages. In the 1st stage, the information is collected and cleaned to eliminate outliers and missing records. The 2nd stage lies on the feature significance' assessment by the BRR algorithm. The wavelet transformation provides the time-frequency information to the data representation. The predictions produced by Catboost model are linked to Inverse CWT. In the 3rd stage, the forecasting performance of the proposed model is assessed using CV,
<figure><img src="image_33.png" /><figcaption>Figure 3.18: Proposed model architecture [3].</figcaption></figure>

error criteria, and simulation graphs.

### 3.3.2 Simulation results

#### Exploratory data analysis and preprocessing

In this case scenario, the DKASC data is used for verification [90]. Obviously, weather sensors may measure erroneous data due to various reasons such as periodic maintenance and dust. The misleading information generated from a faulty measurement has a drastic impact on the forecasting model in the sense that it concludes with erroneous assumptions. In order to overcome this issue, the Not A Number (NAN) values and the outliers are removed from the rows data. For the sake of ensuring intrinsic information with high quality, the data passes with an extensive cleaning process to remove the undesired samples. The score metrics scores deployed involve the $R^2$, RMSE, MAE, and MdAE [90].

#### Numerical modeling

The input values are arranged by their significance utilizing Bayesian Ridge Regression (BRR) model. The BRR model attributes weights to each variable as per their impact on the domain knowledge. According to Fig. 3.19, The horizontal irradiation and the previous PV power are the most significant attributes since they generate positive weights values with the predominance of the horizontal irradiation. The simulation results are computed using 248503 rows data by a LENOVO Ideapad L340 with the only positive values with the superiority of the horizontal irradiation. Fig. 3.20 presents the curve shaping of one-day PVPF in normal weather states.

Moreover, Table. 3.16 introduce the resume of the core calculation formula of score metrics.
<figure><img src="image_34.png" /><figcaption>Figure 3.19: Assessment of variable contributions using BRR [3].</figcaption></figure>

<table><caption>Table 3.16: Evaluation scores [3].</caption><thead><tr><th>Score metrics</th><th>MSE</th><th>RMSE</th><th>MAE</th><th>MdAE</th><th>R<sup>2</sup>(%)</th><th>Maximum error (kW)</th></tr></thead><tbody><tr><td>Values</td><td>14.55</td><td>3.81</td><td>2</td><td>0.1</td><td>0.99</td><td>18.51</td></tr></tbody></table>

From Fig. 3.20, the curves harmonization leads to conclude that the hybrid model is well-suited for PVPF. The shape of the forecasted points curve follows the actual one with a small variation from the 150-160 time step. Regarding Table. 3.16, the RMSE and the MSE values are 3.81 and 14.55 respectively which is reasonable for real-world energy operations. Further, the squared R-value is 0.99 which reflects the high model' effectiveness. Fig. 4.5 represent the 10-fold cross-validation (CV) curve of the proposed method.

Moreover, the precise values of R² are concluded in Table 3.17.

<table><caption>Table 3.17: Evaluation scores [3].</caption><thead><tr><th>Fold number</th><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>6</th><th>7</th><th>8</th><th>9</th><th>10</th></tr></thead><tbody><tr><td>R<sup>2</sup>(%)</td><td>98.7</td><td>99.2</td><td>95.33</td><td>90.23</td><td>95.07</td><td>98.51</td><td>99.28</td><td>98.85</td><td>99.51</td><td>99.53</td></tr></tbody></table>

Concerning Table 3.17, the validation process from a 10 fold CV shows that the precision generally diminishes during sunlight production. The squared R records a lower value of 90.23% with a 9% decrease
<figure><img src="image_35.png" /><figcaption>Figure 3.20: Real and predicted PV power output in steady weather conditions [3].</figcaption></figure>

compared to Table 3.16 from the previous simulation. Notwithstanding these reductions, the model is still well-performing since all the folds have an R% threshold of 90.23%. In this manner, the proposed model affirms its high adequacy on consistent weather disturbance. Nevertheless, in extreme weather situations, forecasting models face a relatively challenging task to cope with sudden variations for the majority of ML models. The CWT model evaluation against the PV power high frequencies reflects its role in improving the accuracy results. The assessment of the proposed model is given in Fig. 3.22 with quantitative results in Table 3.18.

<table><caption>Table 3.18: Evaluation scores [3].</caption><thead><tr><th>Score metrics</th><th>MSE</th><th>RMSE</th><th>MAE</th><th>MdAE</th><th>R<sup>2</sup>(%)</th><th>Maximum error (kW)</th></tr></thead><tbody><tr><td>Values</td><td>15.29</td><td>3.91</td><td>2.01</td><td>0.19</td><td>0.99</td><td>23.28</td></tr></tbody></table>
<figure><img src="image_36.png" /><figcaption>Figure 3.21: 10-fold CV for the proposed model [3].</figcaption></figure>

<figure><img src="image_37.png" /><figcaption>Figure 3.22: Real and predicted PV power output in unsteady weather conditions [3].</figcaption></figure>

According to Fig. 3.22, the effectiveness of the proposed technique does not highly been influenced due to the fact that the WT decomposition enhances the quality of the data. The RMSE increases from 14.55kW to 15.29kW, and the Maximum error jumps to 23.28 kW compared with 18.51kW. The MdAE value has reached 0.19kW. Notwithstanding, this performance enhancement, the model still resists the unexpected
extreme variation and random climate conditions.

## 3.4 Summary

Increasingly high penetration level of PV generation arises in SG. The forecasting of the high intermittency and randomness of PV energy in SG is a persisting challenge. To tackle energy management under uncertainty, two efficient methods have been proposed for this chapter. Firstly, an efficient method based on the combination of NARXNN and LSTM is optimized by TSA. Numerical results show that the proposed approach achieves better prediction accuracy than the traditional methods. Moreover, the NARX-LSTM model showed excellent performance and good generalization capabilities from two different locations and utility scales—secondly, a novel predictive model based on Bayesian Ridge Regression, CWT, and Catboost models. CWT is employed to enhance the feature representation of the original data. Case studies based on the measured datasets from utility-scale PV-power systems from the US and Australia have been used to validate the proposed models' efficiency. To further extend the energy management under uncertainty, the demand side has been given a particular focus in the next chapter to stabilize the PV energy-based supply system and the energy consumption side.

The main content of this chapter has been published in [2], [3], [25], [130], [131], and [76].
# Chapter 4

## Control management and optimization

### 4.1 Introduction

The transition to a climate-neutral world with large-scale RES deployment, such as PV farms, has heavily influenced the users' power consumption behavior and electricity generating and distribution companies. Further, the steady growth in energy consumption leads to uncertain and volatile load demand. The demand profiling is essentially driven by the meteorological factors, industrial and socio-economic and demographic elements, and contextual variables such as customer behavior [132]. Thus, a precise and trustworthy load forecast is a guarantee to keep the balance between power generation (PV power) and demand. Furthermore, it helps in the correct and efficient scheduling of power system operations [133].

As a part of power planning, load forecasting plays a fundamental role in achieving safe operation and scientific management of the system. Accurate load forecasting is a basic guarantee for improving the utilization of generation equipment and the effectiveness of economic dispatch. Furthermore, it has important significance for the optimal combination of units, economic dispatch, utility asset management, grid stability, etc. According to the forecasting horizon, the significant benefits for the grid utility from the short-term load forecasting (STLF) are accorded to hourly predictions since it directly affects the load profile and yields ultimately to more economic profits [134]. Automated and accurate STLF is essential for daily dispatching of the power grid. However, the non-stationary characteristics of the load series add to the challenge of this task.

Alongside load forecasting, grid stability assessment present the key solution to mitigating PV fluctuations with a set of control management strategies. PGS can be classified into voltage stability, transient stability, frequency stability, rotor angle stability, resonance stability, and converter-driven stability [135]. Voltage stability refers to the electric system's capability to supply the system's reactive power close to nominal value at all buses in the system after facing unexpected perturbation. Transient stability is the ability of synchronous machines, when subjected to a critical collapse, such as islanding, and grid faults, to maintain in its synchronism state [135]. Frequency stability is concerned with an electric system's capability to conserve
its normal frequency behavior irrespective of the existing disturbance. Rotor angle stability lies in the EPS's potential to conserve synchronism under small disturbances [135]. Converter-driven stability refers to system-wide stability problems driven by fast dynamic interactions of power electronics-based systems with slow- and fast-response components. Resonance stability encompasses Sub-Synchronous Resonance (SSR), whether associated with an electromechanical resonance or an entirely electrical resonance [135]. In the smart grid era, the power system stability can be carried out by ample energy storage to take care of any eventuality. Fig. 4.1 presents the common machine learning applications to stability analysis and control. These applications are categorized based on stability control fault diagnosis, stability assessment, and security assessment.

<figure><img src="image_38.png" /><figcaption>Figure 4.1: AI applications for power grid stability.</figcaption></figure>

This chapter will present a set of efficient tools for control management and optimization of PV power flow without compromising the grid stability and reliability. These tools are short-term load forecasting and PGS assessment and classification. The proposed methodologies lie in advanced machine learning methods that employ real-world databases for training.

The chapter is organized as follows: In section 4.2, load forecasting is tackled. Furthermore, two novel short-term load forecasting methodologies have been described. Section 4.3 triggers the PGS between the PV generation and the load consumption. Two efficient computing frameworks are firstly depicted to provide a convenient route for PGS evaluation. Extensive simulation results and comparison analyses are conducted to validate the performance superiority of the proposed approaches. Finally, the chapter summary is provided in Section 4.4.
## 4.2 Load demand forecasting

In this subsection, load forecasting has been tackled using two case studies and multiple testing scenarios, as shown in fig4.2. The proposed models are dedicated to both small and large data sizes. The first proposed model takes advantage of the enormous capabilities of DL by implementing convolutional bidirectional LSTM model and Savitzky-Golay filter for data smoothing. The second model uses MLP, XGB, and LGBM to build a solid and efficient forecasting model for STLF.

<figure><img src="image_39.png" /><figcaption>Figure 4.2: Load demand forecasting procedure.</figcaption></figure>

### 4.2.1 Case study 1

The proposed SG-CBilSTM is composed by a virtue of processing units characterized in three principal stages: data engineering and preprocessing, object determination, model construction, and model assessment as represented in Fig. 4.30.
<figure><img src="image_40.png" /><figcaption>Figure 4.3: The adopted methodology for the proposed SG-CBiLSTM model [4].</figcaption></figure>

Regarding Fig.4.30, the proposed methodology is led utilizing four fundamental stages: (1) data processing and feature engineering, object determination, model construction, and evaluation stages. For additional clarification of the proposed strategy, a complete flow chart is represented in Fig. 4.4.

<figure><img src="image_41.png" /><figcaption>Figure 4.4: High-level flowchart of SG-CBiLSTM algorithm [4].</figcaption></figure>
According to Fig. 4.4, the SG-CBiLSTM framework is introduced as follows: The data is gained from a publicly released web source and cleaned from erroneous values in the data preprocessing and feature engineering stage. At that point, a data augmentation technique is adopted to improve the meaning of the information and enhance the learning capacity of the entire framework. Moreover, data transformation s led to transform all the data to its numeric form using the on-hot encoder function. Then, data smoothing-based SG filter and data normalization were carried out to eliminate the noise from the load demand and normalize the data under an interval [0,1]. At this level, the SG filter can dismiss the noise proficiently alongside minimal distortion from the real load information. All the system inputs are given likelihood values to assess their relevance to the load demand determination. Over a particular edge, the features are kept, and the rest are ignored. Thereafter, the object determination stage split the data into training, validation, and testing intervals. The HO is mandatory for CBiLSTM model to provide more exact outcomes. The last stage is the evaluation of the model. In this stage, the point forecast assessment is conducted utilizing error measurements. Besides, K-fold cross-validation technique is utilized in this investigation as a reliable way to assess the proposed model exactness to an obscure testing information [136]. K-fold cross-validation schemes are frequently used in complex problems [3, 25]. The methodology of K-fold cross-validation is outlined in Fig. 4.5.

<figure><img src="image_42.png" /><figcaption>Figure 4.5: 10-fold cross-validation methodology [4].</figcaption></figure>

According to Fig. 4.5, K-fold cross-validation methodology is led as follows. First and foremost, k equal-sized sub-samples are paralleled out the true data set. 10-fold cross-validation (10-CV) is widely used to demonstrate the model effectiveness with cross-validated data. The ten parts evaluate the model generalization to an independent set. For each 10-CV round, a random data separation into *k* folds {D<sub>1</sub>, D<sub>2</sub>, ..., D<sub>k</sub>} is built. The proposed model is repeatedly trained by nine training set folds and tested against the remaining testing *k* − 1 set fold. The holdout method is repeated ten times. In this regard, the bias and variance are
remarkably reduced due to the use of the original data set in the fitting and validation procedure at the same time. For checking the model competitiveness, the proposed SG-CBILSTM is compared to a bench of DL models to approve its exhibition predominance. The execution details of each part are provided in the next section.

## Simulation results

In this part, the demonstration of the proposed SG-CBILSTM for STLF is exhibited through extensive experimental studies with an open-source dataset. The assessment system incorporates dataset description, model construction stages, error measurements, and benchmarks examination investigation. The simulation configuration mainly utilizes the ISO-New England incorporation (ISO-NE) dataset [137]. The ISO-NE is non-benefit incorporation to manage everyday tasks with more than 200 market members in New England [137]. The freely accessible information data covers the hourly temperature and load data for the public and private electricity market of New England from 2003-2014, as shown in Fig. 4.6.

<figure><img src="image_43.png" /><figcaption>Figure 4.6: Load and temperature variation between 2003-2011 (103753 instances) for the ISO-NE data set [4].</figcaption></figure>

The gathered information comprise 103753 rows. The used data is cleaned from erroneous and Not a Number (NAN) samples. The cleaning stage is obligatory to feed the model with complete data samples and remove any data anomalies or deceiving values. Nonetheless, the hourly temperature for the ISO-NE
data is the only indicator of the load demand. This data is insufficient for accurate prediction and may lead to underfitting. Consequently, data augmentation is led to distinguish and produce artificial features using Microsoft Excel tool as demonstrated in Table 4.1.

<table><caption>Table 4.1: Generated feature vectors [4].</caption><thead><tr><th>Data fold</th><th>Augmented data</th><th>Formula</th><th>Unit/range</th></tr></thead><tbody><tr><td rowspan="2">Load forecast</td><td>Load(t-1), ..., Load(t-23)</td><td>L<sub>(t-1)</sub>, ..., L<sub>(t-23)</sub></td><td>MW</td></tr><tr><td>Load demand</td><td>L<sub>t</sub></td><td>MW</td></tr><tr><td rowspan="10">Datetime</td><td>Year</td><td>Y<sub>t</sub> = YEAR(Datetime)</td><td>[2003,2014]</td></tr><tr><td>Month</td><td>M<sub>t</sub> = MONTH(Datetime)</td><td>[1,12]</td></tr><tr><td>Day</td><td>D=DAY(Datetime)</td><td>[1,31]</td></tr><tr><td>Weekday</td><td>W<sub>d</sub> = WORKDAY(Datetime)</td><td>[1,7]</td></tr><tr><td>Hour</td><td>H<sub>t</sub> = HOUR(Datetime)</td><td>[1,24]</td></tr><tr><td>Season of the year</td><td>S=SEASON(Datetime)</td><td>[0, 3]</td></tr><tr><td>Week number</td><td>W<sub>n</sub> = WEEKNUMBER(Datetime)</td><td>[1,52]</td></tr><tr><td>Weekend</td><td>W<sub>n</sub>d = IF(W<sub>d</sub> > 5 Then Yes, Else, No)</td><td>Boolean</td></tr><tr><td>Weather data</td><td>Temperature</td><td>T<sub>t</sub></td><td>°C</td></tr></tbody></table>

As per Table 4.1, the total features are the DateTime components and hourly lagged load for the last twenty-three hours. The Boolean features are transformed to numerical values using hot encoder function. At that point, the load demand is smoothed via SG filter to improve the pattern recognition of the forecasting system. The cleaned information is sent to the feature engineering phase. Four case studies were examined according to the value of the polynomial function specifically three, five, seven, and ten. The load demand and its smoothed derivations are shown in Fig. 4.7.
<figure><img src="image_44.png" /><figcaption>Figure 4.7: Comparative results of SGCBiLSTM model with different polynomial functions [4].</figcaption></figure>

It is apparent from Fig. 4.7 that the increment of the probability values (P-values) results in a more smoothed signal. The created signals are evaluated together in similar conditions to select the optimum SG polynomial value based on the less error generated. Moreover, the data standardization is carried out using the Min-Max Scaler function from Scikit-Learn [138]. The normalized data from zero to one to keep the values within limits of the activation function and to lessen the impact of various units on the prediction system.

The analysis of the most significant features in the data set is a fundamental stage for a better explainability of framework execution. This procedure exploits the P-values for each feature to map the domain representation and assess the weights for each feature input. Thus, the irrelevant and redundant inputs are discarded from the feature vectors. These artificial features are associated in tandem with the temperature to make a prediction. The data augmentation technique is commonly utilized for low-dimensional frameworks to follow the irregularity and trends of the target variation and acquire more information about domain knowledge. The ranking of the feature is fundamental at this stage to make the framework more proficient with less computational time due to the high number of features. The PLS technique strategy is utilized to diminish the data dimension. PLS is a straightforward dimensionality reduction technique that maps the variables in new feature space with lower dimensions. The Variable Importance of consumption Patterns (VIP) for thirty-two features is illustrated in Fig. 4.8.
<figure><img src="image_45.png" /><figcaption>Figure 4.8: Variable importance of load patterns with a threshold value (red line) equal to VIP=0.5 [4].</figcaption></figure>

Regarding Fig. 4.8, the most important features are hour, workday, temperature and lagged load ($t-x$) with $x \in [1, 2, 3, 4, 5, 6, 7, 11, 12, 13, 17, 18, 19, 20, 21, 22, 23]$. Thus, the selected threshold is VIP=0.5. The rest of the simulation results adopted a feature representation including 20 feature inputs. The data is split into three sets: training, validation, and testing sets with a proportion of 75%, 15%, and 10%, respectively. More precisely, the training, validation, and testing instances comprise 77814, 15562, and 10376 samples, respectively.

## Model construction and parameter settings

In this part, the assessment of the proposed SG-CBiLSTM model is reported. The simulation results were conducted using PYTHON programming, and Keras library [139]. Keras is a high-level framework specialized in DL. All simulations were run on a Lenovo Intel ® i7 ® Nvidia Geforce GTX 1650@ 2.30 GHz. The simulation analysis-based on trial and error method is firstly adopted to limit the solution space as much as possible for the optimization process. This method is conducted by selecting arbitrary extreme
values of BiLSTM and CNN layers and measuring the accuracy of the forecasting engine in terms of R² score. Next, new random values are defined to decrease the possibilities of hyperparameters values further. The final results of the trial and error method show that three convolutional layers with a kernel size equal to 1 × 1 associated with three successive BiLSTM layers achieve better results according to the highest the R² criteria. For the rest of the optimization work, ES is employed to tune the critical hyperparameters of the proposed computing framework, specifically, the batch size, the number of units, the number of training epochs [139,140]. To tune these hyperparameters, Hyperactive library is employed on open source [119]. It is worth noting that the search space of the ES method only considers the key hyperparameters of the proposed C-BiLSTM to reduce the number of combinations and thus accelerate the computational time. In the model construction stage, the solution space of the proposed CBiLSTM model is resumed in Table 4.2.

<table><caption>Table 4.2: hyperparameter settings of the proposed model [4].</caption><thead><tr><th>Model</th><th>Configuration/Neurons/Filters</th><th>Search space</th><th>ES</th></tr></thead><tbody><tr><td rowspan="3">CNN</td><td>CNN Layer1</td><td>[32,64,72,128]</td><td>32</td></tr><tr><td>CNN layer 2</td><td>[32,64,128]</td><td>64</td></tr><tr><td>CNN layer 3</td><td>[32,64,128]</td><td>64</td></tr><tr><td rowspan="5">BiLSTM</td><td>LSTM layer 1</td><td>[100,150,250]</td><td>150</td></tr><tr><td>LSTM layer 2</td><td>[32,50,150,250]</td><td>50</td></tr><tr><td>LSTM layer 3</td><td>[32,50,150,250]</td><td>32</td></tr><tr><td>Batch size</td><td>[32,64,72,256,512]</td><td>512</td></tr><tr><td>Iterations number</td><td>[20,30,70,100]</td><td>100</td></tr><tr><td colspan="2">Time(min)</td><td></td><td>131.64</td></tr><tr><td colspan="2">Time/iteration (min)</td><td></td><td>8.77</td></tr></tbody></table>

The best hyperparameters are selected as follows. Firstly, the population of ES is randomly initialized. Each candidate solution represents a possible configuration of the optimized CBiLSTM model. From the data training, the CBiLSTM model is trained with a specific structure and parameter settings included in Table 4.2. Finally, the testing set is used to evaluate each population's performance based on MAPE. Regarding the ES performance in Table 4.2, the ES algorithm is characterized by a fast convergence speed with 8.77min for every iteration. From Table 4.2, the input layer consists of a three successive one-dimensional Convolutional layer (Conv1D), with 32,64 and 64 filters of the convolution, respectively. The Conv1D layers are associated with a ReLU activation function. The ReLu function is employed to solve the vanishing