# Fault Prediction and Reconfiguration Optimization in Smart Grids: AI-Driven Approach
## 3 Data Source Search, Analysis, Purge, and Selection
### SustDataED--Dataset Shortcomings  
Since the literature on SustDataED did not provide clarity on household-level photovoltaic energy production measurements, an alternative was sought to form a new production dataset. To perform this production data simulation, two tools were chosen in order to compare and ensure that the results obtained from them were consistent:  
* Global Solar Atlas [38]: this platform is funded by the Energy Sector Management Assistance Program (ESMAP) with the aim of mapping renewable energy resources globally, providing access to both long-term averaged data and real-time data for any location on Earth. * PVWatts [39]: this platform is provided by the National Renewable Energy Laboratory (NREL), which is the national laboratory of the U.S. Department of Energy. \begin{table}
\begin{tabular}{c c} \hline \hline
**Type of Measurement** & **Utility** \\ \hline Energy consumption measurements & ✓ \\ \hline Energy production measurements & **?** \\ \hline Demographic measurements & ✓ \\ \hline Environmental and climatic conditions measurements & ✓ \\ \hline User event measurements & X \\ \hline Power event measurements & X \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Types of measurements in SustDataED dataset. Utility: useful (✓), useless (✓), questionable (7). Figure 1: Time series of measurements of the SustDataED dataset. For each tool, the location was set to Funchal (Portugal), since the analysis should correspond to the data collected in the SustDataED dataset. Once the location was established, the photovoltaic system for each household was configured, and the relevant data layers extracted like the Direct Normal Irradiation (DNI) (\(\frac{\text{KWh}}{\text{m}^{2}}\)) and Photovoltaic Power Potential (PVOUT) (\(\frac{\text{KWh}}{\text{KWh}}\)), which are key parameters for estimating the power output of photovoltaic systems. Additionally, climatic factors play a crucial role in verifying the consistency of the generated power. According to the [PERSON] and Trewartha's climate classifications, Funchal features a Mediterranean climate with oceanic influences or a temperate climate with dry summers. Located in a subtropical zone, the city experiences minimal daily temperature variations, leading to relatively stable temperatures throughout the year. After configuring both systems, the dates corresponding to the SustDataED time series were aligned to collect photovoltaic production data. In both tools, as shown in Figure 2, a comparative analysis of the same parameters, DNI and PVOUT, was conducted for the same time periods. The results, as illustrated in Figure 2, were found to be nearly identical. Following this verification of consistency between the two tools for simulating photovoltaic production values, PVWatts was selected due to its more streamlined export capabilities, facilitating the integration with the SustDataED data for the construction of the final dataset. In addition, a comprehensive correlation analysis was performed on the photovoltaic power generation parameters for both SustDataED and PVWatts datasets to conclusively evaluate the reliability of the energy production data. When examining the correlation between the climatic variables and the power generation for both datasets, as shown in Figure 3, distinct patterns emerged. For SustDataED (Figure 3a), it was evident that the temperature exhibited a weak correlation with the generated power, yielding a coefficient of only 0.279. This lack of strong correlation was consistent across other variables, making it difficult to analyze energy production in relation to climatic conditions. In contrast, for the PVWatts dataset (Figure 3b), most parameters showed a high degree of correlation, approaching unity. For instance, the correlation between array plane radiation and power generation was nearly ideal, with a coefficient of 0.999, as clearly shown in the graph. Figure 2: Comparison of Global Solar Atlas and PVWatts, total monthly values of photovoltaic energy production (PVOUT) versus normal direct irradiation (DNI).