dataesr/scientific-paragraphs-categorization huggingface dataset:
| EDA of the power time series datasets To assess the meaning, accuracy, and robustness of the calculated results for a particular dataset, it is useful to determine during initial exploratory data analysis (EDA), the appropriate statistical measures of PV dataset quality. This provides insights into which datasets are robust to analysis, and which datasets may fail at particular steps or for certain types of analysis, such as the case where different aspects of data missingness makes analysis impossible. 48, 49 In addition these measures can guide the user on the expected uncertainties in comparing multiple PV systems, which may have quite different equipment and operational histories, and therefore dataset quality. From time series analysis of building electricity time series data to perform virtual building energy audits, a time series dataset grading schema has been developed which has proven useful to alert users to expect high, or low, quality of results of data analysis of the systems.
50 We have adapted this approach for application to PV system time series datasets such as the power and irradiance time series and have implemented PV dataset quality grading in the PVplr package. We statistically characterize the power time series and then grade each dataset in three areas, outliers, missing datapoints, and data gaps. We have developed a grading schema, summarized in Table  2 , and the measures and grades are summarized in Table  3. Outliers are typically defined as points that are greater than ±1. 5 times the IQR and may be anomalous datapoints. 48 In time series, such as PV time series, outliers can arise from causes that fall beyond the expectations of a model, so, for example, for the power generated by a PV system, if there were no clouds, then a linear second-order model can fit the daily and seasonal changes in power production.
51 But clouds, being statistically random, are not easily modeled, and the power drops due to cloud shading would be outlier datapoints, while still physically meaningful. Anomalies are a subset of outliers, and correspond to datapoints that are not physically reasonable, but arise due to a mistaken measurement or malfunction of a piece of equipment. 52 For outlier detection we use the tsoutliers R package, which identifies time series outliers arising from clouds and anomalous datapoints such as arise from measurement errors. 53, 54 rious of the filtering and correction methods discussed here are examples of approaches to address dataset outliers. Missing datapoints in a dataset is another typical data error that can impact analysis. A set of up to 5 sequential missing datapoints can be imputed rather easily using simple interpolation, so we consider the % missing datapoints the second important dataset characteristic to identify, and if desired, to impute.
Longer data gaps in a time series dataset can arise from system or communications outages, can be quite problematic for different analysis methods, and are hard to correct or impute in an attempt to mitigate their impact, so we consider this the third important characteristic. These three categories of statistical properties give us a quantitative sense of the "missingness" of the dataset, and learning which filters, and methods are robust in the face of outliers, missing datapoints and data gaps, is important to advance the field. There is much active work on data imputation to address both outliers, missing datapoints and data gaps, but we have not implemented these here. Note: Outliers include the impact of clouds and anomalous datapoints; missing data is 5 or fewer sequential datapoints; and the longest gap is of all the data gaps in the dataset.
| Dataset quality issues Various dataset quality issues are present in the selection of PV system power time series, as can be seen in Table  3 , and are discussed here. • EURAC system: No major data quality issues have been detected. This system has more than 10% outliers, probably due to cloudiness. • FOSS system: No major data quality issues have been detected. This system had a higher amount of missing datapoints. • RSE systems: No major data quality issues have been detected. These systems had a 25-day-long gap in the datasets. • Pfaffstaetten systems: A relatively low amount of measured data has been reported. This can be seen in the missing % and the # of datagaps. • US DOE RTC baseline systems: Several data quality problems have been detected, which resulted in some filter-metric-methods to be unable to calculate sensible PLR results for some methodologies.
| Uncertainty contributions to reported PLR results Uncertainties in the final reported PLR result arise from multiple contributions, including measurement and sampling uncertainties. The power, temperature, and irradiance time series datasets which are used as the input to PLR determination, are a sample of a real-world PV system. Consider two different samples of this system, a 1-min interval and a 5-min interval time series, acquired over the same time period, but with different instruments. These two time series, measured by different instruments, would exhibit different standard deviations (σ meas ) due to the instrumentation's characteristics. If instead we use two identical measurement instruments, each measuring the system over the same time period, for example, by lagging on measurement by 30 s, these should have the same measurement uncertainty σ meas , but two independent samples, their sampling uncertainties will vary when one considers the population mean μ of the PLR of the PV system, and will have sampling standard deviations (σ samp ) that are also different.
Of course, each dataset is a sample of the system and contains contributions to their standard deviation arising from measurement and sampling uncertainty, intrinsically. The easiest way to determine the dataset standard deviation s of the time series is by seasonal decomposition, into its seasonal, trend, and residual components, and the sampling standard deviation is easily calculated from the time series residuals. The PVplr package reports the dataset standard deviation, as a % of initial power, as one of its measures of PV data quality. 22  many fields of research, it has long been reported that researchers do not distinguish the descriptive statistics of the standard deviation of a set of measurements and the standard error of the mean. 56 As a descriptive statistic the standard deviation is a measure of the variability of a set of measurements, arising from instrument and sampling effects.
The standard error of the mean provides an estimate of our uncertainty in the "voted" value of a population mean, which in this study is supposed to be the true PLR value of the realworld PV system. 57 In this study we are benchmarking which filtermetric-method approach can determine the "true" voted value for a system. Researchers using common approaches to PLR determination, such as in the RdTools or PVplr packages, or using their own methods, are producing results which should replicate the "true" PLR of the system. We are benchmarking methods applied to diverse datasets, to identify the most robust approaches to determine the replication means for these 19 systems. 58 Typically replication studies attempt exactly the same method, and the standard error of the mean measures the variance among the attempts, to determine the population mean value of PLR.
This is the basis of the "voting" method we apply here so that we can determine the most likely mean value of PLR for the 19 real-world PV systems, but utilizing many data filtering and statistical modeling methods and for each given dataset, the mean PLR value is probably the most likely. 59 And the standard error of the mean is a measure of the variance of these methods in determining the mean PLR. For comparing multiple methods of PLR determination for a single system, we can compare the different results and their 95% confidence interval (CI). With this, we will find the true population PLR mean, with in range of the 95% CI 19 out of 20 times. To determine if different methods show statistically similar, or different, estimates of the PV system's true PLR, we can check that the 95 % CIs are overlapping.
The PLR of the digital system with degradation and real weather data was investigated next to the EURAC system. Figure  4  shows an example power versus irradiance plot for Filter #3 applied to the EURAC system. The power along the y-axis has been normalized to the nominal power of the system and a POA irradiance interval from 0 to 1250 W/m 2 is depicted along the x-axis. The blue dots represent the raw data, the green ones a first threshold filter, and the red points the final filtered dataset which is used for the subsequent PLR calculation steps. Some filters only include a threshold filter. In this case, the inbetween filter step has been omitted, and just the final filtered data are shown. Figure  5  shows the calculated PLR values of the digital plant in dependence on the used filter together with power versus POA irradiance plots in order to get an idea of the impact of the individual filter.
| Digital plant with degradation and real weather data Looking at Figure  5 , it is clear that the choice of filter does affect the PLR to a large extent. The calculated PLR values range from -4. 48%/a to -5. 47%/a. It seems that filters with similar irradiance cutoff thresholds are clustered together when using the same metric and calculation method. This correlation is especially pronounced for the datasets of the digital plants which are, due to their nature, not subject to outliers, and the outlier grade or filtering (mainly PR related) does not have any effect on the outcome. • Filter #1: 33. 7% • Filter #2: 61. 1% • Filter #4: 2. 7% • Filter #7: 0. 4% Filter #1 already applies a strict irradiance threshold at 500 W/ m 2 , but the data within the considered irradiance interval, although just being 33.
7% of the total amount of data, still account for roughly 80% of the produced power by the EURAC system and can therefor be considered as being representative. Instead, a vast amount of data is excluded in the PLR calculation using Filter #4 and #7. Although the methodologies perform well on the example dataset above and on some of the high-quality datasets in Section 4. 6. 2, it is believed that such a small amount of remaining data (2. 7% and 0. 4%) does possibly not represent the overall performance evolution well. Furthermore, depending on the location, such strict irradiance thresholds might reduce the amount of available data even further. Instead, given the used metric and statistical method, a narrow powerirradiance interval seems to be the filter of choice for real datasets including outlier.
| Benchmarking PLR digital datasets The digital datasets have been introduced along with the real PV plants in Section 2. Table  4  shows the most important characteristics of the plants. The individual methods used in this comparison have been introduced in Table  1. Figure  7  shows the calculation results for the two digital plants without degradation. The colors of the dots indicate the chosen metrics and the symbols the applied statistical models. Two horizontal lines are visible in the figures, the orange one representing the degradation value for P mpp at STC, and the green one the energy degradation. For this systems with no induced degradation, these values are very similar to one another. Looking at the results, it is obvious that the calculation accuracy varies depending on the source of weather data.
While all used approaches yield the correct value within a ±0. 05% interval for the system with repeating weather data, greater deviations can be detected looking at the figure where real weather data with an induced colder year were used. Especially, the last induced colder year in the weather dataset seems to bias the results towards negative values. This colder year, subject to artificially induced lower radiation, yields just 63% of the initial power output for this system, without being subject to degradation. Methods using high irradiance threshold filter (VAR1, R-LR1, LS-LR1, and STL1) or predictive power approaches as metrics (i. e. , XbX(UTC)) seem to be able to remove this induced effect and provide reliable results. For the two systems without degradation, the methodologies VAR1, LS-LR5, STL4, R-LR1, LS-LR1, YoY5, and STL1 yield the most accurate results with average deviations below -0.
025%/a. For the systems with induced degradation, the application of YoY3, STL4, YoY5, and YoY2 returns results with deviations lower than -0. 05%/a compared to the energy related PLR. It is believed that a threshold filter for irradiance and power affect the final PLR calculation results in a way such that higher thresholds yield lower PLR values. All four systems have very high-quality data where the effect of filtering irradiance-power pairs seems secondary in the sense that the main purpose of these filters, namely, outlier removal, is unnecessary as the raw datasets do not have any outlying values. An evaluation of the overall results for real datasets is fairly complicated since the true value of the PLR for each respective system is unknown. To rate the methodologies among each other, we used a replication study approach, where multiple filter-metric-method approaches to PLR were used to determine the PLR values and comparing this sample of PLR values as a sample of the true population mean of the PV system.
SCSF1 practitioner's analysis suggested a positive sensor drift for the EURAC POA pyranometer accounts for the difference of their reported result from the mean result (a difference of +0. 47%/a). Combining the suggested sensor drift with the estimation of degradation for the system, the deviance of the results using SCSF1 is reduced compared to the mean and yields a value similar to the majority of other methods. The difference from the normalized mean PLR was subsequently calculated for all systems, and the values were averaged to see which methodologies seemingly yield the highest accuracy by cross-comparison. Here, two key performance indicators (KPIs) were then identified to benchmark the proposed combinations for the calculation of PLR. • Absolute average deviation from the mean value considering all datasets = The first KPI provides an indication of how a particular filtermetric-method performs overall in terms of estimating the average value over all considered datasets.
Thereby, the absolute average of the differences between the mean PLR ( PLR i ) and the PLR for each statistical method is calculated, where i refers to a specific PV system of the n PV systems being analyzed. The second KPI provides an indication on how the average value deviates from dataset to dataset. P n i = 1 ðPLR i -PLR i Þ PLRi n : ð9Þ • Standard error of the average deviation = ffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi X n i = 1 ðPLR i -PLR i Þ PLR i -μ 2 n -1 v u u u t = ffiffiffi n p : ð10Þ F I G U R E 8 (A) Here, μ is the mean of the numerator of Equation (  9 ) over all systems for one filter-metric-method. Finally, all results are averaged in a target plot to see which methodologies perform the best across all systems.
The results are discussed in Section 4. 6. 2. Figure  9  shows, in addition to the absolute PLR values, the uncertainties reported by the analysts for each applied methodology, which unfortunately were a variety of standard deviations, standard errors, and CIs, so are not actually comparable. Since there is no consensus on how to report PLR uncertainty values in the PV community, the analyst reported uncertainties were omitted in the final evaluation. In Table  5 , the mean PLR for all systems are depicted to get an overall impression of the degradation of the systems under evaluation. The calculated PLR for all systems considering all methodologies can be found in Appendix C1. Unfortunately, some of the datasets under investigation had certain dataset quality issues, as discussed in Section 4.
| PLR analysis Based on the discussion above the PLR evaluation is subdivided into different groups considering a varying number of analyzed systems. That was done to also study the variability of the results. For example, the PLR of the systems NREL3 and NREL4 was only calculated using five different methods of which a few of these results were strong outliers. Since the PLR evaluation is based on the mean of the calculated PLR values, a small predictor dataset may yield biased results. The idea is that an increasing number of calculated values decreases the average PLR variability and therefore increases the accuracy of the estimated reference PLR. Thereby, the trustworthiness of the PLR The provided modeled irradiance dataset should have been used, which was not done by all participants.
The reported PLR corresponds to the average PLR of SCSF1, YoY2, and STL1 (see Figure  A3 ). The system power was for approximately the first half of its recorded lifetime limited by inverter clipping. Afterwards, the output power was not capped anymore. A calculation of PLR using this power data series, which has been done by all participants, does not correspond to the mean PLR. F I G U R E 9 (A) c The power datasets were subject to data shifts at the beginning of operation. These shifts were detected only by the participants applying STL1 and YoY2. The average of the results of these methodologies is reported as PLR (see Figure  A4 ). as being close to the "real" PLR is higher if more PLR values are included. With the voted results of our interlab/intermethod comparison shown in Table  5 , we may actually have an ensemble learning approach to determine the PLR i accurate and reproducible.
G7-3-24: Evaluation of seven systems with 24 approaches The first evaluation is based on the results calculated for seven datasets excluding all NREL and US DOE PV systems based on the dataset issues discussed before. For a PLR calculation method to be included in this analysis, at least three PLR values have to be calculated. The number of calculated PLR per system over these seven systems ranges from 20 to 27. subject to temperature correction is subject to high variations. The statistical method with the lowest accuracy in direct comparison is LS-LR2, an approach using the PR as metric and LR as calculation method. The filter (see Table  1 ) used for LS-LR2 only applies a PR threshold of 0% to 100% and thereby does not exclude outlier sufficiently, which are ultimately affecting the final result since LR is strongly affected by nonvalid datapoints.
Methods using the common approaches of YoY and STL are performing with relatively lower uncertainty throughout, but also alternative models such as HW, FBP, or the VAR method yield satisfactory results. It is interesting to observe the deviance in the results looking at with the PLR calculated for the EURAC system using SCSF1, the difference of the PLR using statistical method SCSF1 compared to the result decreases. A similar observation was made for the FOSS system. The calculated PLR (using SCSF1) is with -0. 35%/a clearly below the mean PLR of -0. 7%/a. An evaluation of the irradiance sensor data is still ongoing to verify a possible drift, but it has been ensured that both sensors are calibrated according to existing guidelines and standards. Instead, the results for the Pfaffstaetten systems using SCSF1 are quite close to the mean PLR, although a strong irradiance sensor drift of +0.
67%/a is suggested using this statistical method. Irradiance drifts have to be considered with care, since interannual variations might contribute to this effect and are not excluded while estimating a sensor drift using the SCSF approach. Additionally, solar brightening effects are taking place since the early 1980s to this date, describing an increase of solar irradiation on the earth's surface in certain parts of the world. 67 For example, Kiefer et al. 68   A minimum of seven calculated PLR was set for a statistical method to be included in this second benchmark category. At the same time at least one calculated PLR had to be for a NREL or US DOE dataset in order to avoid having redundant results compared to the first evaluation. The threshold reduced the number of considered methodologies from 24 to 17.
| CONCLUSIONS In this work, several experts in the field of PV reliability have applied their preferred methods for the calculation of PLR including various combinations of filters, metrics, and statistical methods. The aim of the exercise was to determine the capabilities and uncertainties of all the various approaches tested and to determine the "best" voted PLR estimates for the 23 systems under investigation. Furthermore, we wanted to quantify the variation and sensitivity of the individual PLR calculation steps based on the uncertainty, reproducibility, and bias of the results. This work is an important step forward to support the community to have tools to assess data quality of research and commercial PV systems, to utilize various filters, metrics, corrections and methods, and to have these tools be accessible and broadly usable.
Examples are the software packages RdTools, PVlib and PVplr. Based on the presented results, one uniform way of calculating reliable PLR, including fixed filtering approaches, the same metric, and statistical method, does not seem to exist at that point. Instead, it was shown that a thorough data quality check together with careful filtering approaches are absolutely crucial steps in calculating PLR, especially if the PV system dataset is subject to monitoring data related issues. In terms of calculation approaches, not only the most popular ones, STL and YoY, but also newly developed or less common ones, such as the VAR method, HW or FBP, demonstrated reliable results. While a standardized way of calculating PLR would be the desirable outcome of this study in order to reliably intercompare results across PV systems and operators, it was highlighted that a sensitive combination of filtering/metrics and statistical methods is crucial for achieving reliable results.
Even if we currently cannot define a single way to calculate the PLR of a PV system, this study does suggest that the voting, or preference aggregation, approach used here, may itself represent an accurate ensemble approach for PLR determination. It seems that a calculating PLR using many filters, performance metrics, corrections, and statistical modeling approaches does appear to provide consistent and robust estimates of PLR i for system i. This multiple method approach may serve as an ensemble model in which inaccuracies of all the different approaches are minimized in the voted result of the ensemble calculation of PLR i. Often, the filtering step is either performed insufficiently or not reported clear enough in corresponding literature. Therefore, when calculating PLR, an exhaustive report on filter selection and data flagging is vital to better comprehend, or even reproduce, the calculation steps and to enable the reader to compare the results.
First, the steps of calculating a PLR are shown as a detailed example using the EURAC dataset. In Section 4. 2, the statistical characteristics of the PV power time series datasets are characterized and presented. A discussion on the contributions to the uncertainty of PLR results follows, including measurement uncertainty and the standard deviation of the power time series. Afterwards, the impact of different data filters utilized in the benchmarking exercise are compared by applying them on two PV system datasets while holding the other calculation steps the same. In Section 4. 5, the calculated PLR of the digital plants are investigated.