<figure><img src="image_1.png" /></figure>

Article

# Quality Assessment of PROBA-V Surface Albedo V1 for the Continuity of the Copernicus Climate Change Service

Jorge Sánchez-Zapero <fnref n="1" />\*\d, Fernando Camacho <fnref n="1" />, Enrique Martínez-Sánchez <fnref n="1" />, Roselyne Lacaze <fnref n="2" />, Dominique Carrer <fnref n="3" />, Florian Pinault <fnref n="3" />, Iskander Benhadj <fnref n="4" /> and Joaquín Muñoz-Sabater <fnref n="5" />\d

Received: 15 July 2020; Accepted: 10 August 2020; Published: 12 August 2020

<figure><img src="image_2.png" /></figure>

check for
updates

**Abstract:** The Copernicus Climate Change Service (C3S) includes estimates of Essential Climate Variables (ECVs) as a series of Climate Data Records (CDRs) derived from satellite data. The C3S Surface Albedo (SA) v1.0 CDR is composed of observations from National Oceanic and Atmospheric Administration (NOAA) Very High Resolution Radiometers (AVHRR) (1981–2005), and VEGETATION sensors onboard Satellites for the Observation of the Earth (SPOT/VGT) (1998–2014) and Project for Onboard Autonomy satellite (PROBA-V) (2014–2020), and will continue with Sentinel-3 (from 2020 onwards). The goal of this study is to assess the uncertainties associated with the C3S PROBA-V SA v1.0 product, with a focus on the transition from SPOT/VGT to PROBA-V. The methodology followed the good practices recommended by the Land Product Validation sub-group (LPV) of the Working Group on Calibration and Validation (WGCV) of the Committee on Earth Observing Satellites (CEOS) for the validation of satellite-derived global albedo products. Several performance criteria were evaluated, including an intercomparison with National Aeronautics and Space Agency (NASA) MCD43A3 C6 products. C3S PROBA-V SA v1.0 and MCD43A3 C6 showed similar completeness but had higher fractions of missing data than C3S SPOT/VGT SA v1.0. C3S PROBA-V SA v1.0 showed similar precision (~1%) to MCD43A3 C6, improving the results of SPOT/VGT SA v1.0 (2–3%), but C3S PROBA-V SA v1.0 provided residual noise in the near-infrared (NIR). Good spatio-temporal continuity between C3S PROBA-V and SPOT/VGT SA v1.0 products was found with a mean bias between ±2%. The comparison with MCD43A3 C6 showed positive mean biases (5%, 8%, and 12% for visible, NIR and total shortwave, respectively). The accuracy assessment with ground measurements showed a median error of 18.4% with systematic overestimation (positive bias of 11.5%). The percentage of PROBA-V retrievals complying with the C3S target requirements was 28.6%.

**Keywords:** surface albedo; validation; PROBA-V; C3S; CEOS LPV; SPOT/VGT; MCD43

<footnote n="1">¹ Earth Observation Laboratory (EOLAB), Parc Científic Universitat de València, C/Catedrático Augustín Escardino 9, 46980 Paterna (Valencia), Spain; fernando.camacho@eolab.es (F.C.); enrique.martinez@eolab.es (E.M.-S.)</footnote>

<footnote n="2">² HYGEOS, Euratechnologies, 165 Avenue de Bretagne, 59000 Lille, France; rl@hygeos.com</footnote>

<footnote n="3">³ Centre National de Recherches Météorologiques (CNRM), Université de Toulouse, Météo France, CNRS 42, Avenue Gaspard Coriolis, 31057 Toulouse, France; dominique.carrer@meteo.fr (D.C.); florian.pinault@meteo.fr (F.P.)</footnote>

<footnote n="4">⁴ Flemish Institute for Technological Research (VITO), Remote Sensing Unit, Boeretang 200, B-2400 Mol, Belgium; iskander.benhadj@vito.be</footnote>

<footnote n="5">⁵ European Centre for Medium-Range Weather Forecasts (ECMWF), Shinfield Road, Reading RG2 8JD, UK; joaquin.Munoz@ecmwf.int</footnote>

<footnote>\* Correspondence: jorge.sanchez@eolab.es; Tel.: +34-963-769-448</footnote>
# 1. Introduction

The land surface albedo, which is defined as the ratio of the radiant flux reflected from the Earth's land surface to the incident flux, is a key forcing parameter controlling the planetary radiative energy budget and the partitioning of the radiative energy between the atmosphere and surface. The surface albedo varies in space and time as a result of both natural processes (e.g., solar illumination, snowfall, and vegetation growth) and human activities (e.g., the clearing and replanting of forests, the sowing and harvesting of crops, and the burning and grazing of rangelands) and is a sensitive indicator of environmental vulnerability [1]. Consequently, a long-term record of the surface albedo for the global landmass is required by climate, biogeochemical, hydrological, and weather forecast, models at a range of spatial (from a few meters to 30 km) and temporal (from daily to monthly) scales, and it is therefore, mandatory to quantitatively, and efficiency assess the uncertainties in SA measurement using these products.

The European's Union funded C3S [2] provides key indicators on climate change drivers by means of combining the observations of the climate system with the latest science to develop authoritative, quality-assured information about the past, current and future states of the climate. The C3S portfolio includes consistent estimates of multiple Essential Climate Variables (ECVs), including the surface albedo, which is primarily based on satellite data and provided as a series of gridded CDRs. In the frame of C3S, SA products have been derived from the data collected by NOAA/AVHRR (1981–2005) and SPOT/VGT (1998–2014). The Project for Onboard Autonomy satellite (PROBA-V) satellite [3], which was launched on May 2013 for seven years, was designed to bridge the gap in space-borne vegetation observation between SPOT/VGT (March 1998–May 2014) and the Sentinel-3 satellites. PROBA-V is comparable to the VGT sensors on SPOT platforms. In C3S, the PROBA-V SA v1.0 products are generated with the aim to extend the CDRs based on AVHRR and VGT observations over time. The algorithms for these C3S SA products were designed by Meteo-France based on previous research conducted within the Satellite Application Facility for Land Surface Analysis (LSA SAF) program of EUMETSAT [4,5]. Meteo-France developed the SA algorithms, Refs. [6–9] based on MSG/SEVIRI and MetOp/AVHRR within the EUMETSAT LSA SAF program, and were later adapted to these other sensors in C3S and Copernicus Global Land Service (CGLS) [10]. These products are generated by the processing line infrastructure implemented by VITO and openly distributed through the C3S Climate Data Store (CDS) [11].

Within the C3S CDS, a framework for the Evaluation and Quality Control (EQC) of climate data products derived from satellite and in situ observations to be catalogued was developed [12]. Validation, or quality assessment, is one of the main components defined in this EQC framework, and it is defined as the process of independently assessing the quality of the data products derived from the system outputs [13]. In terms of satellite-derived land products, validation is the procedure to assess their accuracy and quantify their uncertainties via analytical comparisons with reference data. Validation also constitutes the means of guaranteeing the compliance of products with user requirements, and helps end-users understand to what extent the product is suitable for their specific applications [14]. Based on previous validation works, mainly based on NASA products [15–19], but also on EUMETSAT [7], the CEOS WGCV LPV sub-group [20] has defined the global albedo product validation good practices [21]. Two main validation approaches have been defined: (i) The quantitative and qualitative product-to-product validation approach, which is referred to as satellite product intercomparison (i.e., indirect validation); and (ii) the direct point-to-pixel validation, which involves comparisons of satellite products with the albedo measured from in situ tower-based instruments (i.e., direct validation).

Indirect validation is helpful because most validation metrics cannot be computed using ground data, due to the limitations of ground measurements in terms of global conditions. Indirect validation approach mainly consists of comparing satellite-derived albedo products, particularly new products, with heritage albedo products [7,22–28]. In general, product intercomparison offers a means of assessing the discrepancies (systematic or random) between products. This method is particularly
valuable for finding spatial disagreements between products over large areas and for a wide range of cover types. However, this approach does not yield absolute validation results, and satellite product intercomparisons alone are insufficient to validate a new product. Then, direct validation enables the assessment of uncertainties, and it may be argued that only such methods can be seen as actual validation in the field of remote sensing [29]. Direct validation consists of comparing satellite retrievals with in-situ albedo measurements [30]. It is, therefore, mandatory to evaluate the spatial representativeness of ground albedo measurements, which depends on the land surface heterogeneity [31–34].

As the PROBA-V mission is arriving at the end of its lifetime, the aim of this paper is to perform the full quality assessment of PROBA-V based C3S SA v1.0 products. To achieve that, the main objectives are the following:

1. To evaluate the product completeness and spatio-temporal consistency of C3S PROBA-V SA v1.0 products over global conditions compared to SPOT/VGT SA v1.0 (and MCD43A3 C6 for benchmarking) to verify the continuity of the C3S time series;

2. To assess the precision of C3S PROBA-V SA v1.0 and to intercompare it with reference satellite products (C3S SPOT/VGT SA v1.0 and MCD43A3 C6);

3. To assess the accuracy and the associated uncertainties of the products via direct validation with ground measurements; and

4. To assess the compliance of the product with regards to climate user requirements.

The remainder of this paper is structured as follows: Sections 2 and 3 describe the datasets used in the study, which are for satellite-based products and ground data, respectively; Section 4 presents the validation methodology; Sections 5 and 6 gather and discuss the intercomparison and validation results, respectively; and the conclusions are summarized in Section 7.

## 2. Remote Sensing Surface Albedo Products

The main features of the three SA products investigated in this work (C3S PROBA-V SA v1.0, C3S SPOT/VGT SA v1.0 and MCD43A3 C6) are summarized in Table 1. The information of the spectral characteristics of the four optical bands of the VEGETATION sensors onboard the PROBA and SPOT satellites, and its equivalent Moderate resolution Imaging Spectroradiometer (MODIS) bands, is provided in Table 2.

<table><caption>**Table 1.** Characteristics of the global remote sensing SA products under study. GSD stands for Ground Sampling Distance.</caption><thead><tr><th>Product</th><th>Satellite /Sensor</th><th>BRDF Model: Volumetric /Geometrical Kernels</th><th>Frequency /Composite Period</th><th>GSD /Projection</th><th>Reference</th></tr></thead><tbody><tr><td>C3S PROBA-V SA v1.0</td><td>PROBA /VEGETATION</td><td>Ross_Thick /Roujean</td><td>10 days /30 days</td><td>1 km /Plate Carrée</td><td>[35]</td></tr><tr><td>C3S SPOT/VGT SA v1.0</td><td>SPOT /VEGETATION</td><td>Ross_Thick /Li_Sparse_Reciprocal</td><td>10 days /20 days</td><td>1 km /Plate Carrée</td><td>[36]</td></tr><tr><td>NASA MCD43A3 C6</td><td>TERRA+AQUA /MODIS</td><td>Ross_Thick /Li_Sparse_Reciprocal</td><td>Daily /16 days</td><td>500 m /Sinusoidal</td><td>[37]</td></tr></tbody></table>
<table><caption>**Table 2.** Spectral characteristics of PROBA-V, and its equivalent channels in the VGT-2 and MODIS sensors.</caption><thead><tr><th rowspan="2"></th><th colspan="3">PROBA-V</th><th colspan="3">VGT-2</th><th colspan="3">MODIS</th></tr><tr><th>Acronym</th><th>Center (nm)</th><th>Width (nm)</th><th>Acronym</th><th>Center (nm)</th><th>Width (nm)</th><th>Acronym</th><th>Center (nm)</th><th>Width (nm)</th></tr></thead><tbody><tr><td>Blue</td><td>B0</td><td>463</td><td>46</td><td>B0</td><td>458</td><td>37</td><td>Band3</td><td>489</td><td>20</td></tr><tr><td>Red</td><td>B2</td><td>655</td><td>79</td><td>B2</td><td>653</td><td>74</td><td>Band1</td><td>645</td><td>50</td></tr><tr><td>NIR</td><td>B3</td><td>845</td><td>144</td><td>B3</td><td>838</td><td>109</td><td>Band2</td><td>858.5</td><td>35</td></tr><tr><td>SWIR</td><td>MIR</td><td>1600</td><td>73</td><td>MIR</td><td>1635</td><td>101</td><td>Band6</td><td>1639</td><td>24</td></tr></tbody></table>

The quality flag information of each product was used to filter low-quality pixels (Table 3). For the C3S PROBA-V SA v1.0 products, the land pixels showing an input status “out of range” or “invalid” or “saturated” in the B2 and B0 channels were discarded. The SPOT/VGT SA v1.0 pixels where the algorithm failed were not considered in the validation exercise. Additionally, two ancillary variables were also taken into account: The uncertainty (ERR) and the mean age (AGE, in number of days) of the observations that are used to produce the SA. The SPOT/VGT SA v1.0 pixels with associated uncertainty greater than 0.2 and an AGE greater than 20 were discarded. For MODIS C6, the quality flags are given in the MCD43A2 product, and those pixels with fewer than seven valid observations were discarded.

<table><caption>**Table 3.** Quality flag (QFLAG) information used to filter pixels flagged as 'low-quality'.</caption><thead><tr><th>Product</th><th>Quality Control Used to Discard Pixels</th></tr></thead><tbody><tr><td>C3S PROBA-V SA v1.0</td><td>Sea (bit 1)<br>Input status out of range or invalid (bit 6)<br>B2 saturated (bit 10)<br>B0 saturated (bit 11)</td></tr><tr><td>C3S SPOT/VGT SA v1.0</td><td>Sea and continental water (bits 0-1 of QFLAG)<br>Algorithm Failed (bit 6 of QFLAG)<br>ERR &gt; 0.2<br>AGE &gt; 20</td></tr><tr><td>MCD43A2 C6</td><td>BRDF_Albedo_Band_Quality_Band1-7:<br>Magnitude inversion (number of observations lower than 7)</td></tr></tbody></table>

## 2.1. Evaluated Dataset: C3S SA v1.0 Based on PROBA-V Data

PROBA-V has operated since May 2013 at an altitude of 820 km in a sun-synchronous orbit with a local overpass time at launch of 10:45 a.m. However, because the satellite has no onboard propellant, this local overpass time gradually differs from the at-launch value [3]. The payload consists of three identical cameras, which are equipped with a very compact Three Mirror Anastigmat telescope. Each camera has two focal planes, one for the short-wave infrared (SWIR) and one for the visible and near-infrared (VNIR) bands, with a slightly different off-nadir along track viewing direction. PROBA-V observes four spectral bands: Blue (called B0: centered at 0.463 µm), Red (B1: 0.655 µm), NIR (B2: 0.845 µm), and SWIR (MIR: 1.600 µm). The target on the ground is imaged at different times and with different viewing angles. This specific technical concept makes the angular configurations of the observations in the VNIR and SWIR bands different, and hence, different angular information is provided. Observations are taken at resolutions from 100 to 180 m at nadir and up to 350 m and 660 m at the swath extremes for the VNIR and SWIR channels, respectively [38]. The final PROBA-V data [39], disseminated by the PROBA-V Ground Segment at 100 m, 300 m and 1 km resolutions, have been available on [40] since the end of 2013.
The C3S PROBA-V SA v1.0 products (example in Figure 1) are freely available in the C3S CDS [11]. They also provide Directional-Hemispherical albedos (AL-DH)—also called Black-Sky Albedo (BSA)—and the Bi-Hemispherical albedos (AL-BH)—also called White-Sky Albedo (WSA)—in three broadband spectral domains (visible [VI: 0.4–0.7 µm], NIR [NI: 0.7–4 µm] and total shortwave [BB: 0.3–4 µm]). These broadband albedos are taken from the CGLS [10]. In addition, the spectral AL-DH and AL-BH albedos in the four spectral channels (see Table 2) are made available in the C3S. These products have been produced every 10 days from the end of 2013 onwards, with the production dates on the 3rd, 13th, and 23rd of each month and delivered with a 12 day lag in near real-time. The spatial resolution of the grid is 1/112°, resulting in a pixel size of approximately 1 km at the equator. Apart from the layers corresponding to albedo retrievals, the ancillary layers corresponding to their respective errors (ERRs), the associated quality flag (QFLAG) and the number of valid observations during the synthesis period (NMOD) are provided. The information of each layer is described in the Product User Guide and Specification (PUGS) document [41].

<figure><img src="image_3.png" /><figcaption>**Figure 1.** Global distribution of PROBA-V SA v1.0 black-sky albedo values in the total shortwave (AL-DH-BB) on 23 September 2014.</figcaption></figure>

The methodology, which is described in the Algorithm Theoretical Basis Document (ATBD) [35], follows the approach of separating the atmospheric correction, directional reflectance normalization, and albedo determination. First, the top-of-atmosphere (TOA) data are processed to obtain the cloud-free top-of-canopy (TOC) reflectances. Then, the spectral TOC reflectances acquired under different solar-viewing configurations during the synthesis period are normalized by inverting a linear kernel-driven model [42]. The synthesis period is 30 day and a semi-Gaussian weighting function with the maximum weight on the last observation of the period was selected for near real-time production. Then, the spectral albedos are computed using the angular integration of the kernel functions with the retrieved parameters for each pixel. Finally, the broadband albedo is defined as a linear combination of the spectral albedo values in the available spectral channels. The narrow to broadband conversion coefficients are applied both for the black-sky and for the white-sky albedos.

## 2.2. C3S SA v1.0 Based on SPOT/VGT Data

C3S SPOT/VGT SA v1.0, which is available from C3S CDS [11], provides a CDR of the global black-sky and white-sky albedo estimates for the period from 1999 to 2014 at a ground sampling distance of 1/112° and a temporal frequency of 10 days using a compositing window of 20 days. The dates of production are the 10th, 20th and 30th of each month. As for PROBA-V, the albedo quantities are provided for the three broadband domains (visible [0.4–0.7 µm], NIR [0.7–4 µm] and total shortwave [0.3–4 µm]) and in the four spectral channels (see Table 2) of the VEGETATION sensors.
The albedo products are based on the data acquired by the VEGETATION-1 and VEGETATION-2 sensors aboard the SPOT-4 and SPOT-5 satellites, respectively.

The retrieval algorithm [6,7,9] was developed based on the previous developments of the SA products [6–9] based on MSG/SEVIRI and Metop/AVHRR in the framework of the EUMETSAT LSA SAF project [4], and it was later adapted here to these other sensors in CGLS and C3S [43]. The input data were Collection 3 of SPOT/VGT, which corrects the bug previously detected regarding the incorrect calculation of the Sun-Earth distance of Collection 2, includes improved cloud and snow/ice detection, and revises the absolute radiometric calibration for the entire archive [44]. The main differences with the C3S PROBA-V SA v1.0 algorithm are related to the use of a different Bidirectional Reflectance Distribution Function (BRDF) model and temporal composite scheme. PROBA-V SA v1.0 implemented the Ross_Thick kernel for volumetric scattering and the Roujean kernel for geometric scattering [42], whereas SPOT/VGT SA v1.0 implemented the Ross_Thick kernel [42] for volumetric scattering and the Li_Sparse_Reciprocal kernel for geometrical scattering [45]. The synthesis period of PROBA-V SA v1.0 is 30 days using a semi-Gaussian weighting function. The SPOT/VGT SA v1.0 retrieval methodology uses a 20-day temporal composite, meaning that each surface albedo product is built from the valid SPOT/VGT observations corresponding to the 20-day period preceding the calculation date. At the BRDF inversion step, the previous inversion result is used as a priori information. Hence, a recursive temporal composition of the information over a longer time period (approximately three times longer than the production period) can be achieved to guarantee the coherence and spatial completeness of the product. The “age” (AGE parameter in the output product) of the clear-sky observations exploited in the recursive inversion scheme is an important piece of information for potential applications, and is therefore, also made available to users. This age corresponds to the mean age with respect to the date of the calculation of the clear observations considered for the albedo calculation. More details are given in the ATBD [36].

Sánchez-Zapero et al. [46] showed that the C3S SPOT/VGT SA v1.0 products were good quality based on most of the criteria evaluated, reaching validated stage 1 in the CEOS LPV hierarchy. However, two main limitations were found: (i) Some temporal noise existed at the short-time scale; and (ii) the product was not able to capture some snow events, showing large uncertainties (>0.2) in those cases. Comparisons with ground measurements (461 samples, 2000–2005 period) from 15 FLUXNET stations showed similar overall uncertainty (RMSD = 0.05) as other satellite products (MCD43A3 C6, GlobAlbedo, and GLASS), but a positive bias (14%) was found.

## 2.3. NASA MCD43 C6 Based on MODIS Data

The MODIS BRDF/Albedo (MCD43A3) Collection 6 (C6) dataset, which is available from [47], provides both directional albedo at local solar noon and bi-hemispherical albedo for MODIS bands 1–7 and for three broadbands (visible [0.3–0.7 µm], NIR [0.7–5.0 µm], and total shortwave [0.3–5.0 µm]). The MCD43A3 albedo quantities are delivered at a resolution of 500 m in a sinusoidal projection. They have been produced every day since 2000 with a synthesis period of 16 days. Data from both Terra and Aqua satellites are used in the generation of this product.

The MODIS albedo algorithm uses atmospherically corrected reflectance data (MOD09 product) to establish the best fit to a linear kernel-driven BRDF model, with the exception of the observations flagged as “cloud”, “cirrus high” or “aerosol high”. Like for C3S SPOT/VGT SA v1.0, the parametric BRDF model uses the Ross_Thick kernel for volumetric scattering and the Li_Sparse_Reciprocal kernel for geometrical scattering [45,48]. A full retrieval of the model is attempted if there are seven or more high-quality observations well distributed over the viewing hemisphere during the 16 day synthesis period. When the number of observations is strictly less than seven and strictly greater than two, or if observations are not well sampled or do not well fit the BRDF model, a back-up algorithm with prior information is used. Then, a fill value is stored if the number of observations is strictly less than three, and the separated snow-free gap-filled products are also accessible [49]. Then, the BRDF model parameters are used for estimating spectral albedos from angular integration. The broadband albedos
are computed using the spectral to broadband conversion approach [50]. The MCD43 C6 products use
an improved back-up database, which is pixel-based updated from the latest full inversion as opposed
to the land cover-based database used in the previous Collection 5.

MCD43A3 SA products have reached CEOS LPV validation stage 3 [51]. Sánchez-Zapero et al. [46] reported an overall uncertainty (RMSD) of 0.053 and a low negative bias (–11.9%) of MCD43A3 C6 compared with the ground data from 15 FLUXNET homogeneous sites for the 2000–2005 period (653 samples), and a slight overall uncertainty (RMSD = 0.032) was reported compared with European FLUXNET measurements over snow-free conditions [52]. Existing studies of the previous collection 5 indicate that the accuracy of the MODIS shortwave broadband albedo meets the requirements (<5%) for both snow-free and snow-covered surfaces [31–34].

3. Ground Dataset

3.1. *Ground-Based Observations for Validation (GBOV) Database*

The CGLS GBOV [53] initiative aims at facilitating the use of observations from operational ground-based monitoring networks and their comparison to Earth Observation (EO) products. In the case of the SA, GBOV provides the Reference Measurements (RMs) and upscaled Land Products (LPs) based on the RM data [54,55] from over 24 sites coming from different networks, such as BSRN, FLUXNET, or SURFRAD. Currently, the GBOV database is made available for the 2012–2018 period [53]. The RM GBOV data corresponds to blue-sky albedo, which is defined as the fraction between the downward and the upward shortwave radiative flux. The diffuse fraction is also included in the RM dataset.

The innovative approach for the generation of GBOV LPs could be very useful for the validation of satellite SA products since it can be applied to both homogeneous and heterogeneous land surfaces. However, we can expect large discrepancies over heterogeneous sites compared to homogeneous sites [54,56]. The CEOS LPV albedo validation protocol [21] recommends the use of ground values measured at the flux tower (i.e., the RM) for the direct validation of EO products. For those reasons, GBOV RMs are used in this study for the accuracy assessment. Note that RMs are provided daily in the GBOV database with a typical temporal step of 30 or 1 min depending on the station. The estimation of the uncertainty associated with a solar radiation measurement by a commercial pyranometer is around 5% (at a 95% confidence level) for daily values under ideal conditions [57]. Since satellite products are defined to provide SA retrievals at solar local noon, the RMs used in this study have been taken at the same time. For this study, the concomitant period to PROBA-V was used (i.e., 2014–2018). The main characteristics of the 20 GBOV sites providing the RMs at solar local noon for the period under study are summarized in Table 4.

3.2. *Analysis of the Spatial Representativeness of Tower Measurements*

The reference albedo measured from towers covers a circular footprint that varies with the tower height. It is unlikely that the footprint of the ground measurements exactly matches the satellite pixel sizes. Then, the spatial representativeness of the tower-based measurements should be evaluated to minimize the issues associated with spatial representativeness in the point-to-pixel comparison [31,32].

The semivariogram [58,59] is one of the most efficient tools for describing spatial representativeness. Semivariograms were computed for the stations under evaluation using Sentinel-2 surface reflectances at a spatial resolution of 10 m near-nadir. Two different periods (leaf-off and leaf-on) periods during the year were considered to evaluate the spatial representativeness of the region around the ground tower. The spatial attributes (e.g., the range, sill and nugget) were computed by fitting the variogram estimator to an isotropic spherical model [60], and they can reveal the spatial variability of land surfaces and the scaling effects associated with remotely sensed data [31–34,61–63]. The methodology adopted in this study for the estimation of the geostatistical indexes is based on the comparison of the variogram model parameters retrieved at different spatial resolutions (i.e., from 1.0 km² to 1.5 km²
squared subsets). Four different geostatistical attributes were used [16,32]: The relative coefficient of variation ($R_{CV}$), the scale requirement index ($R_{SE}$), the relative strength of the spatial correlation ($R_{ST}$) and the relative proportion of the structural variation ($R_{SV}$).

<table><caption>**Table 4.** Characteristics of the 20 GBOV sites providing data during the 2014–2018 period.</caption><thead><tr><th>#</th><th>Site ID</th><th>Name</th><th>Country</th><th>Network</th><th>Land Cover</th><th>Lat (deg)</th><th>Lon (deg)</th></tr></thead><tbody><tr><td>1</td><td>USA_BND</td><td>Bondville</td><td>USA</td><td>SURFRAD</td><td>Croplands</td><td>40.052</td><td>-88.373</td></tr><tr><td>2</td><td>USA_BAO</td><td>Boulder</td><td>USA</td><td>BSRN</td><td>Croplands</td><td>40.050</td><td>-105.004</td></tr><tr><td>3</td><td>BEL_BRA</td><td>Brasschaat</td><td>Belgium</td><td>FLUXNET</td><td>Mixed Forest</td><td>51.309</td><td>4.521</td></tr><tr><td>4</td><td>NET_CAB</td><td>Cabauw</td><td>Netherlands</td><td>BSRN</td><td>Grasslands</td><td>51.971</td><td>4.927</td></tr><tr><td>5</td><td>AUS_CLP</td><td>Calperum</td><td>Australia</td><td>FLUXNET</td><td>Shrublands</td><td>-34.003</td><td>140.588</td></tr><tr><td>6</td><td>USA_DRA</td><td>Desert Rock</td><td>USA</td><td>SURFRAD</td><td>Bare Soil</td><td>36.624</td><td>-116.019</td></tr><tr><td>7</td><td>USA_FPK</td><td>Fort Peck</td><td>USA</td><td>SURFRAD</td><td>Grasslands</td><td>48.308</td><td>-105.102</td></tr><tr><td>8</td><td>GER_GEB</td><td>Gebesee</td><td>Germany</td><td>FLUXNET</td><td>Croplands</td><td>51.100</td><td>10.914</td></tr><tr><td>9</td><td>NAM_GOB</td><td>Gobabeb</td><td>Namibia</td><td>BSRN</td><td>Bare Soil</td><td>-23.561</td><td>15.042</td></tr><tr><td>10</td><td>USA_GCM</td><td>Goodwin Creek</td><td>USA</td><td>SURFRAD</td><td>Decidous Broadleaf</td><td>34.255</td><td>-89.873</td></tr><tr><td>11</td><td>FRA_GRI</td><td>Grignon</td><td>France</td><td>FLUXNET</td><td>Croplands</td><td>48.844</td><td>1.952</td></tr><tr><td>12</td><td>FRA_GUY</td><td>Guyaflux</td><td>French Guyana</td><td>FLUXNET</td><td>Evergreen Broadleaf</td><td>5.279</td><td>-52.925</td></tr><tr><td>13</td><td>GER_HAI</td><td>Hainich</td><td>Germany</td><td>FLUXNET</td><td>Mixed Forest</td><td>51.070</td><td>10.450</td></tr><tr><td>14</td><td>USA_NRF</td><td>Niwot Ridge</td><td>USA</td><td>FLUXNET</td><td>Evergreen Needelleaf</td><td>40.033</td><td>-105.546</td></tr><tr><td>15</td><td>ITA_REN</td><td>Renon</td><td>Italy</td><td>FLUXNET</td><td>Evergreen Needelleaf</td><td>46.587</td><td>11.434</td></tr><tr><td>16</td><td>USA_PSU</td><td>Rock Springs</td><td>USA</td><td>SURFRAD</td><td>Decidous Broadleaf</td><td>40.720</td><td>-77.931</td></tr><tr><td>17</td><td>USA_SFS</td><td>Sioux Falls</td><td>USA</td><td>SURFRAD</td><td>Croplands</td><td>43.730</td><td>-96.620</td></tr><tr><td>18</td><td>USA_SGP</td><td>Southern Great Plains</td><td>USA</td><td>FLUXNET</td><td>Croplands</td><td>36.606</td><td>-97.489</td></tr><tr><td>19</td><td>USA_TBL</td><td>Table Mountain</td><td>USA</td><td>SURFRAD</td><td>Bare soil and Rocks</td><td>40.125</td><td>-105.237</td></tr><tr><td>20</td><td>AUS_TMB</td><td>Tumbarumba</td><td>Australia</td><td>FLUXNET</td><td>Evergreen Broadleaf</td><td>-35.657</td><td>148.152</td></tr></tbody></table>

The four geostatistical attributes can be combined in a compact metric ($ST_{score}$ [31,32], see Equation (1)), which represents a spatial representativeness score using $R_{SE}$ as the primary marker and the others like secondary weights. When the spherical variogram model does not provide a good fit to the variogram estimator, another indicator ($RAW_{score}$ [31,32], see Equation (2) could be used to provide a spatial representativeness score, which is only based on the $R_{CV}$.

$$ST_{score} = \left( \frac{|R_{CV}| + |R_{ST}| + |R_{SV}|}{3} + R_{SE} \right)^{-1} \quad (1)$$

$$RAW_{score} = |2 R_{CV}|^{-1} \qquad (2)$$

Both scores are directly proportional to the site representativeness. We consider that sites are heterogeneous or not spatially representative when both scores are lower than 2.0 (see Table 5) because large differences, due to spatial heterogeneity, are expected for scores below this threshold [16]. Based on it, the ground measurements coming from *Boulder* (USA_BAO), *Renon* (ITA_REN), *Rock Springs* (USA_PSU) and *Table Mountain* (USA_TBL) sites are not used for the accuracy assessment during the leaf-off seasonal period. In addition, the *Cabauw* (NET_CAB), *Goodwin Creek* (USA_GCM) and *Gobabeb* (NAM_GOB) sites were discarded during leaf-on seasonal period. Figure 2 shows two examples of sites of stations spatially representative (top side), and two examples of not spatially representative (bottom side).
<table><caption>**Table 5.** Geospatial statistics of the 20 selected GBOV sites for the accuracy assessment.</caption><thead><tr><th>#</th><th>Site ID</th><th>Footprint (m)</th><th>Seasonal Period</th><th>R<sub>CV</sub> (%)</th><th>R<sub>SE</sub> (%)</th><th>R<sub>ST</sub> (%)</th><th>R<sub>SV</sub> (%)</th><th>ST<sub>score</sub></th><th>RAW<sub>score</sub></th></tr></thead><tbody><tr><td rowspan="2">1</td><td rowspan="2">USA_BND</td><td rowspan="2">126</td><td>Leaf-off</td><td>5.24</td><td>42.63</td><td>-5.17</td><td>59.16</td><td>1.52</td><td>9.54</td></tr><tr><td>Leaf-on</td><td>-4.56</td><td>48.50</td><td>2.84</td><td>37.10</td><td>1.58</td><td>10.9</td></tr><tr><td rowspan="2">2</td><td rowspan="2">USA_BAO</td><td rowspan="2">3788</td><td>Leaf-off</td><td>175.37</td><td>0.00</td><td>-3.30</td><td>-15.64</td><td>1.54</td><td>0.29</td></tr><tr><td>Leaf-on</td><td>19.43</td><td>0.00</td><td>2.77</td><td>-44.65</td><td>4.49</td><td>2.57</td></tr><tr><td rowspan="2">3</td><td rowspan="2">BEL_BRA</td><td rowspan="2">505</td><td>Leaf-off</td><td>11.24</td><td>0.01</td><td>0.15</td><td>-4.37</td><td>19.02</td><td>4.45</td></tr><tr><td>Leaf-on</td><td>12.74</td><td>0.06</td><td>-0.39</td><td>-14.61</td><td>10.74</td><td>3.93</td></tr><tr><td rowspan="2">4</td><td rowspan="2">NET_CAB</td><td rowspan="2">213</td><td>Leaf-off</td><td>11.31</td><td>0.00</td><td>-2.61</td><td>7.13</td><td>14.25</td><td>4.42</td></tr><tr><td>Leaf-on</td><td>32.62</td><td>0.00</td><td>0.74</td><td>11.70</td><td>1.66</td><td>1.53</td></tr><tr><td rowspan="2">5</td><td rowspan="2">AUS_CLP</td><td rowspan="2">253</td><td>Leaf-off</td><td>-5.83</td><td>31.92</td><td>4.15</td><td>-4.62</td><td>2.72</td><td>8.58</td></tr><tr><td>Leaf-on</td><td>-11.07</td><td>29.70</td><td>2.53</td><td>-3.13</td><td>2.83</td><td>4.52</td></tr><tr><td>6</td><td>USA_DRA</td><td>126</td><td>Single season</td><td>18.88</td><td>23.5</td><td>6.0</td><td>83.9</td><td>1.67</td><td>2.65</td></tr><tr><td rowspan="2">7</td><td rowspan="2">USA_FPK</td><td rowspan="2">126</td><td>Leaf-off</td><td>7.01</td><td>19.5</td><td>-2.03</td><td>-1.25</td><td>4.37</td><td>7.13</td></tr><tr><td>Leaf-on</td><td>42.27</td><td>23.5</td><td>3.36</td><td>69.44</td><td>1.62</td><td>1.18</td></tr><tr><td rowspan="2">8</td><td rowspan="2">GER_GEB</td><td rowspan="2">76</td><td>Leaf-off</td><td>10.35</td><td>81.49</td><td>-1.70</td><td>41.86</td><td>1.01</td><td>4.83</td></tr><tr><td>Leaf-on</td><td>-24.49</td><td>73.04</td><td>5.35</td><td>-13.41</td><td>1.14</td><td>2.04</td></tr><tr><td rowspan="2">9</td><td rowspan="2">NAM_GOB</td><td rowspan="2">10</td><td>Leaf-off</td><td>11.67</td><td>51.27</td><td>2.37</td><td>40.41</td><td>1.44</td><td>4.29</td></tr><tr><td>Leaf-on</td><td>26.47</td><td>58.17</td><td>0.92</td><td>47.89</td><td>1.20</td><td>1.89</td></tr><tr><td rowspan="2">10</td><td rowspan="2">USA_GCM</td><td rowspan="2">126</td><td>Leaf-off</td><td>-17.17</td><td>75.3</td><td>2.94</td><td>10.00</td><td>1.17</td><td>2.91</td></tr><tr><td>Leaf-on</td><td>30.21</td><td>75.2</td><td>-5.02</td><td>45.76</td><td>0.98</td><td>1.65</td></tr><tr><td rowspan="2">11</td><td rowspan="2">FRA_GRI</td><td rowspan="2">67</td><td>Leaf-off</td><td>5.30</td><td>68.84</td><td>-4.99</td><td>29.89</td><td>1.22</td><td>9.44</td></tr><tr><td>Leaf-on</td><td>10.62</td><td>61.83</td><td>4.17</td><td>0.42</td><td>1.49</td><td>4.71</td></tr><tr><td>12</td><td>FRA_GUY</td><td>253</td><td>Single season</td><td>-0.72</td><td>7.85</td><td>2.47</td><td>-1.98</td><td>10.45</td><td>69.33</td></tr><tr><td rowspan="2">13</td><td rowspan="2">GER_HAI</td><td rowspan="2">530</td><td>Leaf-off</td><td>7.66</td><td>0.00</td><td>5.94</td><td>22.01</td><td>8.43</td><td>6.53</td></tr><tr><td>Leaf-on</td><td>-6.37</td><td>0.00</td><td>1.94</td><td>7.40</td><td>19.11</td><td>7.85</td></tr><tr><td rowspan="2">14</td><td rowspan="2">USA_NRF</td><td rowspan="2">322</td><td>Leaf-off</td><td>-19.95</td><td>7.43</td><td>-12.98</td><td>-10.77</td><td>4.55</td><td>2.51</td></tr><tr><td>Leaf-on</td><td>6.71</td><td>6.61</td><td>-7.07</td><td>3.28</td><td>8.13</td><td>7.45</td></tr><tr><td rowspan="2">15</td><td rowspan="2">ITA_REN</td><td rowspan="2">12</td><td>Leaf-off</td><td>59.74</td><td>15.75</td><td>7.80</td><td>84.41</td><td>1.51</td><td>0.84</td></tr><tr><td>Leaf-on</td><td>21.44</td><td>32.85</td><td>-2.00</td><td>10.26</td><td>2.27</td><td>2.33</td></tr><tr><td rowspan="2">16</td><td rowspan="2">USA_PSU</td><td rowspan="2">126</td><td>Leaf-off</td><td>66.21</td><td>24.94</td><td>12.73</td><td>61.96</td><td>1.39</td><td>0.76</td></tr><tr><td>Leaf-on</td><td>5.06</td><td>22.86</td><td>3.16</td><td>-14.45</td><td>3.29</td><td>9.89</td></tr><tr><td rowspan="2">17</td><td rowspan="2">USA_SFS</td><td rowspan="2">32</td><td>Leaf-off</td><td>7.33</td><td>75.5</td><td>1.56</td><td>-6.74</td><td>1.24</td><td>6.82</td></tr><tr><td>Leaf-on</td><td>21.08</td><td>49.5</td><td>7.29</td><td>104.41</td><td>1.07</td><td>2.37</td></tr><tr><td rowspan="2">18</td><td rowspan="2">USA_SGP</td><td rowspan="2">152</td><td>Leaf-off</td><td>-14.76</td><td>37.13</td><td>-13.21</td><td>23.66</td><td>1.84</td><td>3.39</td></tr><tr><td>Leaf-on</td><td>10.02</td><td>35.26</td><td>-10.84</td><td>104.30</td><td>1.30</td><td>4.99</td></tr><tr><td rowspan="2">19</td><td rowspan="2">USA_TBL</td><td rowspan="2">126</td><td>Leaf-off</td><td>26.68</td><td>69.72</td><td>-4.78</td><td>79.70</td><td>0.94</td><td>1.87</td></tr><tr><td>Leaf-on</td><td>-17.46</td><td>64.91</td><td>6.59</td><td>-2.34</td><td>1.36</td><td>2.86</td></tr><tr><td>20</td><td>AUS_TMB</td><td>884</td><td>Single season</td><td>18.41</td><td>0.00</td><td>0.06</td><td>7.27</td><td>11.65</td><td>2.72</td></tr></tbody></table>
<figure><img src="image_4.png" /><figcaption>**Figure 2.** Sentinel-2 surface reflectance (Band 8A) images (3 km × 3 km) at a spatial resolution of 10 m centered over a selection of 4 GBOV sites (a) GER_HAI, (b) AUS_TMB, (c) ITA_REN, (d) USA_PSU). ST<sub>score</sub> and RAW<sub>score</sub>, indicators of the spatial representativeness, are displayed in the plots.</figcaption></figure>

# 4. Quality Assessment Methodology

## 4.1. Uncertainty Requirements

The accuracy assessment results were analyzed against predefined uncertainty levels based on a review of the existing user requirements from the Global Climate Observing System (GCOS), the World Meteorological Organization (WMO) and the key performance indicators of C3S.

In the last update of the GCOS requirements [64], there is a distinction between the products targeted for “adaptation” and “modelling” applications that results in different needs for the horizontal resolution. Modelling requirements (i.e., uncertainty Max [5%; 0.0025], see Table 6) are the focus of this study since modelling is the main application targeted. Other requirements come from the WMO [65], which aids in the setting of the priorities to be agreed upon by WMO members and their space agencies for enhancing the space-based GCOS system. The WMO specifies the requirements for the SA for climatologic applications at three quality levels (Table 6): Threshold (minimum requirement), breakthrough (significant improvement) and goal (optimum, no further improvement required). The stated “goal” WMO uncertainty requirement of 5% is, thus, equivalent to the GCOS requirement in relative terms. Apart from the GCOS and WMO, the Key Performance Indicator (KPI) of the maximum accuracy being between 10% and 0.01 was defined in the C3S program [66].

<table><caption>**Table 6.** GCOS, WMO and C3S SA uncertainty requirements.</caption><thead><tr><th></th><th>GCOS</th><th>WMO</th><th>C3S KPI</th></tr></thead><tbody><tr><th>Surface Albedo requirements</th><td>Max (5%; 0.0025)</td><td>Goal: 5%<br>Breakthrough: 10%<br>Threshold: 20%</td><td>Max (10%; 0.01)</td></tr></tbody></table>

Based on the existing requirements, three different levels (i.e., optimal, target and threshold) are predefined in this study (Table 7) with the aim to verify whether the results fit the purpose (Table 6). The optimal level (Max [5%, 0.0025]) was selected according to the GCOS uncertainty requirement and is equivalent to the WMO goal. The target level (Max [10%, 0.01]) is equivalent to the C3S KPI and partly equivalent to the WMO breakthrough level. Finally, the threshold level (Max [20%, 0.02]) was
adopted from the WMO. Poor performances of the product correspond to values above the threshold level (minimum requirement). Figure 3 displays the selected uncertainty levels as a function of the product values.

<table><caption>**Table 7.** Predefined uncertainty requirement levels used for SA validation.</caption><thead><tr><th></th><th>Optimal</th><th>Target</th><th>Threshold</th></tr></thead><tbody><tr><th>Surface Albedo<br>Uncertainty Requirements</th><td>Max [5%, 0.0025]</td><td>Max [10%, 0.01]</td><td>Max [20%, 0.02]</td></tr></tbody></table>

<figure><img src="image_5.png" /><figcaption>**Figure 3.** Uncertainty levels as a function of SA values.</figcaption></figure>

## 4.2. Validation Methods

The validation methodology follows the CEOS LPV good practice protocol for the validation of satellite-derived albedo products [21], and the validation metrics are presented in Section 4.2.1. The different strategies for product intercomparison and direct validation are described in Sections 4.2.2 and 4.2.3, respectively.

Figure 4 shows the global distribution of the sampling used in this study. The product intercomparison is evaluated over a 725-site land validation network (LANDVAL) of sites [67], which is designed to globally represent the variability of land surface types, and was used as the spatial sampling to evaluate these criteria. This network also includes 20 well-known desert calibration sites [26] for the precision evaluation, due to their high temporal stability. For the direct validation, the 20 selected GBOV homogeneous stations with available ground data (see Table 5) were used.

<figure><img src="image_6.png" /><figcaption>**Figure 4.** Global distribution of the 725 LANDVAL (including 20 desert calibration sites) and 20 GBOV sites.</figcaption></figure>
### 4.2.1. Validation Metrics

The definitions of the completeness, precision, uncertainty and accuracy (Table 8), which are applicable to SA validation, are drawn from the experimental recommendations of the Joint Committee for Guides in Metrology (JCGM) regarding the expression of uncertainty in measurement [68] and from the GCOS [64].

<table><caption>**Table 8.** Validation metrics for product validation.</caption><thead><tr><th>Quantity</th><th>Validation Metric</th></tr></thead><tbody><tr><td>Completeness</td><td>Gap size distribution (spatial and temporal)<br>Gap length</td></tr><tr><td>Precision</td><td>Median 3-point difference (intra-annual precision)<br>Median absolute deviation (inter-annual precision)</td></tr><tr><td>Accuracy</td><td>Median Error<br>Box-plots of the bias per albedo range</td></tr><tr><td>Uncertainty</td><td>Root mean square deviation (RMSD)<br>Scatter-plots of match-ups (MAR Linear models and correlation)</td></tr></tbody></table>

Completeness is the proportion of valid retrievals over an observation domain at any given time. It is, therefore, mandatory to document the completeness of the product (i.e., the distribution in space and time of missing data).

Two aspects of the precision, which is also called the repeatability, are evaluated: The intra-annual and inter-annual precision. The intra-annual precision (smoothness or δ function) corresponds to the temporal noise assumed to have no serial correlation within a season. The δ function is computed for each triplet of consecutive observations [69,70] as the absolute value of the difference between the center and the corresponding linear interpolation between the two extremes, and the median of the δ values is provided as an indicator of the intra-annual precision of satellite albedo products [21]. The inter-annual precision (i.e., the dispersion of the albedo values from year to year) was assessed over the 20 desert calibration sites between two consecutive years, and the median absolute deviation is provided as an indicator of the inter-annual precision [21].

Accuracy is the degree of “closeness of the agreement between the result of a measurement and a true value of the measurand” [68]. Commonly, accuracy is used to describe systematic errors and measure statistical bias, but the best practice is to provide the median error as an indicator of accuracy [21].

Uncertainty is a “parameter, associated with the result of a measurement, that characterizes the dispersion of the values that could reasonably be attributed to the measurand” [68]. Uncertainty includes the bias and precision errors and can be estimated by the Root Mean Square Deviation (RMSD). Additionally, the linear model fits are used to quantify the goodness of fit. For this purpose, Major Axis Regressions (MARs) were computed instead of Ordinary Least Squares (OLS) because it is specifically formulated to handle errors in both the x and y variables [71]. Other metrics are used, such as the number of samples (N), which is indicative of the power of the validation; or the correlation coefficient (R, estimated as the Pearson coefficient), which indicates the descriptive power of the linear accuracy test. The percentage of pixels within the predefined uncertainty levels (Table 7, Figure 3) is also quantified.

### 4.2.2. Satellite Product Intercomparison

The intercomparison of satellite products must account for the differences in the spatial and temporal characteristics of the different datasets. Consequently, a spatial support area and temporal support period were defined consistent with the C3S SA characteristics (Table 1). Thus, the MCD43A4 C6 products were re-sampled to Plate Carrée using a 1 km spatial sampling grid. Satellite products
were compared using the closest date of their different temporal composites, using the 10 day temporal frequency of C3S SA products.

The completeness and precision of the different products were evaluated and compared, as well as the temporal consistency among the several products, which was qualitatively assessed per biome type. The uncertainty estimates from the intercomparisons provided an overall figure of the spatio-temporal consistency between products. Scatter-plots and the associated metrics between pairs of products were analyzed and complemented with box-plots of the bias per bin albedo value. Our analysis focuses on the consistency during the 6-month overlap period between PROBA-V SA v1.0 and SPOT/VGT SA v1.0, whereas the intercomparison with MCD43A3 C6 was conducted using one year (2014) of data.

### 4.2.3. Ground-Based Validation

The main steps in the accuracy assessment of albedo products include the generation of blue-sky albedo [30] for a direct comparison with in-situ measurements and the test of the spatial representativeness of the in situ albedometer footprints for the satellite pixel resolution of interest according to in situ measurements standards [31,32]. Consequently, a careful selection of ground points and the characterization of their spatial representativeness are crucial for a meaningful point-to-pixel comparison, as presented in Section 3.2. For the first step, the blue-sky albedo from a satellite is estimated from the retrieved AL-DH-BB and AL-BH-BB satellite EO product under study, and weighted by the fraction of diffuse down-welling shortwave radiation from the ground station [30]. The next step is the careful selection of the homogeneous sites, which are similar to the footprints of the satellite pixel resolution of interest. The accuracy assessment of the SA satellite products was performed against the measurements coming from the 20 selected GBOV homogeneous stations presented in Section 3.

The study here was extended to the period from 2014 to 2018 to have the maximum concomitant samples from the GBOV dataset to PROBA-V observations. The accuracy assessment of MCD43A3 C6 for the same period and sampling was performed for benchmarking. This exercise was carried out at a resolution of 1 km, which is equivalent to one pixel in the case of PROBA-V SA v1.0, and averages of 2 × 2 pixels in the case of MCD43A3 C6. The 10 day frequency of C3S products was used as temporal sampling, and the average values of the daily ground data were computed to compare with satellite estimations during the corresponding temporal composite window of each product (see Table 1). This analysis was performed over the best quality pixels of C3S PROBA-V SA v1.0 and MCD43A3 C6, according to QFLAGs (Table 3).

## 5. Results

### 5.1. Satellite Product Intercomparison

#### 5.1.1. Product Completeness

The global map of the percentage of gaps (i.e., values with missing data) during one whole year of data (2014) for the PROBA-V SA v1.0 products (Figure 5) shows poor spatio-temporal continuity over latitudes higher than 45° north and over the equatorial belt, with a percentage of missing values up to 100% in some pixels over these areas. Note that the information from the quality flags was not considered here in the computation of the gaps (i.e., a gap or missing value corresponds to a fill value in the product). C3S products are not provided over areas out of the region from latitudes of 60°S to 75°N.

The temporal evolution of missing values (Figure 6a) shows the highest percentage of missing PROBA-V SA v1.0 values during wintertime in the northern hemisphere. The percentage of missing values ranges, on average, from approximately 12% (July and August 2014) to 32% (January and December 2014). PROBA-V SA v1.0 shows a similar temporal trend of missing data to MCD43A3 C6. Better completeness was found for SPOT/VGT SA v1.0 with almost no missing data during the months from March to May and a percentage up to 10% in wintertime in the northern hemisphere.
<figure><img src="image_7.png" /><figcaption>**Figure 5.** Percentage of gaps (%) during the year 2014 for the C3S PROBA-V SA v1.0 product considering all land pixels.</figcaption></figure>

<figure><img src="image_8.png" /><figcaption>**Figure 6.** Temporal variations of missing values for the year 2014 (a), and distribution of the temporal length of the missing values from December 2013 to May 2014 (b). Computation over LANDVAL sites for C3S PROBA-V SA v1.0 (purple), SPOT/VGT SA v1.0 (blue) and MCD43A3 C6 (green).</figcaption></figure>

The distribution of the temporal length of the missing values (Figure 6b) allows one to better understand the impact of the gaps for monitoring the temporal variations. PROBA-V SA v1.0 shows that approximately 65% of the gaps are shorter than 30 days and approximately 35% are shorter than 10 days (time resolution of the product). Similarly, typically, approximately 35–40% of the SPOT/VGT SA v1.0 gaps are shorter than 10 days, and approximately 75% of gaps are shorter than 30 days during the overlap period between SPOT/VGT and PROBA-V. MCD43A3 C6 has a shorter length of gaps, with approximately 40% of the gaps corresponding to 5 days.

### 5.1.2. Temporal Consistency

The temporal profiles of the different SA products (PROBA-V SA v1.0, SPOT/VGT SA v1.0 and MCD43A3 C6) in the three broadband domains (visible, NIR, and total shortwave) were analyzed over the 725 LANDVAL sites for each main biome type (Figure 7). The analytical focus of the transition between SPOT/VGT and PROBA-V and the 2013–2014 period was represented. All the satellite products are displayed at the center of their temporal composite windows (30 days in the case of PROBA-V SA v1.0, 20 days in the case of SPOT/VGT SA v1.0 and 16 days in the case of MCD43A3 C6). Note that the information of the PROBA-V SA v1.0 QFLAG was also displayed in these graphs: Filled dots correspond to pixels flagged as good quality, and unfilled dots correspond to pixels flagged as low-quality (land pixels with bit 6, 10 or 11 to 1) according to Table 3.
<figure><img src="image_9.png" /><figcaption>**Figure 7.** SA time series of the C3S PROBA-V SA v1.0, C3S SPOT/VGT SA v1.0 and MCD43A3 C6 broadband black-sky albedos for selected LANDVAL sites (a) Evergreen Broadleaf Forest, (b) Broadleaved Deciduous Forest, (c) Needle-leaf Forest, (d) cultivated, (e) shrublands/herbaceous/sparse vegetation, and (f) bare areas during the 2013–2014 period. Information on the sites is shown at the top of the corresponding figures. In the case of PROBA-V, filled dots correspond to ‘good quality’ pixels, and unfilled dots correspond to pixels flagged as ‘low-quality’ according to QFLAG (Table 3).</figcaption></figure>
For evergreen broadleaved forests, some temporal noise was observed in all satellite products. However, PROBA-V SA v1.0 and MCD43A3 C6 seem to provide less noise (i.e., flatter temporal trajectories) than SPOT/VGT SA v1.0, which seems to be more realistic for this biome type. For the rest of the biomes, the PROBA-V SA v1.0 profiles follow the temporal trends of SPOT/VGT SA v1.0 and MCD43A3 C6. The presence of rapid changes due to snow events, the occurrence of stable profiles and the phenological changes were consistent among the three datasets. However, PROBA-V SA v1.0 displays slightly large variability compared to other satellite products in the NIR domain (also affecting the total spectrum). Note that the use of the C3S PROBA-V SA v1.0 Quality Flag in northern latitudes removes valid snow observations in most cases.

A seasonality effect was observed with the sign of the bias between C3S PROBA-V SA v1.0 and SPOT/VGT SA v1.0 during the short overlap period over desert areas for the NIR and total spectrum. PROBA-V SA v1.0 tends to provide slightly higher values than SPOT/VGT SA v1.0 from December 2013 to February 2014, and the opposite trend was found from March 2014 to May 2014. Additionally, the SPOT/VGT SA v1.0 temporal trajectories show some temporal noise over desert sites, which is not observed for the other satellite products (PROBA-V SA v1.0 and MCD43A3 C6).

### 5.1.3. Spatio-Temporal Consistency

The overall spatio-temporal consistency between PROBA-V SA v1.0 and the reference products is assessed over the LANDVAL network sites considering all good quality observations according to the quality flags (see Table 3).

* PROBA-V SA v1.0 versus SPOT/VGT SA v1.0 (overlap period)

For the visible domain (Figure 8a–d and Table 9), PROBA-V SA v1.0 tends to provide slightly lower values than SPOT/VGT SA v1.0, with small negative mean biases of –2.2% and –2.8% for black-sky and white sky albedos, respectively. Optimal linear regression relationships from the MAR were found (offset ~0 and slope close to 1). Worse results were found in terms of the RMSD (uncertainty), with values of approximately 0.05 (~35%).

<table><caption>**Table 9.** Main performance statistics of C3S PROBA-V SA v1.0 versus SPOT/VGT SA v1.0 broadband albedo products over all LANDVAL sites during December 2013 to May 2014 period. The computation was performed over good quality pixels according to PROBA-V and SPOT/VGT QFLAGS.</caption><thead><tr><th></th><th colspan="6">PROBA-V SA v1.0 vs. SPOT/VGT SA v1.0 (Dec 2013-May 2014)</th></tr><tr><th></th><th>AL-DH-VI</th><th>AL-DH-NI</th><th>AL-DH-BB</th><th>AL-BH-VI</th><th>AL-DH-NI</th><th>AL-BH-BB</th></tr></thead><tbody><tr><th>Median_Err<br>(Median_Err %)</th><td>0.01<br>(8%)</td><td>0.014<br>(5%)</td><td>0.01<br>(4.7%)</td><td>0.013<br>(10%)</td><td>0.02<br>(6.3%)</td><td>0.014<br>(5.9%)</td></tr><tr><th>Bias<br>(Bias %)</th><td>-0.003<br>(2.2%)</td><td>0.007<br>(2.3%)</td><td>0.002<br>(0.9%)</td><td>-0.024<br>(2.8%)</td><td>0.007<br>(2.4%)</td><td>0.002<br>(1.0%)</td></tr><tr><th>RMSD<br>(RMSD %)</th><td>0.044<br>(35.3%)</td><td>0.031<br>(10.5%)</td><td>0.032<br>(14.5%)</td><td>0.047<br>(36.1%)</td><td>0.037<br>(11.7%)</td><td>0.036<br>(15.2%)</td></tr></tbody></table>

For the near-infrared (Figure 8b–e and Table 9), positive biases (PROBA-V SA v1.0 > SPOT/VGT SA v1.0) of 2.3–2.4% were found, with an RMSD of approximately 10% and high correlations (>0.94). PROBA-V SA v1.0 tends to provide higher values than SPOT/VGT SA v1.0 for albedo values lower than 0.5 and the opposite trend for albedo values higher than 0.5 (typically snow cases). In all cases, a median bias close to zero was found, which was within the optimal level of consistency.
<figure><img src="image_10.png" /><figcaption>**Figure 8.** Top: C3S PROBA-V SA v1.0 versus C3S SPOT/VGT SA v1.0 products' scatter-plots, where the dashed lines correspond to the optimal, target and predefined threshold uncertainty levels around the 1:1 relation (continuous line). Bottom: Box-plots of the bias per bin albedo range, where red bars indicate the median values, blue boxes stretch from the 25th percentile to the 75th percentile of the data and whiskers include 99.3% of the coverage data (±2.7 σ). Outliers are not displayed. The green, blue and orange lines correspond to optimal, target and threshold uncertainty levels, respectively. Computation over all LANDVAL sites for the December 2013–May 2014 period over good quality pixels according to the quality flags (Table 3) for AL-DH-VI (**a,d**), AL-DH-NII (**b,e**), and AL-DH-BB (**c,f**).</figcaption></figure>

Remarkably low mean biases (<1%) were found for the total shortwave (Figure 8c–f and Table 9), as well as high correlations (>0.93). Total uncertainties (RMSD) of 0.03 (~15%) were found. A systematic positive bias was found for almost all ranges, except for values lower than 0.1 and higher than 0.7, with the median bias typically within the optimal (GCOS) level of consistency (Table 6).

* PROBA-V SA v1.0 versus MCD43A3 C6 (2014 year)

Positive bias (PROBA-V SA v1.0 > MCD43A3 C6) of ~5% was found for the visible domain (Figure 9a–d and Table 10) with uncertainties (RMSD) lower than 0.05. Box-plots show slight median positive bias for albedo ranges up to 0.4, which is the range where most of the samples are located in this spectral domain. Large discrepancies were found for the highest ranges, mainly affected by snow cases, with a tendency towards the negative sign of the bias.

PROBA-V SA v1.0 tends to provide higher values than MCD43A3 C6 (~8–9%) for the near-infrared (Figure 9b–e and Table 10), with overall RMSD of approximately 14–15%. PROBA-V SA v1.0 tends to provide higher values than MCD43A3 C6 for albedo values lower than 0.5, and the opposite trend for albedo values higher than 0.5.
<figure><img src="image_11.png" /><figcaption>**Figure 9.** Top: C3S PROBA-V SA v1.0 versus MCD43A3 C6 products scatter-plots, where the dashed lines correspond to the optimal, target and predefined threshold uncertainty levels around the 1:1 relation (continuous line). Bottom: Box-plots of the bias per bin albedo range, where red bars indicate the median values, blue boxes stretch from the 25th percentile to the 75th percentile of the data and whiskers include 99.3% of the coverage data (±2.7 σ). Outliers are not displayed. The green, blue and orange lines correspond to the optimal, target and threshold uncertainty levels, respectively. Computation over all LANDVAL sites during the year 2014 over good quality pixels according to the quality flags for AL-DH-VI (a,d), ALI-NI (b,e), and AL-DH-BB (c,f).</figcaption></figure>

<table><caption>**Table 10.** Main performance statistics of C3S PROBA-V SA v1.0 versus MCD43A3 C6 broadband albedo products over all LANDVAL sites during the year 2014. The computation was performed over good quality pixels according to PROBA-V and MCD43A2 QFLAGS.</caption><thead><tr><th></th><th colspan="6">PROBA-V SA v1.0 vs. MCD43A3 C6 (2014)</th></tr><tr><th></th><th>AL-DH-VI</th><th>AL-DH-NI</th><th>AL-DH-BB</th><th>AL-BH-VI</th><th>AL-BH-NI</th><th>AL-BH-BB</th></tr></thead><tbody><tr><th>Median_Err<br>(Median_Err %)</th><td>0.011<br>(9.8%)</td><td>0.025<br>(9%)</td><td>0.027<br>(13.4%)</td><td>0.012<br>(10.1%)</td><td>0.029<br>(9.9%)</td><td>0.031<br>(14.5%)</td></tr><tr><th>Bias<br>(Bias %)</th><td>0.006<br>(5.3%)</td><td>0.024<br>(8.4%)</td><td>0.024<br>(11.9%)</td><td>0.006<br>(5.1%)</td><td>0.029<br>(9.6%)</td><td>0.029<br>(13.3%)</td></tr><tr><th>RMSD<br>(RMSD %)</th><td>0.048<br>(43.3%)</td><td>0.039<br>(14%)</td><td>0.044<br>(21.5%)</td><td>0.048<br>(41.4%)</td><td>0.045<br>(15.1%)</td><td>0.048<br>(22.2%)</td></tr></tbody></table>

The worse performance in the comparison PROBA-V SA v1.0 versus MCD43A3 C6 was found for the total spectrum (Figure 9c–f and Table 10), with a large positive bias of ~12–13%. As observed for the NIR, PROBA-V tends to provide higher values than MODIS C6 for albedo values lower than 0.5 and the opposite trend for albedo values higher than 0.5 (i.e., snow cases).
* Compliance with user requirements

The compliance matrix of C3S PROBA-V SA v1.0 versus the reference products (SPOT/VGT SA v1.0 and MCD43A3) with predefined uncertainty levels based on different user requirements is presented in Table 11.

<table><caption>**Table 11.** Compliance matrix (percentage of pixels filing the predefined uncertainty levels) of C3S PROBA-V SA v1.0 versus SPOT/VGT SA v1.0 (December 2013 to May 2014 period) and MCD43A3 C6 (2014 year) broadband albedo products. Computation over LANDVAL sites for good quality pixels according to QFLAGs (Table 3).</caption><thead><tr><th></th><th colspan="6">PROBA-V SA v1.0 vs. SPOT/VGT SA v1.0 (December 2013-May 2014)</th></tr><tr><th></th><th>AL-DH-VI</th><th>AL-DH-NI</th><th>AL-DH-BB</th><th>AL-BH-VI</th><th>AL-BH-NI</th><th>AL-BH-BB</th></tr></thead><tbody><tr><th>% optimal (GCOS)</th><td>28.9</td><td>48.6</td><td>50.2</td><td>23.9</td><td>39.9</td><td>41.8</td></tr><tr><th>% target (C3S KPI)</th><td>62.9</td><td>76.8</td><td>78.5</td><td>51.3</td><td>68.5</td><td>69.9</td></tr><tr><th>% threshold</th><td>83.1</td><td>92.6</td><td>88.1</td><td>72.8</td><td>90.2</td><td>82.6</td></tr></tbody><thead><tr><th></th><th colspan="6">PROBA-V SA v1.0 vs. MCD43A3 C6 (2014 year)</th></tr><tr><th></th><th>AL-DH-VI</th><th>AL-DH-NI</th><th>AL-DH-BB</th><th>AL-BH-VI</th><th>AL-BH-NI</th><th>AL-BH-BB</th></tr></thead><tbody><tr><th>% optimal (GCOS)</th><td>20.4</td><td>24.5</td><td>6.5</td><td>20.9</td><td>22.9</td><td>6.7</td></tr><tr><th>% target (C3S KPI)</th><td>55.5</td><td>50.4</td><td>24.8</td><td>52.7</td><td>46.4</td><td>23</td></tr><tr><th>% threshold</th><td>83.5</td><td>78.7</td><td>50.7</td><td>80.5</td><td>77.4</td><td>49.8</td></tr></tbody></table>

5.1.4. Intra-Annual Precision

The Probability Density Functions (PDFs) of the intra-annual precision (the so-called smoothness) are analyzed (Figure 10). The computation was performed over LANDVAL sites during the overlap period between SPOT/VGT, PROBA-V and MODIS products (i.e., December 2013-May 2014 period).

<figure><img src="image_12.png" /><figcaption>**Figure 10.** Histograms of the δ function (smoothness) for black-sky C3S PROBA-V SA v1.0, C3S SPOT/VGT SA v1.0 and MCD43A3 C6 for visible (a), NIR (b) and total shortwave (c). Computation over LANDVAL sites during the overlap period (December 2013-May 2014 period). The median δ values are presented for each product.</figcaption></figure>

The three products present similar distributions of the smoothness (δ). Most of the delta values are below 0.01, which demonstrates the high stability over a short time scale for the albedo products. In all satellite products, worse intra-annual precision (i.e., higher δ values) was found for white-sky albedos compared with equivalent black-sky albedos. The median δ values (Table 12), which are indicative of the intra-annual precision, show improved results (i.e., lower δ values) of PROBA-V SA v1.0 compared to SPOT/VGT SA v1.0 for visible and total shortwave and worse performance for NIR. Both C3S products provide much worse intra-annual precision than MODIS C6.
<table><caption>**Table 12.** Intra-annual precision indicator (Median of the 3-point difference) of the C3S PROBA-V SA v1.0, C3S SPOT/VGT SA v1.0 and MCD43A3 C6 products. Computation over LANDVAL sites during the overlap period (December 2013–May 2014 period).</caption><thead><tr><th rowspan="2"></th><th colspan="6">Median of 3-Point Difference (i.e., Median δ Values)</th></tr><tr><th>AL-DH-VI</th><th>AL-DH-NI</th><th>AL-DH-BB</th><th>AL-BH-VI</th><th>AL-BH-NI</th><th>AL-BH-BB</th></tr></thead><tbody><tr><th>PROBA-V SA v1.0</th><td>0.0026</td><td>0.0052</td><td>0.0035</td><td>0.0039</td><td>0.0074</td><td>0.0050</td></tr><tr><th>SPOT/VGT SA v1.0</th><td>0.0042</td><td>0.0047</td><td>0.0041</td><td>0.0057</td><td>0.0063</td><td>0.0055</td></tr><tr><th>MCD43A3 C6</th><td>0.0012</td><td>0.0022</td><td>0.0016</td><td>0.0018</td><td>0.0032</td><td>0.0023</td></tr></tbody></table>

### 5.1.5. Inter-Annual Precision

PROBA-V SA v1.0 black-sky albedo shows an inter-annual precision of approximately 1%, showing improved results compared to SPOT/VGT SA v1.0 (2–3%). MCD43A3 C6 provides better inter-annual precision, with median absolute deviations lower than 1%. As observed for the intra-annual precision, all products provided worse results for white-sky albedos compared to black-sky albedos (Table 13).

<table><caption>**Table 13.** The median absolute deviation between two consecutive years of the PROBA-V SA v1.0, SPOT/VGT SA v1.0 and MCD43A3 C6 products. Computation over desert calibration sites.</caption><thead><tr><th rowspan="2"></th><th colspan="6">Inter-Annual Precision (Median Absolute Deviation)</th></tr><tr><th>AL-DH-VI</th><th>AL-DH-NI</th><th>AL-DH-BB</th><th>AL-BH-VI</th><th>AL-BH-NI</th><th>AL-BH-BB</th></tr></thead><tbody><tr><th>PROBA-V SA v1.0</th><td>0.004 (1.2%)</td><td>0.006 (1.1%)</td><td>0.004 (0.9%)</td><td>0.004 (1.3%)</td><td>0.008 (1.3%)</td><td>0.006 (1.2%)</td></tr><tr><th>SPOT/VGT SA v1.0</th><td>0.008 (2.6%)</td><td>0.012 (2%)</td><td>0.009 (2%)</td><td>0.011 (3.7%)</td><td>0.02 (3.3%)</td><td>0.014 (3%)</td></tr><tr><th>MCD43A3 C6</th><td>0.003 (0.9%)</td><td>0.005 (0.8%)</td><td>0.004 (0.8%)</td><td>0.003 (1.2%)</td><td>0.006 (1.1%)</td><td>0.005 (1.1%)</td></tr></tbody></table>

### 5.2. Ground-Based Direct Validation

To investigate the accuracy of C3S PROBA-V SA v1.0 and MCD43A3 C6 satellite albedo products, scatter plots versus field measurements (GBOV RM) were produced for the 2014–2018 period over 20 homogeneous sites (see Section 3) with different vegetation types. Figure 11 and Table 14 show the scatter-plots, and relevant statistics from the direct validation exercise, whilst Table 15 summarizes the compliance of both satellite products with user requirements (predefined in Section 4.1). The relevant statistics per biome type are presented in Tables 16 and 17 for PROBA-V SA v1.0 and MCD43A3 C6, and the scatter-plots per biome type can be found in Appendix A. The temporal trajectories of both PROBA-V SA v1.0 and MCD43A3 C6 compared to the daily GBOV RMs are presented in Appendix B.

C3S PROBA-V SA v1.0 shows an overall accuracy (median error) of 18.2%, with a tendency to overestimate ground values (positive bias of 11.5%). This positive bias was mainly observed for low albedo values (up to 0.2, forest sites) showing an offset of 0.07 and a slope of 0.7 in the MAR relationship. MCD43A3 C6 provides a lower median error of 11.2% with a negative mean bias of -5.9%. C3S PROBA-V SA v1.0 products provide similar results in terms of uncertainty (RMSD of 22.4%) than MCD43A3 C6 (RMSD of 24.8%).
<figure><img src="image_13.png" /><figcaption>**Figure 11.** Direct validation of the best quality pixels of the C3S PROBA-V SA v1.0 (a) and MODIS MCD43A3 C6 (b) blue-sky albedo satellite products versus the ground measurements (RMs) at 20 GBOV stations during the 2014–2018 period. The continuous black lines correspond to the 1:1 lines and the dashed lines correspond to the optimal (GCOS uncertainty requirement), target (C3S KPI) and predefined threshold levels. The red lines correspond to the Major Axis Regression (MAR).</figcaption></figure>

<table><caption>**Table 14.** Relevant statistics of the direct validation of the best quality pixels (Table 3) of the C3S PROBA-V SA v1.0 and MODIS MCD43A3 C6 blue-sky albedo satellite products versus the GBOV ground values coming from 20 GBOV stations during the 2014–2018 period.</caption><thead><tr><th></th><th>PROBA-V SA v1.0</th><th>MCD43A3 C6</th></tr></thead><tbody><tr><td>Median Error (Median Error %)</td><td>0.032 (18.2%)</td><td>0.018 (11.2%)</td></tr><tr><td>Bias (Bias %)</td><td>0.020 (11.5%)</td><td>-0.010 (5.9%)</td></tr><tr><td>RMSD (RMSD %)</td><td>0.04 (22.4%)</td><td>0.04 (24.8%)</td></tr></tbody></table>

Note that 15.5% of the 1715 PROBA-V SA v1.0 samples achieved the optimal predefined level (i.e., GCOS requirements) and 28.6% of the target level (i.e., C3S KPI), as shown in Table 15. Slightly improved results were found for MCD43A3 C6 (23.7% and 45.5% of the optimal and target levels, respectively). Per biome type, PROBA-V SA v1.0 provides a large positive bias for forest (22.4%) and desert (10.4%) sites compared to crop (7.4%) and grassland/shrubland (4.2%) sites. MCD43A3 C6 systematically provides a large negative bias for most biome cases (croplands, grassland/shrublands and desert) except for forests, where a low positive bias was found (2.8%).

<table><caption>**Table 15.** Compliance matrix of the direct validation of the C3S PROBA-V SA v1.0 and MODIS MCD43A3 C6 blue-sky albedo satellite retrievals (best quality pixels, Table 3) versus ground values (N = 643) from 20 GBOV stations during the 2014–2018 period. N stands for the number of samples.</caption><thead><tr><th></th><th>PROBA-V SA v1.0</th><th>MCD43A3 C6</th></tr></thead><tbody><tr><td>% optimal (GCOS)</td><td>15.5</td><td>23.7</td></tr><tr><td>% target (C3S KPI)</td><td>28.6</td><td>45.5</td></tr><tr><td>% threshold</td><td>44.2</td><td>58.0</td></tr></tbody></table>
<table><caption>**Table 16.** Relevant statistics per main biome type and the compliance matrix of the direct validation of the C3S PROBA-V SA v1.0 blue-sky albedo satellite retrievals (best quality pixels, Table 3) versus the ground values (RMs) from 20 GBOV stations during the 2014–2018 period.</caption><thead><tr><th></th><th colspan="4">PROBA-V SA v1.0 vs. GBOV RM</th></tr><tr><th></th><th>Forest</th><th>Crops</th><th>Grass/Shrubs</th><th>Desert</th></tr></thead><tbody><tr><th>Number of Samples (stations)</th><td>608 (8)</td><td>488 (6)</td><td>316 (3)</td><td>303 (3)</td></tr><tr><th>Median Error (Median Error %)</th><td>0.037 (27.0%)</td><td>0.036 (18.3%)</td><td>0.015 (7.8%)</td><td>0.037 (16.7%)</td></tr><tr><th>Bias (Bias %)</th><td>0.030 (22.4%)</td><td>0.014 (7.4%)</td><td>0.008 (4.2%)</td><td>0.023 (10.4%)</td></tr><tr><th>RMSD (RMSD %)</th><td>0.044 (32.6%)</td><td>0.044 (22.4%)</td><td>0.024 (12.2%)</td><td>0.038 (17.2%)</td></tr><tr><th>% optimal (GCOS)</th><td>6.1</td><td>12.4</td><td>38.9</td><td>15.5</td></tr><tr><th>% target (C3S KPI)</th><td>13.8</td><td>25.8</td><td>59.5</td><td>30.4</td></tr><tr><th>% threshold</th><td>22.9</td><td>38.7</td><td>75.3</td><td>63.4</td></tr></tbody></table>

<table><caption>**Table 17.** Relevant statistics per main biome type and the compliance matrix of the direct validation of the MCD43A3 C6 blue-sky albedo satellite retrievals (best quality pixels, Table 3) versus the ground values (RMs) from 20 GBOV stations during the 2014–2018 period. N stands for the number of samples.</caption><thead><tr><th></th><th colspan="4">MCD43A3 C6 vs. GBOV RM</th></tr><tr><th></th><th>Forest</th><th>Crops</th><th>Grass/Shrubs</th><th>Desert</th></tr></thead><tbody><tr><th>Number of Samples (stations)</th><td>608 (8)</td><td>488 (6)</td><td>316 (3)</td><td>303 (3)</td></tr><tr><th>Median Error (Median Error %)</th><td>0.015 (12.1%)</td><td>0.021 (11.3%)</td><td>0.020 (11.5%)</td><td>0.017 (8.4%)</td></tr><tr><th>Bias (Bias %)</th><td>0.003 (2.8%)</td><td>-0.012 (6.7%)</td><td>-0.027 (15.3%)</td><td>-0.013 (6.5%)</td></tr><tr><th>RMSD (RMSD %)</th><td>0.031 (25.2%)</td><td>0.051 (28.0%)</td><td>0.042 (24.1%)</td><td>0.036 (17.8%)</td></tr><tr><th>% optimal (GCOS)</th><td>21.5</td><td>25.4</td><td>18.4</td><td>30.7</td></tr><tr><th>% target (C3S KPI)</th><td>41.1</td><td>44.5</td><td>43.4</td><td>58.1</td></tr><tr><th>% threshold</th><td>58.9</td><td>53.5</td><td>52.8</td><td>68.6</td></tr></tbody></table>

## 6. Discussion

C3S SA v1.0 based on PROBA-V provides continuity to the CDRs of global albedo products in the C3S from December 2013 onwards. Previously, SA products were derived from SPOT/VGT (1998–2014) and NOAA/AVHRR (1981–2005) data. Good spatio-temporal consistency in the transition from SPOT/VGT to PROBA-V for both black-sky and white-sky albedos, with mean biases below ±3% for the overlap period, was found. However, the comparison of C3S PROBA-V SA v1.0 with MCD43A3 C6 SA showed lower spatio-temporal consistency between satellite products with mean biases up to 13%. C3S PROBA-V SA v1.0 (and MCD43A3 C6) displayed more gaps (typically between 10% and 20%) than C3S SPOT/VGT v1.0.

Different aspects of the retrieval methodology play important roles in the existing discrepancies between the satellite products under study (C3S PROBA-V SA v1.0, C3 SPOT/VGT SA v1.0 and MCD43A3 C6) starting from the different input data and different atmospheric correction methods; including the BRDF parameterization, temporal compositing and angular integration; and finalizing with the narrow to broadband conversion.

Regarding the input data, each sensor works in different spectral channels (see Table 2). The SPOT/VGT and PROBA-V channels provide very similar spectral characteristics in the Blue, Red and NIR bands, whereas significant differences are found for the SWIR channel. The central wavelengths of the MODIS spectral bands are comparable to the corresponding bands for PROBA-V and VGT, albeit the MODIS spectral bands are narrower. These differences could translate into reflectance discrepancies in regions with high absorption features like in the visible domain, where large uncertainty values were found between pairs of products.

Each satellite processing chain uses its own method for cloud/shadow screening and atmospheric correction according to the spatial, spectral and directional capabilities of each instrument. Cloud or snow contamination is the main reason for missing data in the EO products derived from optical onboard satellite sensors. The conservative PROBA-V cloud detection algorithm [72] is one of the
reasons for the higher fraction of missing data of PROBA-V SA v1.0 products compared to SPOT/VGT SA v1.0.

Discrepancies between different albedo estimates can also be attributed to the different BRDF models used [7]. Moreover, the performance of the BRDF model for good clear-sky observations also depends on the number of available looks during the synthesis period and the angular distribution of the sampling. Large BRDF uncertainties are associated with snow targets (as observed for the highest albedo ranges) for which none of these parametric BRDF models were well suited [73]. PROBA-V, SPOT/VGT and MODIS are wide-FOV sensors on polar-orbiting platforms, and a low impact of discrepancies is expected, due to different sun-view configurations. However, the different compositing periods (see Table 1) could play an important role in the differences between satellite products, mainly in rapid SA variations, such as snow events. Different techniques for temporal composite approaches also affect the completeness of EO satellite products. SPOT/VGT SA v1.0 uses a recursive temporal composition approach [9], which is the main reason for the improved completeness compared to the other satellite products (C3S PROBA-V SA v1.0 and MCD43A3 C6) that are computed using classical composite schemes based on predefined temporal windows.

In the last step, the broadband albedos are defined using slightly different spectral regions. The same broadband spectral regions are defined in both C3S PROBA-V SA v1.0 and SPOT/VGT SA v1.0 products, which contribute to a better agreement, whereas the MCD43A3 C6 broadband albedos are slightly differently defined.

In addition, the temporal noise (i.e., large temporal variability) observed for C3S PROBA-V SA v1.0 in the NIR domain through the qualitative inspection of the temporal variations (see Section 5.1.2) has a strong relationship with the precision, since low intra-annual precision was found compared to both reference EO products in this spectral range. In terms of the inter-annual precision, improved results were found for C3S PROBA-V SA v1.0 (~1%) compared to C3S SA v1.0 based on SPOT/VGT SA data (2%).

Regarding the accuracy with the ground measurements, PROBA-V SA v1.0 provided a similar accuracy (bias of 11.5%) to that found for C3S SPOT/VGT SA v1.0 during the validation exercise [44], where a positive bias (14%) was also reported for a different sampling (i.e., different stations and dates). PROBA-V SA v1.0 also provided a slightly worse accuracy (median error of 18.2%) than MCD43A3 C6 (median error of 11.2%). PROBA-V SA v1.0 tends to overestimate the ground values, whereas MCD43A3 C6 showed the opposite sign of the mean bias. The positive bias of PROBA-V SA v1.0 was mainly observed for forest sites (SA < 0.2) explained in the fact that the Roujean kernel [42] for geometrical scattering may not fit some cover types well, especially dense forest canopies, where it showed a weak hotspot effect [74–76]. The negative bias of MCD43A3 C6 is mainly influenced by some outliers detected in Gobabeb (bare soil) and Cabaw (grassland). For the Cabaw case, the lower MCD43A3 C6 values are explained by the persistent cloudiness at the MODIS overpass times [56]. It is important to note that only the satellite retrievals classified as the best quality, according to QFLAGs (Table 3), were used in the direct validation. As observed in the temporal consistency, the use of QFLAGs removes most of the valid snow retrievals in the case of PROBA-V. Then, this exercise is almost equivalent to snow-free conditions, which is more convenient for assessing the uncertainty of satellite EO products, since it is expected that the spatial representativeness of the pixels dropped during the fall and winter months as a consequence of the increased sub-pixel heterogeneity, due to processes, such as non-uniform patterns of snowmelt [77]. The positive bias of C3S PROBA-V SA v1.0 is consistent with previous studies performed on the CGLS, where a positive bias of approximately 22% was found compared to ground measurements over 17 stations [27], and a positive bias of approximately 14% was found compared to National Ecological Observatory Network (NEON) ground data [78]. The accuracy assessment of MCD43A3 C6 was also consistent with that previously reported when comparing the combined TERRA+AQUA albedo product with eight field stations during the spring and summer months of 2003 and 2004 (i.e., equivalent to snow-free conditions), where an accuracy of 0.013 was reported in terms of the absolute bias [79]. It should be noted that the estimation of the
uncertainty of ground reference data is around 5% under ideal conditions [58]. Thus, fiducial reference
measurements must be characterized with highly calibrated instrumentation at dedicated cal/val sites
to better estimate the satellite product uncertainty budget.

The compliance of satellite EO products versus ground data with user requirements showed a low percentage of pixels within GCOS (only <25%) and C3S (<50%) requirements, which indicates the difficulties of achieving these requirements using current products. To further improve the compliance with requirements, it is recommended that EO programs provide the uncertainties associated with the processing chain, mainly related to the sensor calibration and atmospheric correction. In that way, the steps providing higher error can be improved. However, the comparison between satellite C3S SA v1.0 products showed an overall good spatio-temporal consistency in the comparison of SPOT/VGT versus PROBA-V during the 6-month overlapping period (December 2013-May 2014), with typically more than 60% of the samples within the C3S target requirements for the visible domain, and more than 75% of the samples within the requirements for the NIR and total shortwave.

7. Conclusions

This paper presents the quality assessment results of C3S PROBA-V SA v1.0 products (broadband albedos) through intercomparison with reference satellite products (C3S SPOT/VGT SA v1.0 and MCD43 C6) at the global scale and the direct validation with a representative amount of ground data (1715 samples) across different biomes types (eight stations over forests, six over crops, three over grassland/shrubs, and three over desert). This validation exercise is a novelty in the literature, since very few global and temporally representative SA validation exercises have been published [21], and they are mainly based on MODIS observations [15–17]. The validation methodology adopted the guidelines, protocols and metrics defined by the CEOS LPV best practices for the validation of global albedo satellite products [21]. Additional results, such as the comparison of the spectral albedos or the presentation of the satellite products intercomparison per biome type, are not shown in this manuscript for the sake of brevity; however, they can be found in the product quality assessment report [66].

The main conclusions are the following.

* The good spatio-temporal consistency of C3S PROBA-V and SPOT/VGT SA v1.0 products assures the continuity of the CDRs in terms of the uncertainty between products. However, C3S PROBA-V SA v1.0 (and MCD32A3 C6) provides low numbers of valid retrievals compared to C3S SPOT/VGT SA v1.0.

* C3S PROBA-V SA v1.0 shows similar inter-annual precision (~1%) to MCD43A3 C6, improving the results of SPOT/VGT SA v1.0 (2–3%), since they provide some temporal instability over desert calibration targets. Both C3S products provide lower intra-annual precision than MCD43A3 C6, mainly in the NIR domain where some temporal noise was found.

* The accuracy of the C3S PROBA-V SA v1.0 best quality retrievals with respect to the ground data over a five-year period (2014–2018) showed systematic positive overestimation, which was mainly observed for the lowest albedo ranges (SA < 0.2) over forest sites. Similar uncertainty (RMSD~ 0.4) was found for MCD43A3 C6 products using the same sampling, showing the opposite sign for the mean bias.

* Few current satellite EO albedo products comply with the GCOS, C3S and WMO uncertainty requirements.
Additionally, it is important to remark that the use of PROBA-V QFLAGs (bit 6, input status; and bits 10–11, B2–B0 saturation status) removes most of the valid snow retrievals. Therefore, masking out data by means of the QFLAGs is not recommended for snow applications (users should ignore the information of QFLAGs over snow targets for specific applications).

Based on these results, C3S PROBA-V SA v1.0 has reached validation stage 3 in the validation hierarchy of the CEOS LPV [20]. The continuity of the C3S SA CDR time series will be ensured using Sentinel-3 OLCI and SLSTR data, and the algorithm and design of the processing chain are currently being developed. C3S is also developing multi-sensor albedo products combining NOAA/AVHRR, SPOT/VGT and PROBA-V input data. The long-term CDRs, provided by the Copernicus Climate Change Service from 1981 to the present (with the aim of extending to the future), are an added extra compared with the existing EO programs, providing the longest and state-of-the-art albedo products.

**Author Contributions:** J.S.-Z. and F.C. conceived and conceptualized the work. J.S.-Z., F.C. and R.L. defined the methodology. J.S.-Z. and E.M.-S. developed the validation software for generating the results. J.S.-Z. wrote the first version of the manuscript. All authors contributed to the analysis, writing, review and editing of the manuscript. The C3S_Lot5 consortium is structured as follows: validation team (J.S.-Z., F.C. and E.M.-S.), algorithm development (D.C. and F.P.), project management (I.B.), and technical officer (J.M.-S.). All authors have read and agreed to published the manuscript.

**Funding:** This work was funded by the European Centre for Medium-Range Weather Forecasts in the framework of the Copernicus Climate Change Service (Official reference number service contract: 2018/C3S_312b_Lot5_VITO/SC1), led by VITO as the prime contractor. Previously, the development of the algorithm and the methodology were defined in the framework of the Copernicus Global Land Service (European Commission/Joint Research Center under Framework Service Contract N°199494).

**Acknowledgments:** This study has been undertaken using data from GBOV "Ground Based Observation for Validation" (https://land.copernicus.eu/global/gbov) founded by the European Commission Joint Research Centre FWC932059 as part of the Global Component of the European Union's Copernicus Land Monitoring Service.

**Conflicts of Interest:** The authors declare no conflict of interest.

## Abbreviations

The following abbreviations are used in this manuscript:

<table><tr><td>AL-DH</td><td>Directional-Hemispherical ALbedos</td></tr><tr><td>AL-BH</td><td>Bi-Hemispherical Albedos</td></tr><tr><td>ATBD</td><td>Algorithm Theoretical Basis Document</td></tr><tr><td>AVHRR</td><td>Advanced Very High Resolution Radiometer</td></tr><tr><td>BB</td><td>total shortwave</td></tr><tr><td>BRDF</td><td>Bidirectional Reflectance Distribution Function</td></tr><tr><td>BSA</td><td>Black-Sky Albedo</td></tr><tr><td>BSRN</td><td>Baseline Surface Radiation Network</td></tr><tr><td>CDR</td><td>Climate Data Records</td></tr><tr><td>CDS</td><td>Climate Data Store of C3S</td></tr><tr><td>CEOS</td><td>Committee on Earth Observing Satellites</td></tr><tr><td>CGLS</td><td>Copernicus Global Land Service</td></tr><tr><td>C3S</td><td>Copernicus Climate Change Service</td></tr><tr><td>C6</td><td>Collection 6 of MODIS products</td></tr><tr><td>ECV</td><td>Essential Climate Variables</td></tr><tr><td>EO</td><td>Earth Observation</td></tr><tr><td>EQC</td><td>Evaluation and Quality Control</td></tr><tr><td>EUMETSAT</td><td>European Organization for the Exploitation of Meteorological Satellites</td></tr><tr><td>FLUXNET</td><td>FLUXes NETwork (network of regional networks)</td></tr></table>
<table>
   <tr>
    <td>GBOV</td>
    <td>Ground-Based Observations for Validation</td>
   </tr>
   <tr>
    <td>CGOS</td>
    <td>Global Climate Observing System</td>
   </tr>
   <tr>
    <td>GLASS</td>
    <td>Global Land Surface Satellite</td>
   </tr>
   <tr>
    <td>GSD</td>
    <td>Ground Sampling Distance</td>
   </tr>
   <tr>
    <td>JCGM</td>
    <td>Joint Committee for Guides in Metrology</td>
   </tr>
   <tr>
    <td>KPI</td>
    <td>Key Performance Indicator</td>
   </tr>
   <tr>
    <td>LANDVAL</td>
    <td>LAND VALidation network</td>
   </tr>
   <tr>
    <td>LSA SAF</td>
    <td>Satellite Application Facility for Land Surface Analysis</td>
   </tr>
   <tr>
    <td>LPV</td>
    <td>Land Product Validation sub-group</td>
   </tr>
   <tr>
    <td>MAR</td>
    <td>Major Axis Regression</td>
   </tr>
   <tr>
    <td>MCD43</td>
    <td>TERRA+AQUA MODIS BRDF/Albedo/NBAR Product</td>
   </tr>
   <tr>
    <td>MetOp</td>
    <td>Polar-orbiting Meteorological satellites</td>
   </tr>
   <tr>
    <td>MIR</td>
    <td>Mid InfraRed</td>
   </tr>
   <tr>
    <td>MODIS</td>
    <td>MODerate resolution Imaging Spectroradiometer</td>
   </tr>
   <tr>
    <td>MSG</td>
    <td>Meteosat Second Generation</td>
   </tr>
   <tr>
    <td>N</td>
    <td>Number of samples</td>
   </tr>
   <tr>
    <td>NASA</td>
    <td>National Aeronautics and Space Agency</td>
   </tr>
   <tr>
    <td>NEON</td>
    <td>National Ecological Observatory Network</td>
   </tr>
   <tr>
    <td>NIR (or NI)</td>
    <td>Near-Infrared</td>
   </tr>
   <tr>
    <td>NOAA</td>
    <td>National Oceanic and Atmospheric Administration</td>
   </tr>
   <tr>
    <td>OLCI</td>
    <td>Ocean and Land Colour Instrument</td>
   </tr>
   <tr>
    <td>OLS</td>
    <td>Ordinary Least Squares</td>
   </tr>
   <tr>
    <td>LP</td>
    <td>Land Products</td>
   </tr>
   <tr>
    <td>PROBA-V</td>
    <td>Project for Onboard Autonomy satellite, the V standing for vegetation</td>
   </tr>
   <tr>
    <td>PUGS</td>
    <td>Product User Guide and Specification document</td>
   </tr>
   <tr>
    <td>QFLAG</td>
    <td>Quality FLAG</td>
   </tr>
   <tr>
    <td>R</td>
    <td>Correlation coefficient</td>
   </tr>
   <tr>
    <td>RM</td>
    <td>Reference Measurements</td>
   </tr>
   <tr>
    <td>RMSD</td>
    <td>Root Mean Square Deviation</td>
   </tr>
   <tr>
    <td>RTLSR</td>
    <td>Ross Thick kernel and Li Sparse-Reciprocal kernels</td>
   </tr>
   <tr>
    <td>SA</td>
    <td>Surface Albedo</td>
   </tr>
   <tr>
    <td>SEVIRI</td>
    <td>Spinning Enhanced Visible and Infrared Imager</td>
   </tr>
   <tr>
    <td>SLSTR</td>
    <td>Sea and Land Surface Temperature Radiometer</td>
   </tr>
   <tr>
    <td>SPOT</td>
    <td>Satellites for the Observation of the Earth</td>
   </tr>
   <tr>
    <td>SURFRAD</td>
    <td>Surface Radiation budget</td>
   </tr>
   <tr>
    <td>SWIR</td>
    <td>Short-Wave InfraRed</td>
   </tr>
   <tr>
    <td>TOA</td>
    <td>Top-Of-Atmosphere</td>
   </tr>
   <tr>
    <td>TOC</td>
    <td>Top-Of-Canopy</td>
   </tr>
   <tr>
    <td>VGT</td>
    <td>VeGeTation sensor</td>
   </tr>
   <tr>
    <td>VI</td>
    <td>Visible domain</td>
   </tr>
   <tr>
    <td>VNIR</td>
    <td>Visible and Near-InfRared</td>
   </tr>
   <tr>
    <td>WGCV</td>
    <td>Working Group on Calibration and Validation</td>
   </tr>
   <tr>
    <td>WMO</td>
    <td>World Meteorological Organization</td>
   </tr>
   <tr>
    <td>WSA</td>
    <td>White-Sky Albedo</td>
   </tr>
  </table>
Appendix A. Ground-Based Direct Validation per Biome Type

<figure><img src="image_14.png" /><figcaption>* C3S PROBA-V SA v1.0</figcaption></figure>

**Figure A1.** Direct validation per biome type (a) Forest, (b) Crops, (c) grass/shrublands, and (d) desert) of the C3S PROBA-V SA v1.0 blue-sky albedo best quality pixels versus the ground measurements (RMs) at 20 GBOV stations during the 2014–2018 period. The continuous black lines correspond to the 1:1 lines and the dashed lines correspond to the optimal (GCOS uncertainty requirement), target (C3S KPI) and predefined threshold levels. The red lines correspond to the Major Axis Regression (MAR).
<figure><img src="image_15.png" /><figcaption>**Figure A2.** Direct validation per biome type (a) Forest, (b) Crops, (c) grass/shrublands, and (d) desert of the MCD43A3 C6 blue-sky albedo best quality pixels versus the ground measurements (RMs) at 20 GBOV stations during the 2014–2018 period. The continuous black lines correspond to the 1:1 lines and the dashed lines correspond to the optimal (GCOS uncertainty requirement), target (C3S KPI) and predefined threshold levels. The red lines correspond to the Major Axis Regression (MAR).</figcaption></figure>
Appendix B. Temporal Realism of Blue-Sky Albedos over Gbov Sites

<figure><img src="image_16.png" /></figure>

**Figure A3.** Time series of the blue-sky C3S PROBA-V SA v1.0 and MCD43A3 C6 satellite albedos and GBOV RMs for the selected sites during the 2014–2018 period. In the case of PROBA-V, the filled dots correspond to ‘good quality’ pixels, and the unfilled dots correspond to pixels flagged as ‘low-quality’ according to the QFLAGs (Table 3).

References

1. Implementation Plan for the Global Observing System for Climate in Support of the United Nations Framework Convention on Climate Change (UNFCCC). GCOS-Nº 92. 2004. Available online: https://library.wmo.int/index.php?lvl=notice_display&id=6678#.Xowv2-oza5t (accessed on 7 April 2020).

2. Copernicus Climate Change Service (C3S). Available online: https://climate.copernicus.eu/ (accessed on 7 April 2020).
3. Wolters, E.; Dierckx, W.; Iordache, M.-D.; Swinnen, E. PROBA-V Products User Manual Document v3.01. Available online: http://proba-v.vgt.vito.be/sites/proba-v.vgt.vito.be/files/products_user_manual.pdf (accessed on 7 April 2020).

4. Trigo, I.F.; Dacamara, C.C.; Viterbo, P.; Roujean, J.-L.; Olesen, F.; Barroso, C.; Camacho-de-Coca, F.; Carrer, D.; Freitas, S.C.; García-Haro, J.; et al. The Satellite Application Facility for Land Surface Analysis. *Int. J. Remote Sens.* **2011**, *32*, 2725–2744. [CrossRef]

5. Land Surface Analysis (LSA-SAF) of EUMETSAT. Available online: https://landsaf.ipma.pt/en/ (accessed on 8 April 2020).

6. Geiger, B.; Carrer, D.; Franchistéguy, L.; Roujean, J.L.; Meurey, C. Land surface albedo derived on a daily basis from meteosat second generation observations. *IEEE Trans. Geosci. Remote Sens.* **2008**, *46*, 3841–3856. [CrossRef]

7. Carrer, D.; Roujean, J.L.; Meurey, C. Comparing operational MSG/SEVIRI Land Surface albedo products from Land SAF with ground measurements and MODIS. *IEEE Trans. Geosci. Remote Sens.* **2010**, *48*, 1714–1728. [CrossRef]

8. Carrer, D.; Roujean, J.-L.; Hautecoeur, O.; Elias, T. Daily estimates of aerosol optical thickness over land surface based on a directional and temporal analysis of SEVIRI MSG visible observations. *J. Geophys. Res.* **2010**, *115*, D10208. [CrossRef]

9. Carrer, D.; Moparthy, S.; Lellouch, G.; Ceamanos, X.; Pinault, F.; Freitas, S.C.; Trigo, I.F. Land surface albedo derived on a ten daily basis from Meteosat Second Generation Observations: The NRT and climate data record collections from the EUMETSAT LSA SAF. *Remote Sens.* **2018**, *10*, 1262. [CrossRef]

10. Copernicus Global Land Service (CGLS) Portal. Available online: https://land.copernicus.eu/global/index.html (accessed on 8 April 2020).

11. Climate Data Store of Copernicus Climate Change Service. Available online: https://cds.climate.copernicus.eu/#!/home (accessed on 10 May 2020).

12. Nightingale, J.; Mittaz, J.P.D.; Douglas, S.; Dee, D.; Ryder, J.; Taylor, M.; Old, C.; Dieval, C.; Fouron, C.; Duveau, G.; et al. Ten Priority Science Gaps in Assessing Climate Data Record Quality. *Remote Sens.* **2019**, *11*, 986. [CrossRef]

13. Justice, C.; Belward, A.; Morisette, J.; Lewis, P.; Privette, J.; Baret, F. Developments in the'validation'of satellite sensor products for the study of the land surface. *Int. J. Remote Sens.* **2000**, *21*, 3383–3390. [CrossRef]

14. Zeng, Y.; Su, Z.; Calvet, J.C.; Manninen, T.; Swinnen, E.; Schulz, J.; Roebeling, R.; Poli, P.; Tan, D.; Riihelä, A.; et al. Analysis of current validation practices in Europe for space-based climate data records of essential climate variables. *Int. J. Appl. Earth Obs. Geoinf.* **2015**, *42*, 150–161. [CrossRef]

15. Liang, S.; Fang, H.; Chen, M.; Shuey, C.J.; Walthall, C.; Daughtry, C.; Morisette, J.; Schaaf, C.; Strahler, A. Validating MODIS land surface reflectance and albedo products: Methods and preliminary results. *Remote Sens. Environ.* **2002**, *83*, 149–162. [CrossRef]

16. Cescatti, A.; Marcolla, B.; Santhana Vannan, S.K.; Pan, J.Y.; Román, M.O.; Yang, X.; Ciais, P.; Cook, R.B.; Law, B.E.; Matteucci, G.; et al. Intercomparison of MODIS albedo retrievals and in situ measurements across the global FLUXNET network. *Remote Sens. Environ.* **2012**, *121*, 323–334. [CrossRef]

17. Liu, Q.; Wang, L.; Qu, Y.; Liu, N.; Liu, S.; Tang, H.; Liang, S. Preliminary evaluation of the long-term GLASS albedo product. *Int. J. Digit. Earth* **2013**, *6*, 69–95. [CrossRef]

18. Liu, Y.; Wang, Z.; Sun, Q.; Erb, A.M.; Li, Z.; Schaaf, C.B.; Zhang, X.; Román, M.O.; Scott, R.L.; Zhang, Q.; et al. Evaluation of the VIIRS BRDF, Albedo and NBAR products suite and an assessment of continuity with the long term MODIS record. *Remote Sens. Environ.* **2017**, *201*, 256–274. [CrossRef]

19. Shuai, Y.; Schaaf, C.B.; Strahler, A.H.; Liu, J.; Jiao, Z. Quality assessment of BRDF/albedo retrievals in MODIS operational system. *Geophys. Res. Lett.* **2008**, *35*, 1–5. [CrossRef]

20. LPV (Land Product Validation). Subgroup CEOS Validation Hierarchy 2019. Available online: https://lpvs.gsfc.nasa.gov/ (accessed on 12 April 2020).

21. Wang, Z.; Schaaf, C.; Lattanzio, A.; Carrer, D.; Grant, I.; Roman, M.; Camacho, F.; Yang, Y.; Sánchez-Zapero, J. Global Surface Albedo Product Validation Best Practices Protocol. Version 1.0. In *Best Practice for Satellite Derived Land Product Validation* (p. 45): Land Product Validation Subgroup (WGCV/CEOS); Wang, Z., Nickeson, J., Román, M., Eds.; Available online: https://lpvs.gsfc.nasa.gov/PDF/CEOS_ALBEDO_Protocol_20190307_v1.pdf (accessed on 1 April 2020). [CrossRef]
22. Taberner, M.; Pinty, B.; Govaerts, Y.; Liang, S.; Verstraete, M.M.; Gobron, N.; Widlowski, J.L. Comparison of MISR and MODIS land surface albedos: Methodology. *J. Geophys. Res. Atmos.* **2010**, *115*, 1–13. [CrossRef]

23. Qu, Y.; Liu, Q.; Liang, S.; Wang, L.; Liu, N.; Liu, S. Direct-estimation algorithm for mapping daily land-surface broadband albedo from modis data. *IEEE Trans. Geosci. Remote Sens.* **2014**, *52*, 907–919. [CrossRef]

24. Sütterlin, M.; Schaaf, C.B.; Stöckli, R.; Sun, Q.; Hüsler, F.; Neuhaus, C.; Wunderle, S. Albedo and reflectance anisotropy retrieval from AVHRR operated onboard NOAA and MetOp satellites: Algorithm performance and accuracy assessment for Europe. *Remote Sens. Environ.* **2015**, *168*, 163–176. [CrossRef]

25. Fell, F.; Bennartz, R.; Loew, A. Validation of the EUMETSAT Geostationary Surface Albedo Climate Data Record -2- (ALBEDOVAL-2). Available online: https://www.eumetsat.int/website/home/Data/TechnicalDocuments/index.html (accessed on 12 April 2020).

26. Camacho, F.; Sánchez-Zapero, J. Quality Assessment Report SPOT/VGT Surface Albedo V1 (I1.10). Gio Global Land Component - Lot I "Operation of the Global Land Component" Framework Service Contract N° 388533 (JRC). Available online: https://land.copernicus.eu/global/sites/cgls.vito.be/files/products/GIOGL1_VR_SAV1_I1.10.pdf (accessed on 25 March 2020).

27. Sánchez-Zapero, J.; de la Madrid, L.; Camacho, F. Validation Report of Surface Albedo (SA) from PROBA-V Collection 1km Version 1.5 (I2.21). Copernicus Global Land Operations CGLOPS-1 (Framework Service Contract N° 199494 - JRC). Available online: https://land.copernicus.eu/global/sites/cgls.vito.be/files/products/CGLOPS1_VR_SA1km-PROBAV-V1.5_I2.21.pdf (accessed on 9 April 2020).

28. Sánchez-Zapero, J.; de la Madrid, L.; Camacho, F. Quality Assessment Report Normalized TOC-r from PROBA-V Collection 1 km Version 1.5 (I2.21). Copernicus Global Land Operations "Vegetation and Energy" "CGLOPS-1" Framework Service Contract N° 199494 (JRC). Available online: https://land.copernicus.eu/global/sites/cgls.vito.be/files/products/CGLOPS1_QAR_TOCR1km-PROBAV-V1.5_I2.21.pdf (accessed on 11 August 2020).

29. Mayr, S.; Kuenzer, C.; Gessner, U.; Klein, I.; Rutzinger, M. Validation of Earth Observation Time-Series: A Review for Large-Area and Temporally Dense Land Surface Products. *Remote Sens.* **2019**, *11*, 2616. [CrossRef]

30. Lewis, P.; Barnsley, M. Influence of the sky radiance distribution on various formulations of the earth surface albedo. In Proceedings of the 6th International Symposium on Physical Measurements and Signatures in Remote Sensing (ISPRS), Val d'Isere, France, 17–21 January 1994; pp. 707–715.

31. Román, M.O.; Schaaf, C.B.; Woodcock, C.E.; Strahler, A.H.; Yang, X.; Braswell, R.H.; Curtis, P.S.; Davis, K.J.; Dragoni, D.; Goulden, M.L.; et al. The MODIS (Collection V005) BRDF/albedo product: Assessment of spatial representativeness over forested landscapes. *Remote Sens. Environ.* **2009**, *113*, 2476–2498. [CrossRef]

32. Román, M.O.; Schaaf, C.B.; Lewis, P.; Gao, F.; Anderson, G.P.; Privette, J.L.; Strahler, A.H.; Woodcock, C.E.; Barnsley, M. Assessing the coupling between surface albedo derived from MODIS and the fraction of diffuse skylight over spatially-characterized landscapes. *Remote Sens. Environ.* **2010**, *114*, 738–760. [CrossRef]

33. Wang, Z.; Schaaf, C.B.; Chopping, M.J.; Strahler, A.H.; Wang, J.; Román, M.O.; Rocha, A.V.; Woodcock, C.E.; Shuai, Y. Evaluation of Moderate-resolution Imaging Spectroradiometer (MODIS) snow albedo product (MCD43A) over tundra. *Remote Sens. Environ.* **2012**, *117*, 264–280. [CrossRef]

34. Wang, Z.; Schaaf, C.B.; Strahler, A.H.; Chopping, M.J.; Román, M.O.; Shuai, Y.; Woodcock, C.E.; Hollinger, D.Y.; Fitzjarrald, D.R. Evaluation of MODIS albedo product (MCD43A) over grassland, agriculture and forest surface types during dormant and snow-covered periods. *Remote Sens. Environ.* **2014**, *140*, 60–77. [CrossRef]

35. Carrer, D.; Ceamanos, X.; Pinault, F.; Benhadj, I.; Toté, C. Algorithm Theoretical Basis Document (ATBD) of PROBA-V CDR and ICDR Surface Albedo v1.0 (Official Reference Number Service Contract: 2018/C3S_312b_Lot5_VITO/SC1). Available online: https://cds.climate.copernicus.eu/cdsapp#!/dataset/satellite-albedo?tab=doc (accessed on 9 April 2020).

36. Carrer, D.; Pinault, F.; Ramon, D.; Benhadj, I.; Swinnen, E. Algorithm Theoretical Basis Document (ATBD) of CDR SPOT/VGT Surface Albedo v1.0 (Official Reference Number Service Contract: 2018/C3S_312b_Lot5_VITO/SC1). Available online: https://cds.climate.copernicus.eu/cdsapp#!/dataset/satellite-albedo?tab=doc (accessed on 9 April 2020).

37. Schaaf, C.B.; Gao, F.; Strahler, A.H.; Lucht, W.; Li, X.; Tsang, T.; Strugnell, N.C.; Zhang, X.; Jin, Y.; Muller, J.P.; et al. First operational BRDF, albedo nadir reflectance products from MODIS. *Remote Sens. Environ.* **2002**, *83*, 135–148. [CrossRef]

38. Francois, M.; Santandrea, S.; Mellab, K.; Vrancken, D.; Versluys, J. The PROBA-V mission: The space segment. *Int. J. Remote Sens.* **2014**, *35*, 2548–2564. [CrossRef]
39. Dierckx, W.; Sterckx, S.; Benhadj, I.; Livens, S.; Duhoux, G.; Van Achteren, T.; Francois, M.; Mellab, K.; Saint, G. PROBA-V mission for global vegetation monitoring: Standard products and image quality. *Int. J. Remote Sens.* **2014**, *35*, 2589–2614. [CrossRef]

40. PROBA-V Products Data Access. Available online: http://proba-v.vgt.vito.be/en/product-types (accessed on 8 April 2020).

41. Carrer, D.; Benhadj, I. Product User Guide and Specification (PUGS) of PROBA-V CDR and ICDR Surface Albedo v1.0 (Official Reference Number Service Contract: 2018/C3S_312b_Lot5_VITO/SC1). Available online: https://cds.climate.copernicus.eu/cdsapp#/dataset/satellite-albedo?tab=doc (accessed on 9 April 2020).

42. Roujean, J.-L.; Leroy, M.; Deschamps, P.-Y. A bidirectional reflectance model of the Earth's surface for the correction of remote sensing data. *J. Geophys. Res.* **1992**, *97*, 20455–20468. [CrossRef]

43. Carrer, D.; Smets, B.; Ceamanos, X.; Roujean, J.-L. SPOT/VEGETATION and PROBA-V Surface Albedo Products—1 Km Version 1; Algorithm Theoretical Basis Document (ATBD), Issue 2.11. Copernicus Global Land Operations CGLOPS-1 (Framework Service Contract N° 199494-JRC). Available online: https://land.copernicus.eu/global/sites/cgls.vito.be/files/products/CGLOPS1_ATBD_SA1km-V1_I2.11.pdf (accessed on 10 May 2020).

44. Toté, C.; Swinnen, E.; Sterckx, S.; Clarijs, D.; Quang, C.; Maes, R. Evaluation of the SPOT/VEGETATION Collection 3 reprocessed dataset: Surface reflectances and NDVI. *Remote Sens. Environ.* **2017**, *201*, 219–233. [CrossRef]

45. Lucht, W.; Schaaf, C.B.; Strahler, A.H. An algorithm for the retrieval of albedo from space using semiempirical BRDF models. *IEEE Trans. Geosci. Remote Sens.* **2000**, *38*, 977–998. [CrossRef]

46. Sanchez-Zapero, J. Product Quality Assessment Report (PQAR) of CDR SPOT/VGT-Based Surface Albedo v1.0 (Official Reference Number Service Contract: 2018/C3S_312b_Lot5_VITO/SC1). Available online: https://cds.climate.copernicus.eu/cdsapp#/dataset/satellite-albedo?tab=doc (accessed on 9 April 2020).

47. Schaaf, C.; Wang, Z. MCD43A3 MODIS/Terra+Aqua BRDF/Albedo Daily L3 Global-500m V006 [Data set]. NASA EOSDIS Land Processes DAAC. 2015. Available online: https://doi.org/10.5067/MODIS/MCD43A3.006 (accessed on 10 April 2020).

48. Lucht, W.; Lewis, P. Theoretical noise sensitivity of BRDF and albedo retrieval from the EOS-MODIS and MISR sensors with respect to angular sampling. *Int. J. Remote Sens.* **2000**, *21*, 81–98. [CrossRef]

49. Sun, Q.; Wang, Z.; Li, Z.; Erb, A.; Schaaf, C.B. Evaluation of the global MODIS 30 arc-second spatially and temporally complete snow-free land surface albedo and reflectance anisotropy dataset. *Int. J. Appl. Earth Obs. Geoinf.* **2017**, *58*, 36–49. [CrossRef]

50. Liang, S.; Strahler, A.H.; Walthall, C. Retrieval of Land Surface Albedo from Satellite Observations: A Simulation Study. *J. Appl. Meteorol. *1999*, *38*, 712–725. [CrossRef]

51. Wang, Z.; Schaaf, C.B.; Sun, Q.; Shuai, Y.; Román, M.O. Capturing rapid land surface dynamics with Collection V006 MODIS BRDF/NBAR/Albedo (MCD43) products. *Remote Sens. Environ.* **2018**, *207*, 50–64. [CrossRef]

52. Roujean, J.L.; Leon-Tavares, J.; Smets, B.; Claes, P.; Camacho De Coca, F.; Sanchez-Zapero, J. Surface albedo and toc-r 300 m products from PROBA-V instrument in the framework of Copernicus Global Land Service. *Remote Sens. Environ.* **2018**, *215*, 57–73. [CrossRef]

53. Ground-Based Observations for Validation (GBOV) of Copernicus Global Land Products Site. Available online: https://land.copernicus.eu/global/gbov (accessed on 1 April 2020).

54. Song, R.; Muller, J.-P.; Kharbouche, S.; Woodgate, W. Intercomparison of Surface Albedo Retrievals from MISR, MODIS, CGLS Using Tower and Upscaled Tower Measurements. *Remote Sens.* **2019**, *11*, 644. [CrossRef]

55. Kharbouche, S.; Song, R.; Muller, J.-P. Algorithm Theoretical Basis Document of Energy products: RM1 (short wave radiation), LP1 (Top Of Canopy Reflectance), LP2 (Albedo). Ground-Based Observations for Validation (GBOV) of CGLS Products (Framework Contract reference: 932059-JRC). Available online: https://gbov.acri.fr/public/docs/products/2019-11/GBOV-ATBD-RM1-LP1-LP2_v1.3-Energy.pdf (accessed on 10 April 2020).

56. Song, R.; Muller, J.P.; Kharbouche, S.; Yin, F.; Woodgate, W.; Kitchen, M.; Roland, M.; Arriga, N.; Meyer, W.; Koerber, G.; et al. Validation of space-based albedo products from upscaled tower-based measurements over heterogeneous and homogeneous landscapes. *Remote Sens.* **2020**, *12*, 833. [CrossRef]
57. Reda, I. Method to calculate uncertainties in measuring shortwave solar irradiance using thermopile and semiconductor solar radiometers, Tech. Rep. NREL/TP-3B10-52194, 20 pp., Natl. Renewable Energy Lab., Golden, Colo. 2011. Available online: http://www.osti.gov/servlets/purl/1021250/ (accessed on 5 August 2020). [CrossRef]

58. Hohn, M.E. An Introduction to Applied Geostatistics: By Edward H. Isaaks and R. Mohan Srivastava, 1989, Oxford University Press, New York, 561 p., ISBN 0-19-505012-6, ISBN 0-19-505013-4. *Comput. Geosci.* **1991**, *17*, 471–473. [CrossRef]

59. Carroll, S.S.; Cressie, N. A COMPARISON OF GEOSTATISTICAL METHODOLOGIES USED TO ESTIMATE SNOW WATER EQUIVALENT. *J. Am. Water Resour. Assoc.* **1996**, *32*, 267–278. [CrossRef]

60. Matheron, G. Principles of geostatistics. *Econ. Geol.* **1963**, *58*, 1246–1266. [CrossRef]

61. Woodcock, C.E.; Strahler, A.H.; Jupp, D.L.B. The use of variograms in remote sensing: I. Scene models and simulated images. *Remote Sens. Environ.* **1988**, *25*, 323–348. [CrossRef]

62. Woodcock, C.E.; Strahler, A.H.; Jupp, D.L.B. The use of variograms in remote sensing: II. Real digital images. *Remote Sens. Environ.* **1988**, *25*, 349–379. [CrossRef]

63. Wang, Z.; Schaaf, C.B.; Sun, Q.; Kim, J.H.; Erb, A.M.; Gao, F.; Román, M.O.; Yang, Y.; Petroy, S.; Taylor, J.R.; et al. Monitoring land surface albedo and vegetation dynamics using high spatial and temporal resolution synthetic time series from Landsat and the MODIS BRDF/NBAR/albedo product. *Int. J. Appl. Earth Obs. Geoinf.* **2017**, *59*, 104–117. [CrossRef]

64. Systematic Observation Requirements for Satellite-Based Data Products for Climate. Supplemental Details to the Satellite-Based Component of the “Implementation Plan for the GCOS in Support of the UNFCCC”. [GCOS-154, 2011 Update]. Available online: https://library.wmo.int/doc_num.php?explnum_id=3710 (accessed on 10 April 2020).

65. World Meteorological Organization (WMO) Requirements for Earth Surface Albedo. Available online: https://www.wmo-sat.info/oscar/variables/view/54 (accessed on 10 April 2020).

66. Sánchez-Zapero, J.; Camacho, F. Product Quality Assessment Report (PQAR) of CDR and ICDR Surface Albedo v1.0 Based on PROBA-V (Official Reference Number Service Contract: 2018/C3S_312b_Lot5_VITO/SC1). Available online: https://cds.climate.copernicus.eu/cdsapp#!/dataset/satellite-albedo?tab=doc (accessed on 9 April 2020).

67. Fuster, B.; Sánchez-Zapero, J.; Camacho, F.; García-Santos, V.; Verger, A.; Lacaze, R.; Weiss, M.; Baret, F.; Smets, B. Quality Assessment of PROBA-V LAI, fAPAR and fCOVER Collection 300 m Products of Copernicus Global Land Service. *Remote Sens.* **2020**, *12*, 1017. [CrossRef]

68. Joint Committee for Guides in Metrology (JCGM)-Guides to the Expression of Uncertainty in Measurement (GUM). [ISO/IEC Guide 98-Part 3, 2008]. Available online: https://www.iso.org/sites/JCGM/GUM-introduction.htm (accessed on 10 April 2020).

69. Weiss, M.; Baret, F.; Garrigues, S.; Lacaze, R. LAI and fAPAR CYCLOPES global products derived from VEGETATION. Part 2: Validation and comparison with MODIS collection 4 products. *Remote Sens. Environ.* **2007**, *110*, 317–331. [CrossRef]

70. Sánchez, J.; Camacho, F.; Lacaze, R.; Smets, B. Early validation of PROBA-V GEOV1 LAI, FAPAR and FCOVER products for the continuity of the copernicus global land service. *Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci.-ISPRS Arch.* **2015**, *40*, 93–100. [CrossRef]

71. Harper, W.V. Reduced Major Axis regression: Teaching alternatives to Least Squares. In Proceedings of the 9th International Conference on Teaching Statistics (ICOTS-9), Flagstaff, AZ, USA, 13–18 July 2014; pp. 1–4. Available online: https://digitalcommons.otterbein.edu/math_fac/24 (accessed on 11 August 2020).

72. Iannone, R.Q.; Niro, F.; Goryl, P.; Dransfeld, S.; Hoersch, B.; Stelzer, K.; Kirches, G.; Paperin, M.; Brockmann, C.; Gomez-Chova, L.; et al. Proba-V cloud detection Round Robin: Validation results and recommendations. In Proceedings of the 2017 9th International Workshop on the Analysis of Multitemporal Remote Sensing Images, MultiTemp 2017, Brugge, Belgium, 27–29 June 2017; pp. 1–8. Available online: https://ieeexplore.ieee.org/document/8035219 (accessed on 11 August 2020). [CrossRef]

73. Maignan, F.; Bréon, F.M.; Lacaze, R. Bidirectional reflectance of Earth targets: Evaluation of analytical models using a large set of spaceborne measurements with emphasis on the Hot Spot. *Remote Sens. Environ.* **2004**, *90*, 210–220. [CrossRef]

74. Liu, S.; Liu, Q.; Liu, Q.; Wen, J.; Li, X. The Angular and Spectral Kernel Model for BRDF and Albedo Retrieval. *IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens.* **2010**, *3*, 241–256. [CrossRef]
75. Chen, J.M.; Cihlar, J. A hotspot function in a simple bidirectional reflectance model for satellite applications. *J. Geophys. Res. Atmos.* **1997**, *102*, 25907–25913. [CrossRef]

76. Wu, A.; Li, Z.; Cihlar, J. Effects of land cover type and greenness on advanced very high resolution radiometer bidirectional reflectances: Analysis and removal. *J. Geophys. Res.* **1995**, *100*, 9179–9192. [CrossRef]

77. Jin, Y.; Schaaf, C.B.; Woodcock, C.E.; Gao, F.; Li, X.; Strahler, A.H.; Lucht, W.; Liang, S. Consistency of MODIS surface bidirectional reflectance distribution function and albedo retrievals: 1. Validation. *J. Geophys. Res. D Atmos.* **2003**, *108*, 1–15. [CrossRef]

78. Sanchez-Zapero, J. Scientific Quality Evaluation (SQE) of PROBA-V Surface Albedo (SA) Collection 1 km Version 1.5 (I2.00). Copernicus Global Land Operations CGLOPS-1 (Framework Service Contract N° 199494-JRC). Available online: https://land.copernicus.eu/global/sites/cglsvito.be/files/products/CGLOPS1_SQE2017_SA1km-V1_I1.00.pdf (accessed on 9 April 2020).

79. Salomon, J.G.; Schaaf, C.B.; Strahler, A.H.; Gao, F.; Jin, Y. Validation of the MODIS Bidirectional Reflectance Distribution Function and albedo retrievals using combined observations from the Aqua and Terra platforms. *IEEE Trans. Geosci. Remote Sens.* **2006**, *44*, 1555–1564. [CrossRef]

<figure><img src="image_17.png" /></figure>

© 2020 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).