dataesr/scientific-paragraphs-categorization huggingface dataset:
The notion of a multi-mission spatiotemporal dataset is now widely democratized in the Earth Observation (EO) community. Data providers have been delivering dedicated products, called Analysis Ready Data (ARD,  [5] ), following the Committee on Earth Observation Satellites (CEOS) Analysis Ready Data for Land (CARD4L) guidance for some of them  [6]. These products are ready to be employed within EO Data Cubes (EODC) implementations such as Digital Earth Australia, the Swiss Data Cube, the EarthServer, the E-sensing platform or the Google Earth Engine  [7]. The handling of multi-mission MS data has several challenges due to the differences between sensor characteristics, product definitions and observation geometries that have a direct impact on the consistency and accuracy of spatiotemporal datasets.
The main aim of this study is to propose a new algorithm for the harmonization and fusion of Landsat 8/9 and S2 MS products to increase the temporal resolution and provide a solution for continuous Earth surface monitoring. A software tool, namely Sen2Like (S2L), has been developed to implement the algorithm and to aid the VC establishment activities within EU Copernicus Program and funded by ESA/EU. The baseline principle of Sen2Like (  [12] ) was to consider S2 as a reference mission in terms of the instrument, product accuracy specification  ([13] ) and product format and to combine the S2 data stream with any other data sources delivered by MS missions with spectral characteristics similar to S2 sensors. The final objective is thus to create a Virtual Constellation (VC) as previously discussed by Wulder et al.
[14]. The S2L is open source and developed in the Python programming language. The S2L methodology analyzes the user products that are stored locally or available at the Copernicus Data and Information Access Service (DIAS) cloud environment and generates the output ARD dataset based on various user configurations such as temporal period, geographical footprint and cloud cover. The final S2L ARD format complies with the CARD4L guidelines  [6]  in terms of general and per pixel metadata information. The S2L implementation involves a data harmonization protocol, and for this purpose, a processing framework is proposed. For any input mission, the processing sequence remains unchanged, including various methods dedicated to geometric and atmospheric issues, normalization of reflectance values, spectral band pass correction and downscaling.
Geometric corrections including the co-registration to a reference image; iii. Inter-calibration (for S2 L1C); iv. Atmospheric corrections; v. Transformation to Nadir Bidirectional Reflectance Distribution Function (BRDF) Adjusted Reflectance (NBAR); vi. Application of Spectral Band Adjustment Factor (SBAF) (for LS8/9); vii. Production of LS8/LS9 high resolution 10 m pixel spacing data (data fusion). As a result, an L2 Harmonized (L2H) or L2 Fused (L2F) ARD dataset matching the Region of Interest (ROI) selected is generated. Regarding tiling, the S2L data are framed in tiles with a size of 110 km × 110 km in the Universal Transverse Mercator (UTM)-based Military Grid Reference System (MGRS,  [15] ). If the ROI extent covers several MGRS tiles, all of them are processed. As a result, an L2 Harmonized (L2H) or L2 Fused (L2F) ARD dataset matching the Region of Interest (ROI) selected is generated.
Sen2Like Software Design Elements The S2L software is designed for on demand processing purposes with major requirements set on near real time production and the provision of harmonized data in a timely manner. In addition, any existing S2L spatiotemporal dataset can be regularly and quickly updated along with the new acquisition. Furthermore, the design enables researchers to configure the software depending on the accuracy and timeliness requirements and the processing system specification. By using a minimal configuration (e. g. , local computer, basic processing), the generation of a single S2L product takes approximately 4-5 min. The S2L framework was developed in the Python 3. 7 programming environment, and the implementation utilizes recent data science libraries such as Numpy, Pandas, Xarray, Scipy, Scikit-Learn, etc.
Product Description and Auxiliary Data The Sen2Like software ingests a stack of L1/L2 S2/LS8 and LS9 products (available from the Copernicus Access Hub  [16] , ESA LS8 web portal  [17] ) and provides two categories of the ARD dataset: i. e. , the L2F and the L2H dataset. Both L2H and L2F products are processed in Standard Archive Format for Europe (SAFE)  [18]  and embed Cloud Optimized (COG) GeoTIFF images  [19]. Furthermore, product metadata are also described with a dedicated Spatio-Temporal Asset Catalogs (STAC,  [20] ) JSON file to expose data. The major difference between the two L2H and L2F product types resides in the spatial resolution of the output images; in the L2H products, the native spatial resolutions of input images are preserved, whilst in the L2F products, the resolutions of the LS8/LS9 image data are upsampled to the pixel spacing of the relevant S2 band.
Geometric Correction The proposed S2L geometric correction methodology is basically categorized into four main components, which are (i) image stitching (mosaicking), (ii) MGRS reframing, (iii) geometric co-registration and (iv) quality control. The stitching and the MGRS reframing steps are essential to ensure full coverage of MGRS frame depending on L1/L2 product footprint conventions. A second goal of the S2L is to co-register all input images with respect to a raster reference. In this way, the geometrical consistency of the spatiotemporal dataset reaches an accuracy goal of 0. 3 pixel (10 m) at 2-sigma depending on the radiometric quality and similarity of the data. The proposed S2L methodology for co-registration relies on dense image matching between search and reference images using the KLT (Kanade Lucas-Tomasi) algorithm  [22, 23]  implemented in the OpenCV  [24, 25]  library in Python.
The feature points extracted with the Good Features algorithm of Shi and Tomasi  [26]  are employed in the process to increase the matching success and the reliability. This methodology was utilized in earlier projects  [27] [28] [29] [30] [31] [32]  demonstrating that computer vision technics are beneficial for EO data processing. As shown in Figure  2 , the geometric assessment results are obtained from the matching between near infrared (NIR) band images of both sensors and then extrapolated to all other bands for correction purposes. (Kanade Lucas-Tomasi) algorithm  [22, 23]  implemented in the OpenCV  [24, 25]  library in Python. The feature points extracted with the Good Features algorithm of Shi and Tomasi  [26]  are employed in the process to increase the matching success and the reliability.
e. , OLI, MSI-A and MSI-B). The parameters in the observation period, such as the atmospheric sea level pressure, the total aerosol optical depth at 550 nm, the ozone content and the water vapor content data, are obtained from the Atmosphere Data Store (ADS) maintained by CAMS  [21] , which are provided with a resolution of approximately 40 km. The meteorological conditions are assumed to be stable over the geographical extent of an S2 MGRS tile footprints. In addition, the atmospheric parameter set matching the MGRS tile scene center and product observation date/time is computed. Furthermore, the solar and viewing angle values are handled differently depending on their availability in the L1/L2 product format. A fall-back option consists of using FMASK  [35]  routines to calculate them. Obtained from their original grids, the viewing and solar viewing angles are upsampled to a finer grid, depending on the resolution of target image grid.
The BELMANIP2 sites are an update of the BELMANIP1 and were selected due to their representativeness and variability of vegetation types and climatological conditions around the world. Second, the surface reflectance data time series (2002-2020) were sorted in ascending values of the NDVI, which were divided into five groups, using the 20th, 40th, 60th and 80th percentiles as group edge values. Third, we extracted the V and R values of each group and spectral band. This means that for every pixel and every band, we obtained five different V and R values. When using all the BELMANIP2 sites, we obtained a total of 2225 points (445 × 5) for each band. Finally, we applied simple regression between the obtained parameters for the VNIR bands and with the NDVI itself, which is summarized in Equation  (5).
Additionally, the low quality of the cloud mask may also introduce noise in the BRDF unmixing. Therefore, based on the individual BRDF unmixing of each image for a given time period, we apply a linear regression for each pixel of V and R parameters versus the NDVI to derive the V 0 , V 1 , R 0 and R 1 parameters at S2L, which are stored as BRDF auxiliary data as the final output of this module. = 0. 91 ×  + 0. 34 ×  The method was evaluated by assessing the impact of the approach on the normalized reflectance. To do so, in Figure  6 , we compare the time series of noise (2002-2020) of the directional reflectance (raw) with the normalized reflectance using the VJB method in HABA and using the model prediction. The results show that the VJB parameters and the predicted parameters provide a very similar reduction of noise.
Rather than considering a single land cover, the NASA HLS team addressed the SBAF issue  [8]  by analyzing MSI/OLI SBAF variability for different biome types. So far, a total of 160 globally distributed Hyperion hyperspectral scenes have been processed to SR, and 500 spectra were analyzed. The study revealed the importance of discerning the differences sourced from the spectral calibration from those coming from the radiometric calibration. Furthermore, it also showed that in the context of S2/LS8, the residual errors associated to SBAF adjustment make a very small contribution in the overall error budget. In addition, S2L adjusts LS8, LS9 and S2B reflectances to S2A reflectance by using univariate linear regression and a single set of coefficients per band. The band-dependent linear transformation factors (slope, intercept) given in  [48]  for distinguishing S2A and S2B data were used here.
Validity Mask The objective of the S2L per pixel validity mask is to support the multi-temporal analysis of the L2H/L2F dataset by flagging invalid pixels caused by clouds, cloud shadows, image processing artifacts, etc. In general, it is expected that such quality assurance information is already available within the L1/L2 products. However, depending on the product version and the data provider, the reliability and completeness of this information is varying. For instance, the accuracies of S2 L1C and L2A validity masks are different as depicted in Figure  7. To mitigate the inconsistencies, a morphological operator (dilation) is applied to S2 L1C quality masks in order to increase their quality by excluding the cloud shadow and haze (Figure  7c ) from a valid pixel sample size. univariate linear regression and a single set of coefficients per band.
Validity Mask The objective of the S2L per pixel validity mask is to support the multi-temporal an ysis of the L2H/L2F dataset by flagging invalid pixels caused by clouds, cloud shado image processing artifacts, etc. In general, it is expected that such quality assurance inf mation is already available within the L1/L2 products. However, depending on the pr uct version and the data provider, the reliability and completeness of this information varying. For instance, the accuracies of S2 L1C and L2A validity masks are different depicted in Figure  7. To mitigate the inconsistencies, a morphological operator (dilati is applied to S2 L1C quality masks in order to increase their quality by excluding the clo shadow and haze (Figure  7c ) from a valid pixel sample size. For Landsat products, the validity mask is constructed from the LS8/LS9 Band Quality Assessment as described in  [54, 55] , whereas for S2 products, the validity mask is constructed from the L1C cloud mask (either in vector or raster format) and from L2A Scene Classification (SCL) map  [56].
Data Fusion The main objective of the S2L data fusion is to produce data similar to S2 at the observation date/time of LS8/LS9 observations. The approach is based on improving the spatial resolution of LS8/LS9 images from 30 m to 10 m in the case of VNIR bands while preserving radiometric inter-calibration between sensors. Increasing the temporal frequency of S2 HR data is fundamental for many applications dealing with the mapping of rapid changes, as discussed in  [58]. The study reported a fusion method to generate Landsat-like images by using MODIS data. Besides this, in the context of monitoring, developing a fusion method for enabling change detection in a timely manner is of great importance for a near real time alert system. The near real time approach requires the improvement of the spatial resolution of a given LS8/LS9 image by exclusively using S2 images observed in the past.
L8 is the original LS8 image resampled from 30 m to 10 m by using bilinear interpolation, which is associated to the phase of signal, • D 10m S2-L8 is the image of differences derived between 30 m and 10 m spatial resolution predicted by using S2 data and associated with the amplitude of the signal. The essence of this method is the computation of images of differences. The prediction is performed based on the historical S2 information assuming a linear relationship between the observations. A linear regression model with least squares fitting is used to estimate the transformation parameters. The selection of multi date pixels in the S2 past dataset is based on validity mask information, which needs to be reliable to obtain high performance.
Validation Results The geometric accuracy performances of the S2 and LS8/9 L1C/L2A products are well known. The S2 Mission Performance Center (MPC) presents the status of products monthly in the data quality reports  ([13] ). The NASA science team releases also regular reports. However, an example from the geometric accuracy results, before and after the application of S2L co-registration, is shown in Figure  9. The geometric co-registration of images with respect to a raster reference is demonstrated with two graphics: circular error plots (left) and multi-temporal plots (right). A three year period dataset observed in 31TFJ MGRS tile was used as reference. The upper graphics are related to the initial geometric accuracy (relative to raster reference) and the lower plots are related to the final S2L accuracy in the figure.
The results were obtained from the comparison of 964 million cloud-free pixels for each date. The number of overlapping pixels varies between 9% and 36% for VZAs from 0 • to 2 • depending on the swath for each MGRS tile and the acquisition date. For VZAs between 2 • to 4 • , the percentage of overlapping data represented between 28% and 45% of the tiles. Additionally, for VZAs between 4 • and 6 • , the number of analyzed pixels represented between 13% and 25%. Finally, for the largest VZA observations (>6 • ), the analyzed pixels represent between 12% and 42%. plots (left) and multi-temporal plots (right). A three year period dataset observed in 31TFJ MGRS tile was used as reference. The upper graphics are related to the initial geometric accuracy (relative to raster reference) and the lower plots are related to the final S2L accuracy in the figure.
The comparison shows that the HABA produced the lowest differences across all bands, while the C-factor correction showed larger discrepancies than the directional product for all VZA pixels aggregated. Table  4  summarizes the metrics obtained from the analysis, demonstrating that the HABA correction decreases the absolute difference up to 13% (in the SWIR bands) compared to the directional product. As shown in the Table, the BRDF processing with the HABA brings substantial improvements regarding time series smoothness. However, as discussed before, this methodology is scene dependent. In addition, from the on-demand production perspective, it is not straightforward to argue that the HABA results systematically reach operational goals for any newly generated dataset. The Temporal Smoothness Index (TSI) is widely used in the community in order to assess the consistency of the spatiotemporal dataset, e.
0. 7) with a 300 m resolution  [61]  is used as supporting data to compute the image statistics over regional LC classes, keeping a valid pixel sample by using the S2L validity mask. The CCI LC map is compared with the TSI correlation extracted from the COG L2H overview images (120 m resolution). The TSI correlation metric results generated for the three methods, i. e. , DIR, C-Factor and the HABA for the 31TFJ multi temporal dataset, which includes 113 LS8/S2 observation (44 S2A, 40 S2B, 28 LS8) dates over the full-year 2019 period, are presented in Figure  12. Based on this experimental TSI correlation metric, it becomes evident that the C-Factor and the HABA methods reduce angular dependencies. Compared to the C-Factor, the HABA performs better in most cases: the correlation results are below 0.
01, except for Tree Cover (LCCS Value 60; 70, 90) and Bareland Areas (LCCS Value 200), in particular for S2L bands B03, B04 and B12. In addition, the results for B02 do not show improvements, even if the same tendency regarding the underperformance of HABA for the LC class previously mentioned is observed. Regarding the data fusion, from a quantitative point of view, we used accuracy precision uncertainty (APU) defined in  [62]  to compare datasets, as performed in  [37]. Thus, higher values of A, P and U reflect higher discrepancies between datasets, and low values reflect a better agreement. The LS8 L2H and L2F images for three bands (B02, B03, B04) were compared with respect to S2 L2H/L2F images. Data acquired at the same date were selected to eliminate the temporal differences. The results given in Table  5  show that the fusion algorithm does not modify the inter-calibration between LS8 and S2A (LS8/S2A) significantly, and the results are in agreement with the cross-calibration results given in  [53].
Moreover, this shows that the inter-calibration does not change depending on the S2L processing level, i. e. , L2H and L2F. Regarding the data fusion, from a quantitative point of view, we used accuracy precision uncertainty (APU) defined in  [62]  to compare datasets, as performed in  [37]. Thus, higher values of A, P and U reflect higher discrepancies between datasets, and low values reflect a better agreement. The LS8 L2H and L2F images for three bands (B02, B03, B04) were compared with respect to S2 L2H/L2F images. Data acquired at the same date were selected to eliminate the temporal differences. The results given in Table  5  show that the fusion algorithm does not modify the inter-calibration between LS8 and S2A (LS8/S2A) significantly, and the results are in agreement with the cross-calibration results given in  [53].
The S2L Framework The S2L algorithm allows the harmonization and fusion of multi-sensor optical data for the purpose of ARD preparation with the ultimate goal of generating VCs. The S2L tool developed for this purpose is able to apply geometric processing, spectral inter-cali bration and AC, BRDF and spectral adjustments and data fusion to ensure that the inpu products are at the same spatial resolution. The tool framework is flexible, and it is de signed to integrate with the future products of different spaceborne and airborne optica missions with the condition that the proper calibration information is available. The too development principle also involves reusability from the existing qualified tools such as SMAC, Sen2Cor and qualitied methods. The major achievements of the study include detailed investigations of the quality o a spatiotemporal dataset, algorithm comparison and the development of an ARD quality assurance approach.
Discussions 4. 1. The S2L Framework The S2L algorithm allows the harmonization and fusion of multi-sensor optical data for the purpose of ARD preparation with the ultimate goal of generating VCs. The S2L tool developed for this purpose is able to apply geometric processing, spectral intercalibration and AC, BRDF and spectral adjustments and data fusion to ensure that the input products are at the same spatial resolution. The tool framework is flexible, and it is designed to integrate with the future products of different spaceborne and airborne optical missions with the condition that the proper calibration information is available. The tool development principle also involves reusability from the existing qualified tools such as SMAC, Sen2Cor and qualitied methods. The major achievements of the study include detailed investigations of the quality of a spatiotemporal dataset, algorithm comparison and the development of an ARD quality assurance approach.
Adaptation to Further Optical Missions The S2L software offers a generic framework and enables the creation of a multi mission seamless dataset. The VC of the S2A/B and LS8/9 sensors already provides a global median average revisit of 2. 3 days  [3]. The inclusion of further missions increases the data availability, shortens the temporal revisit interval and maximizes cloud-free observations. Within this scope, the integration efforts to include further satellites with similar characteristics as well as hyperspectral missions and data obtained from airborne platforms including the UAVs are addressed in this section. It has previously been shown that the integration of Indian Remote Sensing (IRS)-ResourceSat-2 [63] in S2L requires minor development efforts, and the most critical issues remain in the knowledge of sensor radiometric and spectral calibration for which documentation is not disclosed.
The final aggregated TOA dataset can be processed with two different methods. The first method relies on the processing of the dataset in the same way as DESIS data relying only on VNIR bands (DESIS-S2-VNIR). The second method involves processing like S2 using the SWIR-bands (DESIS-S2) as well. Both methods provide similar results, showing no significant differences (Figure  15 ). The missing SWIR bands of the DESIS sensor do not generate artefacts for this study due to differences in processing. The aggregation of hyperspectral data to MS data on L1C Level opens up new perspectives for the integration of current (PRISMA  [66] , EnMAP  [67] ) and future hyperspectral (CHIME,  [68] ) missions into the MS framework. The main drawback of this approach is that it may not be possible to utilize the full potential of the hyperspectral data with the fusion process.
The convolution of hyperspectral BOA reflectance data to MS has several ad-vantages over aggregating hyperspectral data to MS at L1C for processing. The BOA spectra are smoother than the TOA spectra, which includes absorption features. From the computational and storage point of view, only one hyperspectral product will be stored, serving both the S2L and the hyperspectral L2A users, and fewer computation-al resources will be required to convolve the hyperspectral L2A products to any other MS missions, even when the SRFs of MS missions change. These results also confirm that new MS sensors with more spectral bands than S2 can be easily included in the algorithm (e. g. , S2 NG  [68] , L9  [69] ). be processed with two different methods. The first method relies on the processing of the dataset in the same way as DESIS data relying only on VNIR bands (DESIS-S2-VNIR).