dataesr/scientific-paragraphs-categorization huggingface dataset:
AMSR-E LPRM Soil Moisture The Land Parameter Retrieval Model (LPRM) is the baseline model for the soil moisture retrievals from the passive microwave sensors within the ESA ECV CCI soil moisture dataset  [42]. The SM retrievals from LPRM have been extensively validated  [12, 43] , and the model performance has been documented in numerous publications  [6, 44]. LPRM demonstrated a good soil moisture retrieval skill from AMSR-E observations over sparse to moderately-vegetated regions  [15, 45]. LPRM uses vertical and horizontal polarized T b 's as an input and uses a radiative transfer model to simultaneously solve for SM and vegetation optical depth  [6]. An advantage of LPRM is that it can be applied on different frequency bands, including the X, C and L bands. The LPRM SM dataset used in this study was obtained with an optimized single scattering albedo, a new roughness model and using both AMSR-E and SMOS observations  [23, 46].
AMSR-E Reg Soil Moisture The "AMSR-E Reg" SM dataset  [24]  has been obtained by computing local linear regressions linking the surface reflectivity for vertical and horizontal polarization and SMOS L3 SM using a set of three free parameters for each grid point. The local regression parameters were computed in the June 2010-October 2011 period (when both AMSR-E and SMOS were in operation), getting as many triplets of free parameters as land grid points with available AMSR-E T b 's and SMOS L3 SM data (∼1. 5 × 10 5 points). In a second step, those parameters are used to estimate SM from AMSR-E observations over the 2003-2010 period.
ESA Climate Change Initiative Soil Moisture The European Space Agency Climate Change Initiative (CCI) has supported the generation of a long-term SM product based on multiple microwave sensors in space, such as the radiometers (passive) SMMR (Scanning Multichannel Microwave Radiometer), SSM/I (Special Sensor Microwave Imager), Tropical Rainfall Measuring Mission's (TRMM) Microwave Imager (TMI), WindSat Polarimetric Radiometer, AMSR-E and AMSR-2 and the active sensors on board the European Remote Sensing (ERS) satellites (Wind Scatterometer) and the Metop satellites (Advanced Scatterometer, ASCAT)  [47]. Datasets derived from the different instruments are scaled into a common climatology by matching their cumulative distribution functions (CDF) to those of AMSR-E for the passive datasets or ASCAT for the active datasets.
Since the different satellite products are provided in different units, they are scaled into a common level using GLDAS-Noah model simulations. Finally, the active and passive datasets are merged into a combined single dataset. Over areas with low vegetation, the passive one is used, while over areas with moderate vegetation density, the active one is used. In transition areas, if both passive and active data are available, the average is used, otherwise the only available data are used for the final product. The CCI SM dataset used in this study is Version 2. 2 (an evaluation against in situ measurements can be found in  [9] ).
MERRA Land The Modern-Era Retrospective analysis for Research and Applications-Land (MERRA-Land)  [48]  is an atmospheric re-analysis that makes use of the Goddard Earth Observing System model Version 5 and the data assimilation system Version 5. 2. 0 with an updated precipitation forcing and catchment land surface model compared to the normal MERRA dataset. It was developed by NASA and is available from the beginning of 1980 to the present day. In the current study, the top 2-cm SM is used. The data have a temporal resolution of one hour and a spatial resolution of 1/2 • × 2/3 •.
ECMWF ERA-Land Models ECMWF model simulations of snow depth, soil temperature and soil moisture were used to filter the data and to compare with remote sensing soil moisture measurements. The data used in this study were extracted from a global Tiled ECMWF Scheme for Surface Exchanges over Land (TESSEL) with improved hydrology  [49]  simulation of surface parameters from 1979-2014. The land surface model has been forced with the atmosphere from the ERA-Interim re-analysis  [2]. This dataset will subsequently be referred to as ERA-Land and differs from the ERA-Interim/Land  [50]  only in the precipitation forcing, which was not rescaled to match Global Precipitation Climatology Project (GPCP) monthly means since GPCP Version 2. 1 was not available after 2010. A more recent version of GPCP could have been used, but it was decided to use the original ERA-Interim precipitations in this study.
Data Collocation and Filtering The acquisition time difference between AMSR-E and SMOS could be significant for a dynamic variable, such as SM. Therefore, only the night orbits (descending AMSR-E and ascending SMOS) were used to avoid significant differences in the surface SM due to evaporation and convective precipitation during daytime. Possible SM differences induced by precipitation taking place in between AMSR-E and SMOS overpasses were not taken into account, but they will appear in a limited number of points at the global level, and as shown by  [32] , this should not affect the training. The CCI, MERRA-Land and ERA-Land datasets were spatially interpolated to the EASEv2 grid by the area-overlap weighted average. The data have also been interpolated to the time of AMSR-E acquisitions by linear interpolation.
In Situ Measurements and Evaluation Protocol The AMSR-E NN SM dataset discussed in this paper, along with those presented above, was evaluated against in situ measurements for a large number of sites. All of the in situ data were obtained from the International Soil Moisture Network  [53]  except for the Danish Hydrological Observatory (HOBE) and the African Monsoon Multidisciplinary Analysis (AMMA), which have been directly provided by the teams managing the experimental sites. As the C-band radiation detected from space comes from the first few centimeters of the soil, only SM measurements at the 2-, 5-and 8-cm depth range were used. The sites are located in four continents and cover a large spectrum of climate conditions. Table  1  shows a summary of the networks used, and Figure  1  shows their position.
North America The SCAN (Soil Climate Analysis Network) network contains over 100 sensors/sites  [54]  and has been widely used to evaluate modeled and remote sensing soil moisture datasets. The sensors are located in agricultural regions with a relatively homogeneous landscape in many cases. The ARM (Atmospheric Radiation Measurement) network is run by the U. S. Department of Energy as part of the Atmospheric Radiation Measurement Climate Research Facility. SNOTEL (SNOw TELemetry) is a network  [55]  with a large number of stations (420) located mainly in mountainous areas. These networks are sparse networks that contain a large number of sites located at hundreds of kilometers from each other to sample soil moisture conditions at the continental scale. In addition, the United States Department of Agriculture (USDA) Agricultural Research Service (ARS) operates several dense networks.
Europe The Danish Hydrological OBsErvatory (HOBE) network  [14, 58]  is situated in the Skjern River Catchment, and it is a dense network with 30 stations distributed to sample the different soil and environmental conditions on a scale of tenths of kilometers, comparable to the AMSR-E or SMOS footprints. Nearly 80% of this region is under intensive cultivation, intermixed with patches of spruce forest (∼10%), as well as heath/grassland (∼6%). The measurements of the different sensors are averaged before comparing to remote sensing or model SM datasets. The Red de Estaciones de MEDición de la Humedad def Suelo (REMEDHUS) network  [59]  is located in a central sector of the Duero basin in Spain. The climate is semiarid continental Mediterranean, and the area is mainly flat covered by agricultural fields with some patchy forest.
Local Evaluation Strategy The AMSR-E NN SM (Section 4) and the other SM datasets (Section 2) have been evaluated against in situ measurements at the sites discussed above. RFI, snow and frozen soil filters (3 and 4 in Section 2. 8) have been applied to the data. The in situ measurements have been compared to the closest point of the EASEv2 grid. To select a given in situ measurement, a time window of ±30 min around the time of AMSR-E acquisitions has been applied. Data from a given time are used to compute statistics metrics only if an SM value is available for all of the datasets, ensuring that all time series contain exactly the same number of points. A minimum lower limit of 200 points in a time series has been fixed to use the time series to compute the metrics. Table  1  shows the number of sensors that satisfied the previous criteria for each network, and the mean number of points per time series for those sensors.
Neural Network Training Multi-layer feed-forward neural networks are universal approximators  [69] , and previous studies have already shown their ability to perform a non-linear mapping from microwave remote sensing observations to soil moisture  [31, [70] [71] [72] [73]. A training subset with ∼3 × 10 5 samples was obtained taking one fourth of the available dates and one fourth of the grid points from the filtered training database (Section 2. 8). Data from grid points corresponding to in situ measurement sites were also removed from the training subset. A fraction of 60% of the training subset is used for the actual training. An independent fraction of 20% is used to evaluate the NN performances during the training and to stop the training if the performances decrease in the evaluation subset, even if they still increase with the training subset (to avoid over-training).
This technique is usually known as "early stopping". The final independent fraction of 20% of the training subset is used to compute the performances of the trained NN a posteriori. Gradient back-propagation and minimization with the Levenberg-Marquardt algorithm were used. The sensitivity to the initialization of the minimization has been tested, finding that repeating the training with the same data and configuration, the Pearson correlation coefficient (R) of the NN output and the reference SM dataset change by less than 0. 01. The same conclusion holds if the training is done with a different data sub-sample. One single hidden layer with five non-linear neurons (hyperbolic tangents) was used, as this is enough to capture the relationship between the input data and the reference SM. Tests with more neurons did not improve the results.
The comprehensive database constructed for the current study was used here to get further insight on the soil temperature information that can be extracted from AMSR-E data. Several NNs have been trained using AMSR-E T b 's as input and the soil temperature in the first layer of the MERRA-Land and ERA-Land data as a reference. Table  2  shows that using one single observable as the input, the highest correlation of the NN output and ERA temperature is found for V polarization at 23 GHz (R = 0. 91), significantly higher than the score obtained for V polarization at 36 GHz (R = 0. 87). Adding more information as inputs, the NN ability to capture the variability of the soil temperature in the numerical weather prediction models increases. For instance, using both polarizations and three bands (18, 23 and 36 GHz), the correlation is 0.
Input Data Sensitivity to Soil Moisture As a first step, the sensitivity of AMSR-E data to SM was studied using a single AMSR-E polarization and frequency band as input to the NN. The highest correlations of the NN output with the SMOS L3 SM were obtained for C-band and X-band. They were found to be low, with values in the range of 0. 3-0. 4 for both polarizations. Therefore, in a second step, different combinations of polarizations and frequency bands were studied to increase the ability of the NN to capture the variability of the reference SM dataset. In addition, several studies  [24, 29, 34]  have shown the interest of computing derived quantities to increase the sensitivity to SM, such as reflectivities Γ = (1 -T b T soil ), polarization ratios PR = ). Using these extreme-values maps, I λφ (t) can be computed from AMSR-E T b 's for each frequency band and polarization for point λφ and at acquisition time t as: I λφ (t) = SM T min b λφ + [SM T max b λφ -SM T min b λφ ] T b λφ (t) -T min b λφ T max b λφ -T min b λφ (2) Using SM linear expectations limits the coverage to sites where SMOS L3 is available (not affected by RFI) to compute the local extreme values of SM in the reference dataset.
In contrast, the linear expectations give information on the dynamic ranges of both the measured T b 's and the reference SM, and therefore, they help significantly to capture the dynamics of the reference dataset (see below). Taking into account the six frequency bands and the two polarizations, the T b 's and derived quantities, the number of possible combinations of data that can be used as input to the NN is very high (>10 million). Of course, not all of them were evaluated, and a mix of physical insight and numerical tests was used to find the best input data configurations. Table  3  shows a summary of the results. Using one single I as input, the Pearson correlation of the NN output and SMOS L3 increases significantly to 0. 73 for the C-and X-bands in V polarization and the X-band in H polarization and up to 0.
75 for the C-band in H-polarization. This statistical analysis is in agreement with our understanding of the physics of soil emission, as the soil dielectric constant is more sensitive to soil moisture at lower microwave frequencies and horizontal polarization  [79]. On the other hand, using all of the available frequency bands led to improved results. The Pearson correlation of the NN output and SMOS L3 SM increases to 0. 76 using all T b 's (Model 18 in Table  3 ) and to 0. 79 using all I's (Model 19 in Table  3 ). Interestingly, when using T b 's and I's together as input, the ability of the NN to capture the dynamics in the SMOS L3 SM dataset improves significantly as the correlation of the NN output and SMOS L3 increases to 0. 85 when using the six frequency bands (Model 17 in Table  3 ).
As already mentioned, several combinations of the input elements can give the same results. However, Model 15 has the advantage of being relatively simple, giving scores close to the best possible ones and having a clear physical interpretation. Therefore, this is the NN configuration that has been used to invert the AMSR-E data and generate the new AMSR-E NN SM dataset. Figure  4  shows a scatter plot obtained with the test subset of the training database for Model 15. The SM retrieved with the NN is plotted versus the SMOS L3 SM (note that the density scatter plot has a logarithmic scale; therefore, most of the points are within the inner densest region of the scatter plot). The fact that it is possible to find a single global relationship with a limited amount of parameters (66 for Model 15) linking AMSR-E T b 's and SMOS L3 SM retrievals shows that in spite of the different characteristics of both instruments, their measurements are in good agreement over most of the globe.
Evaluation The AMSR-E NN SM has been compared to other datasets using classical statistical metrics, such as the mean, the Pearson correlation (R), the bias and the standard deviation of the difference (STDD). The root mean square of the differences (RMSD) is not discussed, as it is a function of the bias and the STDD (RMSD 2 = STDD 2 + Bias 2 ). In some studies, the STDD is calculated indirectly from the bias and the RMSD and called unbiased-RMSD. The Pearson correlation was used to compare the long-term (seasonal) dynamics of two SM datasets. In addition, the short-scale dynamics were evaluated by computing the Pearson correlation of the SM anomalies (hereafter, R a ). The SM anomaly at given time t, SM a (t), was computed using a 36-day window centered at t as follows: SM a (t) = SM(t) -Mean(SM(t -18, t + 18)) STD(SM(t -18, t + 18)) (3) where SM(t -18, t + 18) represents the ensemble of measurements in the 36-day window.
Comparison to SMOS L3 As discussed in the Introduction, constructing long time series of essential climate variables, such as SM, is important for climate studies. Due to the limited lifetime of space missions, it is necessary to use different instruments covering different time periods. In the case of AMSR-E and SMOS, it is possible to use the concurrent missions' period (from June 2010-October 2011) to compare the AMSR-E NN SM and SMOS L3 SM datasets. It is important to ensure that there are no significant biases, nor significant differences in the temporal dynamics of different datasets; therefore, two important metrics are the Pearson correlation and the bias. Nevertheless, other metrics were also computed. For instance, Figure  5  shows maps of the maximum, minimum, mean and standard deviation of both SM datasets.
The AMSR-E NN SM shows lower maximum values (Figure  5f ) and slightly larger minimum values (Figure  5d ) over some regions. This is also seen in the standard deviation maps. The STD of AMSR-E NN SM (Figure  5h ) in the Amazon and the boreal regions is significantly lower than that of the SMOS L3 SM dataset (Figure  5g ). The underestimation by the NN of the highest SM values in the reference dataset has also been found by previous studies (see, for instance, Figure  2  in  [32]  or Figure  4  in  [31] ). This is due to the fact that extreme values are under-represented in the training dataset and have less weight during the training. This behavior is not exclusive of NNs, but it is common to any regression tool. The maximum and minimum values and the STD can be strongly affected by only a few points with extreme values.
Therefore, it is interesting to compare other metrics as the mean of both datasets. Actually, the mean SM maps are in very good agreement (Figure  5c, d ). Figure  6  shows maps of some statistic metrics to compare further the AMSR-E NN and SMOS L3 SM datasets. The bias (AMSR-E NN -SMOS L3) is shown in Figure  6c. By construction, the AMSR-E NN should not have a significant global bias with respect to the SMOS L3 dataset used for the training. However, there can be local biases and geographical patterns. For instance, the AMSR-E NN dataset is drier than SMOS L3 in the areas of tropical forest. The absolute value of the bias is lower than 0. 05 m 3 /m 3 , which is comparable to the uncertainty level of the SM retrievals  [36]. In Australia, Northern Africa, Western Europe, Central Asia, Argentina and large parts of North America, the bias is close to zero (<0.
02 m 3 /m 3 ). Similarly, the STDD (Figure  6d ) is lower than 0. 04 m 3 /m 3 , except in the tropical forest and to a minor extent in the boreal forest. The seasonal dynamics were compared using maps of the Pearson correlation of AMSR-E NN SM and SMOS L3 SM. The correlation (Figure  6a ) shows a similar behavior to the STDD, with high values, except over dense forest regions. Therefore, the SMOS L3 and the AMSR-E NN SM datasets show similar seasonal dynamics. The main difference with respect to the bias and the STDD is found in the northeast African desert, where the correlation is low, but the bias and STDD are also low. In this region of low intrinsic variability of the SM signal, the low correlation could be due to the fact that the variance is not dominated by an annual cycle, but by noise, whose origin could be RFI, implying that RFI pollution could extend beyond the currently filtered area (blank region in the map).
Comparison to Other AMSR-E SM Datasets The new AMSR-E NN SM dataset has also been evaluated in the 2003-2011 period against the new version of LPRM  [23]  (Section 2. 2). The results are shown in Figure  7. The correlation of both datasets (Figure  7a ) is high (R > 0. 6), except at the locations of dense tropical or boreal forests and the Sahara desert. Globally, the good correlation confirms that the dynamic signatures in both datasets are similar and driven by the dynamics of the AMSR-E T b 's signal, for all but the regions where the dynamics of the T b 's are small. In order to compare also the absolute values of both AMSR-E datasets, the bias and the STDD were computed, and they are shown in Figure  7a, c , respectively. The AMSR-E NN SM dataset is on average drier than the AMSR-E LPRM SM dataset.
This bias is expected, as the SMOS L3 SM used as a reference to calibrate the global non-linear regression has been found to be drier than previous versions of AMSR LPRM SM  [15, 80]. The STDD of both products is low (<0. 04 m 3 /m 3 ) over most of the globe, but increases in forested areas up to values of ∼0. 08 m 3 /m 3 (Figure  7c ). Further evaluation of both products by comparison with the ERA-Land and MERRA-Land datasets is presented in Section 5. 3.
Global Evaluation against Model Data To get further insight into the characteristics of the new AMSR-E NN dataset, it was compared to MERRA-Land and ERA-Land model data. The same comparison has been done for other AMSR-E SM datasets and for the CCI SM data. Firstly, the temporal signatures have been compared by computing the Pearson correlation of both AMSR-E datasets with respect to the two model datasets (Figure  8 ). The temporal variations of SM in the four remote sensing datasets (AMSR-E NN, AMSR-E Reg, AMSR-E LPRM and CCI) are in good agreement, as they show similar spatial patterns. AMSR-E Reg SM shows higher correlation with model data at high latitudes, in particular with respect to MERRA-Land (Figure  8g ); otherwise, the correlation with model data is in general lower than that of other remote sensing datasets.
LPRM shows a widespread high correlation with MERRA-Land over Australia (Figure  8c ); in contrast, all remote sensing datasets show a more inhomogeneous correlation with respect to ERA-Land in this region (Figure  8b, d, f, h ). On the other hand, the correlation is significantly higher with ERA-Land over North America. It is also noteworthy that the correlation maps of AMSR-E LPRM SM and SMOS L3 SM with respect to the ECMWF-DAS2 product (H-TESSEL model with ASCAT SM data assimilation) discussed in  [15]  show similar spatial patterns. In addition to the temporal signatures, in order to compare the absolute values, the STDD and the bias were also computed. It has been shown above that AMSR-E NN is in general drier than AMSR-E LPRM. Figure  9  shows that both ERA and MERRA models are, in general, in between AMSR-E NN and AMSR-E LPRM.
The same behavior has previously been found for SMOS L3 and AMSR-E LPRM with respect to ECMWF-DAS2  [15]. The CCI dataset shows a low bias with respect to MERRA (Figure  9a ), but one should bear in mind that the SM datasets used to compute this product have been re-scaled using a model as a reference. The STDD shows a significant spatial patterns and values close to 0. 1 m 3 /m 3 for LPRM with respect to both MERRA and ERA (Figure  10a, b ), in particular for latitudes above ∼45 •. The STDD computed for the other three products is significantly lower, in particular with respect to MERRA-Land for AMSR-E Reg and NN (Figure  10e ,g), with STDD lower than 0. 05 m 3 /m 3 for most of the globe. The lower bias with respect to MERRA in comparison to ERA could be related to a shallower top soil layer in MERRA (0-2 cm) compared to ERA (0-7 cm).
Local Evaluation against In Situ Measurements The AMSR-E SM datasets, together with the model datasets, were evaluated against in situ measurements using the database and the protocol discussed in Section 3. Figure  11  shows six time series from the over 400 that have been computed. These time series show the dynamics and absolute values of the four remote sensing products, the two models and the in situ measurements compared in this study. For each site, the STDD, R and bias with respect to the in situ measurement have been computed. The results are summarized in Table  4. Firstly, it is interesting to remark that the results for the AMSR-E LPRM and Reg SM and for the CCI SM compare well to other studies carried out using the same data, even if the time period of the evaluation protocol is not exactly the same  [9, 24, 46, 81].