dataesr/scientific-paragraphs-categorization huggingface dataset:
Visible, near-infrared and short wave infrared (VNIR/SWIR, 0. 4-2. 5 μm) hyperspectral satellite imaging is one of the most promising tools for topsoil property mapping for the following reasons: i) it is derived from a laboratory technique that has been demonstrated to be a good alternative to costly physical and chemical laboratory soil analysis for estimating a large range of soil properties; ii) it can benefit from the increasing number of method-ologies developed for VNIR/SWIR hyperspectral airborne imaging; and iii) it provides a synoptic view of the study area. Despite the significant potential of VNIR/SWIR hyperspectral airborne data for topsoil property mapping, the transposition to satellite data must be evaluated. The objective of this study was to test the sensitivity of clay content prediction to atmospheric effects and to degradation of spatial resolution.
This study may offer an initial analysis of the potential of future hyperspectral satellite sensors, such as the HYPerspectral X Imagery (HYPXIM), the Spaceborne Hyperspectral Applicative Land and Ocean Mission (SHALOM), the PRecursore IperSpettrale della Missione Applicativa (PRISMA), the Environmental Mapping and Analysis Program (EnMAP) and the Hyperspectral Infrared Imager (HyspIRI), for soil applications. This study employed VNIR/SWIR AISA-DUAL airborne data acquired in a Mediterranean region over a large area (300 km 2 )withaninitial spatial resolution of 5 m. These hyperspectral airborne data were simulated at the top of the atmosphere and ag-gregated at six spatial resolutions (10, 15, 20, 30, 60 and 90 m)  to correlate with the future hyperspectral satellite sensors. The predicted clay content maps were obtained using the partial least squares regression (PLSR) method.
HyspIRI is an American hyperspectral sensor with a spatial resolution of 60 m over the spectral range of 0. 38 to 2. 5 μm (hyspiri. jpl. nasa. gov). SHALOM is an Italy-Israel initiative with a spectral range of 0. 4 to 2. 5 μm, and a spatial resolution initially defined at 15 m  (Bussoletti, 2012) , which recently evolved from 15 m to 10 m  (Ben-Dor, Kafri, & Varacalli, 2014). HYPXIM is a French hyperspectral sensor with a spatial resolution better than 8 m over the spectral range of 0. 5 to 2. 5 μm, and of 1. 8 m in the panchromatic band  (Briottet et al. , 2013; Carrere et al. , 2013). Despite the significant potential of the VNIR/SWIR hyperspectral airborne data for mapping several soil properties, the application of methods developed for airborne data (such as the PLSR method) to satellite data is not a straightforward process.
Conversely, spatial resolutions planned for future spaceborne sensors (between 8 to 60 m) are coarser than the ones used by airborne sensors, such as HYMAP or AISA-DUAL (approximately 5 m). The various effects of the spatial resolution degradation on multispectral remote sensing data have been studied, including flux prediction (e. g. ,  Kustas & Norman, 2000) , crop area estimation and crop growth monitoring (e. g. ,  Duveiller & Defourny, 2010)  and agricultural land monitoring (e. g. ,  Ismail, Mutanga, Kumar, & Urmilla, 2008; Pax-Lenney & Woodcock, 1997). Some spatial resolution degradation studies have also focused on VNIR/SWIR hyperspectral data for vegetation studies (e. g. ,  Nijland, Addink, De Jong, & Van der Meer, 2009; Rahman, Gamon, Sims, & Schmidts, 2003; Schaaf, Dennison, Fryer, Roth, & Roberts, 2011; Thorp, French, & Rango, 2013; Zhang, Middleton, Gao, & Cheng, 2012).
g. , ~50:1 from 2. 1 to 2. 4 μm)  (Folkman, Pearlman, Liao, & Jarecke, 2001). The objective of this study was to evaluate the sensitivity of VNIR/ SWIR hyperspectral-based soil property prediction to atmospheric effects and degradation in spatial resolution. The selected property was the topsoil clay content, which is one of the basic soil properties used by soil surveyors to describe soil types. It is also a key driver of soil erosion processes  (Le Bissonnais, 1996) , and one of the primary soil properties included in the GlobalSoilMap specifications  (Arrouays et al. , 2014). This study employed the VNIR/SWIR AISA-DUAL hyperspectral airborne data acquired over a large area (300 km 2 ) in a Mediterranean region, with an initial spatial resolution of 5 m. The predicted clay content maps were obtained using the PLSR method.
Hyperspectral airborne radiance data On November 2, 2010, an AISA-DUAL hyperspectral image was acquired over the study area (12 × 25 km) via plane, at an altitude of 3600 m, with a field of view (FOV) of 24°and a spatial resolution of 5m(Fig. 1b ). The AISA-DUAL airborne imaging spectrometer measures the at-flight radiances (L flight_AISA_5m ) in 359 non-contiguous bands, covering the 0. 40 μm to 2. 45 μm spectral domain, with 0. 0046 μm bandwidths between 0. 40 μm and 0. 97 μm, and 0. 0065 μm bandwidths between 0. 97 μm and 2. 45 μm. The field of view of the instrument is 24°. On November 2, 2010, the visibility was estimated at 40 km, using weather reports and visual estimations from the airplane. The cloud cover was 0% during the flight-time, and neither the in-situ atmospheric profiles nor the aerosol data were available during the campaign.
Simulation of the "Top of Atmosphere" radiances The Top of Atmosphere (TOA) synthetic data were obtained by considering the entire atmosphere layer, requiring the definition of an atmosphere profile with a given water vapor content, and the type and abundance of the aerosols. From these inputs, the TOA radiances (L TOA_5m ) were simulated from the at-flight radiances (L flight_AISA_5m ) (Fig. 2 ), according to the following equation: L TOA 5m ¼ τ dir þ τ di f L flight AISA 5m þ L atm ð1Þ where τ dir and τ dif are the atmospheric direct and diffuse transmittance, respectively, and L atm is the upwelling atmospheric radiance (without the contribution of the land). COMANCHE is a radiative transfer model used to solve Eq. (1)  (Miesch et al. , 2005). COMANCHE uses an analytical formulation of the upwelling radiance at the sensor level, in which the atmospheric parameters are independent of the ground parameters.
The MODTRAN 4 radiative transfer code  (Berk et al. , 1999) wasusedto compute most of the atmosphere parameters (e. g. , the columnar water vapor amount), except for the Earth atmosphere coupling irradiance and the environment upwelling radiance, which were estimated with a Monte Carlo kernel. In the COMANCHE tool, a rural aerosol model was used with a 40 km visibility, and the United States Standard 1976 atmosphere model from MODTRAN  (Berk et al. , 1999). This model was chosen because it had the best fit with the value of the total water vapor content, estimated at 1. 5 g/cm 2 by MOD07 (https://lpdaac. usgs. gov/).
This exercise was performed using the Resampling Aggregate tool of the ENVI application using the following equations: Ω agg ¼ NΩ i ð3Þ L TOA xm ¼ 1 N X N i¼1 L TOA 5m;i : ð4Þ The selected ground sampling distances were driven by the GSD of the airborne acquired data (5 m). Then, six new TOA radiance images (L TOA_xm ) were produced at the following GSD values: 10, 15, 20, 30, 60 and 90 m. These values were close to those from the spaceborne missions, as follows: HYPXIM (10 m), SHALOM (15 m), PRISMA (20 m, 30 m), EnMAP (30 m) and HyspIRI (60 m), and from the value of the Global Soil Map (GSM) project (www. globalsoilmap. net), set at 90 m.
S is the atmospheric spherical albedo. These terms are estimated using a Monte Carlo approach. The remaining atmospheric parameters are computed from the MODTRAN 4 radiative transfer code  (Berk et al. , 1999). The inversion process is iterative. In the initial step, the adjacency effects are neglected, and a first estimate of ρ xm _ SAT (i, j)isobtained. Then,Eq. ( 5 ) is used to retrieve the surface reflectance from the simulated TOA radiances (L TOA_xm ). Eq. (  5 ) is used again to retrieve the surface reflectance from the airborne radiances (L flight_AISA_5m ), and in this case, ρ xm _ SAT (i, j)isreplacedbyρ flight _5m (i, j). Finally, to remove the noise effects introduced by this process, the reflectance spectrum of each pixel was smoothed using the following filter: ρ smooth i ¼ 1 N X iþm j¼i-m ρ j ; m ¼ N-1 2 ð6Þ where ρ i smooth is the surface reflectance in the channel number i after the application of the filter, and N is an odd integer indicating the number of wavelength neighbors to be averaged.
Field data One hundred twenty soil samples were collected over the study area (Fig. 1b ). Among this sample set, 50 were collected in June 2008, 30 in October 2009 and 41 in November 2010. All of these soil samples were collected in fields that were bare during the hyperspectral airborne data acquisition in November 2010. All of the samples were composed of five sub-samples, which were collected to a depth of 5 cm at random locations within a 10 × 10 m 2 square centered on the geographical position of the sampling plot, as recorded by a Garmin GPS instrument. Because soil samples were collected over plowed fields, the collected soils from 0 to 5 cm can be considered as having mixed and homogeneous soil properties. After homogenizing the sample, approximately 20 g was allocated for clay content analysis.
The initial samples were air-dried and sieved with a 2 mm sieve prior to being transported to the laboratory for analysis. The determination of the clay content (granulometric fraction b 2 μm) was determined using a pipette method (method NF X 31-107, particle size distribution by sedimentation,  Baize & Jabiol, 1995). The clay content of the 120 soil samples varied between 108 and 772 g/kg, and followed a normal distribution (Table  1 ). A degradation in the spatial resolutions leads to a slight decrease in the suitable surface area that can be mapped by VNIR/SWIR spectroscopy (Section 3. 4), and leads to a decrease in the number of soil samples associated with bare soil pixels. At 5 m, 120 soil samples were available over the bare soil pixels. This 120 sample database is called CalBase-5m.
At 60 and 90 m, only 67 and 65 soil samples, respectively, were available over the bare soil pixels. These databases are called CalBase-60m and CalBase-90m, respectively. The CalBase-5To30m database consists of 82 soil samples, which correlate with the images from 5 to 30 m of resolution. Although the number of reference soil samples varied depending on the spatial resolution, the distribution and statistical characteristics of the soil datasets did not vary. The clay content of the CalBase-5To30m database varied from 122 to 772 g/kg, and followed a normal distribution (Table  1 ). The clay content of the CalBase-60m and CalBase-90m database varied from 122 to 772 g/kg and to 764 g/kg, respectively, and both followed a normal distribution (Table  1 ).
Clay content prediction Partial least squares regression (PLSR) is a multivariate model commonly used for soil property estimation. The PLSR method specifies a linear relationship between a set of dependent (response) variables (Y-variables, the clay content in our case), and a set of predictor variables (X-variables, the spectra)  (Tenenhaus, 1998). A detailed description of the PLSR model can be found in  Wold, Sjöström, and Eriksson (2001). The general concept of PLSR is to extract the orthogonal or latent predictor variables, accounting for the maximum amount of the variation of the Y-variables. A PLSR model is developed from a training set of N observations (the number of spectra in the calibration dataset), with K X-variables (the number of wavelengths in the spectra), and M Y-variables (the number of soil properties).
The training data form matrices X and Y, of dimensions (N × K) and (N × M), respectively. As with all of the factorial methods, the main goals of PLSR are the following: i) to locate a subspace of the spectral space ℝ K on which the spectra are projected, yielding a matrix of N scores T (N × k); and ii) to perform a linear regression between T and Y. The maximum number of latent predictor variables was set to 10 and was determined using the Predicted Residual Sum of Squares (PRESS) statistic, taking care to avoid under-and over-fitting. An analysis was performed to detect the outliers in the calibration dataset. Outliers are commonly defined as observations that are inconsistent with the majority of the data  (Chiang, Pell, & Seasholtz, 2003; Pearson, 2002) , such as observations that deviate significantly from normal values.
An outlier can be defined as the following: (i) a spectral outlier when the sample is spectrally different from the remainder of the samples or (ii) a concentration outlier when the predicted value has a residual difference significantly greater than the mean of the predicted values. To identify the spectral outliers, we used the principle of the Mahalanobis distance  (Mark & Tunnell, 1985)  and applied it to principal component analysis reduced data. The samples with a Mahalanobis distance greater than 3 were identified as outliers and removed from the calibration dataset. For the case of normally distributed data, 99% of the data will be included in an interval of ± 3σ, which will be less than a threshold of 3. Prior to the quantitative statistical analysis, the reflectance was converted into a "pseudo absorbance" (log [1/reflectance]), and a standard normal variate correction was achieved for removal of additive and multiplicative effects  (Barnes, Dhanoa, & Lister, 1993).
The prediction capability of the PLSR was analyzed using two different procedures, which were dependent on the size of the dataset (the number of spectra associated with the clay content). Type 1 was applied to the CalBase-5m and CalBase-5To30m datasets, which contain more than 80 variables. In this case, the dataset was divided into two groups, one group for calibration of the PLSR model (3/4 of the dataset), and a second group used for validation (1/4 of the database). The Y values were sorted in an ascending order. The method starts by selecting the sample with the lower clay content, and placing it in a validation set. Then, the next three samples are placed in the calibration set, and the procedure is continued by alternately placing the following sample in the validation set and the next three samples in the calibration set.
Such a process would ensure a relatively equal distribution of the samples in both sub-datasets. Type 2 is applied to datasets with less than 80 reference samples (CalBase-60m and CalBase-90m). In this case, a subdivision in calibration and validation datasets would provide too little data in the calibration set. So all of the data were used in the calibration dataset, and no independent validation set was used. In both types of procedures, a leave-one-out cross-validation procedure was adopted to verify the prediction capability of the PLSR model for the calibration set. N -1 samples were used to build the regression model from all of the N samples within the dataset. Based on this model, the value for the clay content of the sample not used in the development of the model was predicted. This procedure was repeated for all of the N samples, resulting in predictions for all of the samples.
Variograms provide evidence of spatial autocorrelation when the semi-variances are lower at smaller lags than at larger lags, i. e. , the sampling locations located close to each other have similar values. The autocorrelation of clay distribution can be studied by means of nugget (unexplained variability referring to noise in the data), partial sill (structural component of the total variance) and range (range of the variogram) analysis. The geostatistical operations were performed with R (version 1. 17), using the gstat package  (Pebesma, 2004). To evaluate the rate of soil property variability that was theoretically lost as the spatial resolution increased, the variability in the predicted clay contents for each aggregated image was estimated from the map of clay content predicted using the 5m_AISA airborne data.
Preliminary results A reference PLSR model, following Type 1(Section 3. 4), was built from the 5m_AISA airborne image and the 120 clay content values associated with the 120 available soil samples. Ninety samples were assigned to the calibration database, and thirty to the validation database. The performance of this model was relatively accurate, with an R 2 cal value of approximately 0. 74, an R 2 val value of approximately 0. 75, a RPIQ of 3. 2 and an RMSEP value of approximately 86 g/kg (Table  2 ), as previously shown in  Gomez, Lagacherie, and Bacha (2012). T h e inner-pixel variability, V (Eq. 8), ranges from 27 g/kg (with N =8 2 grids of 15 × 15 m 2 and p = 9) to 66 g/kg (with N =6 5g r i d so f 95 × 95 m 2 and p =361)(Fig. 4 ). This reference PLSR model was applied to all the spectra over bare soil from the 5m_AISA airborne image to produce a predicted clay content map.
Sensitivity to atmospheric effects A PLSR model was built following Type 1, using the 120 spectra associated with the bare soil pixels of the 5m_HYPXIM image, for which soil samples were collected and clay contents were measured. The models built from the 5m_HYPXIM and 5m_AISA images were compared (Table  2 ). The best model was obtained from the 5m_AISA image (Table  2 ,R 2 val ≈ 0. 75 and RPIQ ≈ 3. 2). However, the performance of the model obtained from the satellite-simulated data at 5 m is only slightly less accurate (Table  2 ,R 2 val ≈ 0. 71 and RPIQ ≈ 3). Therefore, the atmosphere appears to slightly affect the performance of the PLSRbased prediction models. No significant difference in the b-coefficients appears between the models (Fig. 6a ). Only the b-coefficients between 0. 932 and 1.
Sensitivity to degradation of spatial resolutions The analysis of the PLSR models, built from the seven simulatedsatellite images, allowed the identification of two groups of sensors with specific behaviors. A first group of spatial resolutions, from 5 to 30 m, corresponding to HYPXIM, SHALOM, PRISMA and EnMAP sensors, offers accurate PLSR models with R 2 cal N 0. 65, RMSEC b 100 g/kg and R 2 val N 0. 6 (Table  2 ). The number of outliers remains stable (between 1 and 3) regardless of the spatial resolution of this first group of sensors. Slight differences in the prediction performance were observed in the validation dataset. The spatial resolutions of 5 and 30 m yielded similar prediction performances, whereas the spatial resolution of 20 m provided a better performance, with an R 2 val value of 0.
4 and RMSEC N 130 g/kg (Table  2 ). The analysis of the combination of the VIP and b-coefficients shows that only approximately ten spectral bands, centered at 2. 206 μm, were significant for the model at 60 m, and no significant spectral bands were identified at 90 m (Fig. 9aandb ). Each PLSR model built from the 82 soil samples of the common dataset was applied to all the spectra over bare soil for its corresponding simulated-satellite image, generating predicted clay maps called 5m_HYPXIM_Clay_2, 10m_HYPXIM-SHALOM_Clay,15m_SHALOM_Clay, 20m_PRISMA_Clay and 30m_PRISMA-EnMAP_Clay. Because the PLSR models built from the simulated-satellite images at 60 and 90 m were inaccurate, no predicted clay maps were built at these resolutions. Regardless of the spatial resolution from 5 to 30 m, no large difference occurs in the absolute prediction in regard to the distribution of the predicted clay content over the entire area (Fig.
Short-scale clay content variations as a driver of spatial resolutions impact The spatial resolutions from 5 to 30 m appear to only slightly affect the performances of the PLSR-based prediction models (Table  2 ). The results of the calibration and validation datasets indicate that the PLSR model performances do not decrease when the spatial resolution becomes coarser, and the degradation in spatial resolution of the image does not linearly affect the model performance. The spatial resolutions of 5 and 30 m yielded similar prediction performances, whereas the spatial resolution of 20 m provided better performance. The slight differences in the performances may arise from the calibration dataset, which contains an acceptable, but not a significant number of data to calibrate the models. The slight effect of the spatial resolutions on the PLSR model performances may be explained by the low rate of shortscale clay content variations, which represents only 22% of the whole variance of clay content in our study area.
Acknowledgments This research was granted by the  TOSCA-CNES  project  «HUMPER-Mission HYPXIM : Apport de la résolution spatiale de la mission HYPXIM pour l'étude des propriétés pérennes des sols et de leur humidité de surface»  ( 2013-2014 ). This research was also supported by the  French National Research Agency (ANR)  through the  ALMIRA  project ( ANR-12-TMED-0003 ). The authors are indebted to  UMR LISAH (IRD, France) and toCNCT(CentreNationaldeCartographieetdeTélédétection,Tunisia) , for providing the AISA-DUAL hyperspectral images for this study. This hyperspectral data acquisition was granted by  IRD (Institut de Recherche pour le Developpement) ,  INRA  ( Institut National de la Recherche Agronomique  ) and the  French National Research Agency (ANR)  ( ANR-O8-BLAN-C284-01 ).
2014a; Anagnostou et al. 2010; Tian et al. 2007). Information from a large number of rain gauges is already assimilated as part of global/regional satellite algorithms; nevertheless, the rain gauge databases sourced by these procedures can exclude more extensive national networks where data accessibility is restrictive, as often is the case in developing countries. In these regions, the global precipitation product may be found to be unsatisfactory and requiring local adjustment (e. g. ,  Lavado-Casimiro et al. 2009). Satellite algorithms are known to internally perform gauge correction, for example, using a mean-field bias correction (MBC) and/or inverse-error-weighted averaging methods  (Huffman et al. 1997; Grimes et al. 1999). Postanalysis merging methods have also been used on the products of these algorithms, for example, the spatial adjustment of TMPA using interpolations by inverse distance weighting  (Lavado-Casimiro et al.
b. Precipitation data The satellite-based product is obtained from the TMPA product, in its latest available version 7, released in November 2012. The TMPA product combines observations from multiple satellites, including TRMM and other geostationary satellites  (Huffman and Bolvin 2013). The performance of this dataset in this study area has been previously assessed by  Zulkafli et al. (2014a). Precipitation estimates from 1998 to 2008 were obtained with a 3-h temporal resolution and 0. 258 3 0. 258 (approximately 27. 8 km 3 27. 8 km) spatial resolution. Values were subsequently aggregated to the daily scale in order to match the temporal resolution of the rain gauge records. The rain gauge dataset is the historical rainfall time series obtained from the Peruvian National Meteorological and Hydrological Service [Servicio Nacional de Meteorología e Hidrología del Perú (SENAMHI)].
The Kalman gain represents the relative measure of uncertainty and is defined as follows: K 5 V « S (V « S 1 V « G ) 21. ( 18 ) The uncertainty associated with the satellite estimates V « S is represented by the covariance matrix of the satellite error field (i. e. , « 5 Z G 2 Z S ), while the uncertainty associated with the rain gauge rainfall estimates V « G is the error variance produced in the kriging interpolation. The variable K increases (decreases) as V « S is larger (smaller) compared to V « G , and consequently, the filter will weigh more in favor of the rain gauge (satellite) estimates in the combination. The Bayesian combination method was initially implemented using the commercial software RAIN-MUSIC (PROGEA Srl 2009). As the software is not open source, the default settings of the method cannot be changed, for example, the theoretical variogram model for the kriging interpolation has to be Gaussian.