dataesr/scientific-paragraphs-categorization huggingface dataset:
It is also quite common that single time series in the database are composed of the consecutive measurements of two or more different sensors when a sensor is replaced after failure. Metadata information can be divided into two categories, i. e. mandatory metadata, which allow for an unambiguous identification of each network, station, and measurement in the ISMN database (Fig. D1 ), and optional metadata, shared by data providers to allow more in-depth analysis of their data sets. To be consistent between sites, the mandatory variables of climate, land cover, and soil characteristics are taken from external databases. Climate classification is taken from the Köppen-Geiger database, with a resolution of 0. 1 •  (Peel et al. , 2007). Dynamically evolving land cover for the years 2000, 2005, and 2010 is obtained from ESA's Climate Change Initiative (CCI) land cover v1.
6. 1 with 300 m resolution. Soil information is retrieved from the Harmonized World Soil Database (HWSD v1. 1; FAO/IIASA/ISRIC/ISS-CAS/JRC, 2009) with a 30 (1 km) sampling, although the actual resolution may strongly vary from location to location. If available, data providers can optionally share their own, more detailed, characterisations of land cover, soil, and quality flags with the ISMN. These are stored in addition to the same variables from external sources. All static variables per measurement site and depth are listed in.
g. metres and degrees Celsius). Then, following the recommendations of the World Meteorological Organization for weather observation and forecasting  (Williams, 2010) , all data are resampled to hourly Coordinated Universal Time (UTC) reference time steps. Data that are available at higher sampling rates are thinned by selecting the individual measurements at the hourly UTC reference time step or the ones that are closest in time within a window of ±0. 5 h. If there is no measurement available within this interval, the respective time step is not stored in the database but can be recreated and filled with a no-data value upon download. The temporal resampling scheme is valid for all included dynamic variables, except precipitation. Since precipitation is a flux and not a state variable, all measurements within the hourly interval are summed to represent the full amount of precipitation within the preceding hour  (Dorigo et al.
Data portal The ISMN can be accessed at https://ismn. earth/ (last access: 28 October 2021) and consists of a project website containing, e. g. , information about networks, data, quality control, and partners and a data interface where users can view, query, and download the data contained in the database. The data interface displays the location and information of networks and single stations and allows plotting of the available data to gain an impression of data availability and quality. Data can be selected for time period, area, single networks, or entire continents. Alternatively, the data download can be selected via an advanced SQL query, which allows users to make more specific selections (e. g. for a sensor brand or a certain depth range). The selected data are directly extracted from the database, and downloads are organised per network.
e. Global Land Data Assimilation System (GLDAS)-Noah). The spectrumbased flags are the result of a series of conditions applied to the soil moisture measurement time series and their first and second derivatives. The geophysical consistency and spectrum-based methods are only applied to the soil mois-ture observations, while geophysical dynamic range verification is applied to all dynamic variables in the database. Recently, the following refinements of the original flagging procedures, as described and assessed in  Dorigo et al. (2013) , were implemented to increase the flagging accuracy: -The outlier detection method (flag D06) now allows spikes to last 2 consecutive time steps instead of the initial 1 h. This occurs when all conditions of Eqs. (  4 )-(  6 ) in  Dorigo et al. (2013)  are met, but the peak value remains unchanged for an additional hour.
This has led to a small overall increase in flags D01, D02, and D04 (< 0. 8 %). All quality control procedures adopted by the ISMN have been made available under the open-source license agreement (see the data availability section; https://github. com/ TUW-GEO/flagit, last access: 28 October 2021). We assessed the refined flagging procedures by applying them to 10 networks with hourly data that include stations with multiple soil temperature, air temperature, or precipitation sensors. Despite the very low overall impact of the refined flags, for some networks they are substantial (Fig. 4 ). Measurements that are detected as erroneous by the quality control procedures are not deleted from the database but tagged as such (Table  3 ). The flag is provided as additional attribute (ISMN quality flag) to the observation upon download and can take one of the main categories, namely C (exceeding plausible geophysical range), D (questionable/dubious), M (missing), or G (good).
The D flag is raised when either a geophysical consistency or a spectrum-based check is positive. An additional number indicates the actual cause for flagging (Table  3 ). A soil moisture observation may receive multiple C-and D-type flags but the good and missing flags are exclusive. Seven networks provide their own soil moisture quality flags, which are added to the ISMN database in addition to the ISMN flags common to all time series. Examples are flags for data quality (without further methodological description) or simple thresholds. For instance, Real-time In-Situ Soil Monitoring for Agriculture (RISMA) tags soil moisture observations for frozen soils when the average temperature of two adjacent soil layers is below 0 • C  (Pacheco et al. , 2014). By contrast, the ISMN flag D01 (soil temperature < 0 • C) only considers the corresponding depth.
, 2019; Skierucha et al. , 2012b, a; Petropoulos and McCalmont, 2017) , soil moisture metadatabases  (Liao et al. , 2019; Xia et al. , 2015) , or validation services  (Kumar et al. , 2012)  and to assess or improve alternative sensing techniques and constellations  (Xaver et al. , 2020; Kapilaratne and Lu, 2017; Nguyen et al. , 2017; Mahecha et al. , 2017).
Automation Part of the contributions to the ISMN (7 networks with around 900 stations) are inserted into the ISMN in NRT. This process is fully automated, including data downloaded from data providers, harmonisation, flagging, insertion into the database, and updating metadata tables. However, because of different data recording and handling mechanisms at the provider side and differences in data sharing mechanisms and policies, the ISMN is also partially manually operated and will probably also have to continue in this way in the future. The differences between the fully automated and the manual approach are confined to the data download and data ingestion into the processing chain. Data harmonisation and quality control are automated for all data sets (see Fig. B1 ). A major challenge in the automation process is the enormous heterogeneity of input data formats.
Transparency and traceability The ISMN data collection is constantly evolving. New data records are added, and existing ones are extended or, if necessary, reprocessed and corrected. Not only the data but also the underlying code changes. These updates and any retroactive changes made to the data archive are tracked within the ISMN but not yet readily passed on to the users. The updates can lead to differences in analyses on the user side, e. g. when considering obvious changes such as temporal extensions or new stations, and lead to non-reproducible results. Therefore, any update must be traceable and clearly communicated, requiring a system to track and store these changes. Version tracking and digital object identifiers (DOIs) are ways of identifying each database access and, therefore, allow tracing back to past states of the ISMN.
Conclusions In this study, we reviewed the first decade of operations of the ISMN. Besides satisfactorily fulfilling its initial target, i. e. supporting satellite soil moisture product validation and calibration, many additional more or less foreseen uses have emerged. In addition, an increasing number of services and product development chains have routinely included the use of ISMN data in their operational structure. The ISMN started as research activity funded by ESA, and ever since, ESA have provided continuous financial support for ongoing research, development, and operations. In spring 2021, a milestone was achieved when the German Ministry of Transport and Digital Infrastructure announced that it will commit to permanently fund the ISMN operations and development from late 2021. The execution will be with the German Federal Institute of Hydrology (BfG) and the International Centre for Water Resources and Global Change (ICWRGC) based in Koblenz, Germany.
A10 CHINA This agricultural monitoring campaign from 1981 to 1999 includes 40 stations with soil moisture measurements up to 1 m depth and was hosted by the Institute of Geographical Sciences and Natural Resources Research at the Chinese Academy of Sciences in Beijing  (Liu et al. , 2001). The data set was transferred to the ISMN from the Global Soil Moisture Data Bank  (Robock et al. , 2000)  and contains measurements made on the 8th, 18th, and 28th of each month.
A30 KHOREZM The KHOREZM network in Uzbekistan is located between the Amu Darya river and the border with Turkmenistan and was part of a project conducted by the University of Würzburg, Germany (Patrick Knöfel). There were seven stations that made soil moisture, soil temperature, air temperature, and surface temperature measurements from 2010 April to 2011 September, and these data are included in the ISMN. Although soil moisture was not observed continuously, the measurements are still a valuable contribution since no other recent observations are available in this region.
A37 MONGOLIA Soil moisture data sets for 44 stations were collected by the National Agency of Meteorology, Hydrology, and Environment Monitoring in Ulaanbaatar. All observations were taken using the gravimetric technique and initially provided as volumetric plant-available water (in percent). Volumetric soil moisture (m 3 m -3 ) was calculated by first extracting texture properties of all sites from the Harmonized World Soil Database and subsequently calculating the wilting levels for all stations using the equations of  Saxton and Rawls (2006). Soil moisture measurements are provided 3 times a month (on the 8th, 18th, and 28th) from 1964 to 2002 during the warm period of the year, which runs from April until the end of October (e. g. Robock et al. , 2000).
A41 NVE This is a network of monitoring stations of the Norwegian Water Resources and Energy Directorate (NVE). Soil moisture, soil temperature, and air temperature are measured. Currently, three out of eight stations are accessible through the ISMN. The stations are situated in the Trøndelag region in eastern Norway. Data are available from 2012 to 2019 at more than five different depth layers and down to 1. 5 to 2 m depth.
A44 PBO_H2O The PBO_H2O network is a former near-real time network of the ISMN hosted by the University of Colorado Boulder, USA. It consisted of 159 stations distributed in the west of the USA, the Bahamas, the Dominican Republic, Puerto Rico, Colombia, South Africa, and Saudi Arabia. Soil moisture (measured using GPS reflections), precipitation, air temperature, and snow depth from 2004 to 2017 is stored in the ISMN database  (Larson et al. , 2008). The network was discontinued in 2018 because of lacking financial support.
A46 REMEDHUS REMEDHUS is the University of Salamanca Soil Moisture Measurement Stations Network and was installed in March 1999 with a set of old TDR stations and manual measurements. It is one of the first soil moisture networks in Europe. The network was automated and upgraded with capacitance probes in 2005. The REMEDHUS data available in the ISMN cover the period since its automation. REMEDHUS is located in an agricultural area in the central part of the Duero basin (Spain). The network currently has 20 stations that measure soil moisture and soil temperature hourly in the 0-5 cm layer  (González-Zamora et al. , 2018).
A47 RISMA The Real-time In-Situ Soil Monitoring for Agriculture (RISMA) network was established in 2011 by Agriculture and Agri-Food Canada at agricultural locations in Ontario, Manitoba, and Saskatchewan  (Ojo et al. , 2015; L'Heureux, 2011; Canisius, 2011). There are currently 23 RISMA stations collecting hourly soil moisture/temperature data at depths to 1-1. 5 m in combination with meteorological data. Calibrated, quality-controlled soil moisture and weather data are provided to the ISMN on an annual basis.
A51 SASMAS The Scaling and Assimilation of Soil Moisture and Streamflow (SASMAS) monitoring network commenced commissioning in late 2002, with a total of 26 stations in operation by 2003 across a 6500 km 2 catchment  (Rüdiger et al. , 2007). Soil moisture is observed up to a depth of 90 cm (where possible) at depth intervals of 0-30, 30-60, and 60-90 cm. The network was developed as a nested catchment with three different types of scales (low-resolution across the entire catchment, medium density across to smaller subcatchment, and very high density across a 175 ha single reach). This site also hosted the second National Airborne Field Experiment (NAFE) campaign, for which additional data were collected.
A52 SCAN The Natural Resources Conservation Service (NRCS) operates the comprehensive, USA-wide Soil Climate Analysis Network (SCAN). SCAN supports natural resource assessments and conservation activities through its network of automated climate monitoring and data collection sites. SCAN focuses primarily on agricultural areas of the USA, Puerto Rico, and the Virgin Islands. The network consists of 216 stations located across the USA and reports soil moisture, soil temperature, precipitation, temperature, and other climatic variables hourly. Soil sensors are situated at 5, 10, 20, 50, and 100 cm depths  (Schaefer et al. , 2007).
A55 SNOTEL NRCS instals, operates, and maintains an extensive, automated data collection network called SNOTEL (short for snow telemetry;  Leavesley et al. , 2008). SNOTEL is part of the Snow Survey and Water Supply Forecasting (SSWSF) programme and is designed to collect snowpack and related climatic data in the western USA and Alaska. The programme operates under technical guidance from the NRCS National Water and Climate Center (NWCC). With the majority of the water supply in the west arriving in the form of snow, data on snowpack provide critical information to decision-makers and water managers. SNOTEL currently consists of a network of over 860 automated SNOTEL stations, of which 463 stations have soil moisture and soil temperature sensors.
A60 TERENO TERENO consists of four terrestrial observatories that represent typical landscapes in Germany and central Europe and are considered to be highly vulnerable to the effects of global and climate change  (Zacharias et al. , 2011; Bogena et al. , 2012). TERENO combines observations with comprehen-sive large-scale experiments, integrated modelling, remote sensing, and novel measurement technologies to increase our understanding of the functioning and feedbacks of terrestrial ecosystems  (Bogena, 2016). The long-term observation platform of TERENO is composed of various measurement systems, including networks of climate and lysimeter stations, eddy covariance towers, and networks of soil, surface water, and groundwater sensors. Almost all online measurements are freely accessible via the TERENO data portal (http://www.
A61 UDC_SMOS The former German UDC_SMOS network was hosted by the Department of Geography at the University of Munich, in cooperation with the Bavarian State Research Center for Agriculture, and funded by the German Aerospace Centre (DLR). It was located in grassland in the Bavarian region around Munich as an official European SMOS calibration/validation test site. In total, 11 stations provided soil moisture data from 2007 until 2011, up to 40 cm depth, as measured by several types of sensors  (Loew et al. , 2009; Schlenz et al. , 2012a).
A67 VDS The VDS network is run by VanderSat, a Dutch company that specialises in providing global satellite-observed data and services over land. The VDS network consists of four stations located near the city of Bago in Myanmar. The network was installed to validate satellite soil moisture products in the tropics. The network has two measurement periods, i. e. one from June 2017 to July 2018 and one from March 2020 onwards.
A69 WSMN The Wales Soil Moisture Network (WSMN) was founded in July 2011. It consists of a total of nine monitoring sites located in mid-Wales representing a range of conditions typical of the Welsh environment, with climate ranging from oceanic to temperate and the most typical land use/cover types. The data set acquired in the network is composed of 0-5 (or 0-10) cm soil moisture, soil temperature, precipitation, as well as other ancillary data  (Petropoulos and McCalmont, 2017). Code and data availability. Upon registration, all data and metadata described in this paper can be downloaded for free from https://ismn. earth (ISMN, 2021). A Python package to read and plot the data and metadata (TUW-GEO/ismn: v1. 1) can be accessed at https://doi. org/10. 5281/zenodo. 855308 (Preimesberger et al.
, 2021). An example of output of this package is shown in Fig. 1. A Python package containing the ISMN quality control procedures for in situ soil moisture is available on GitHub (https://github. com/ TUW-GEO/flagit;  Aberer et al. , 2021). Author contributions. WD conceived the paper and wrote it together with IH, DA, LS, IP, LZ, and WP. All other authors contributed data to the ISMN and provided feedback on the paper. Competing interests. The authors declare that they have no conflict of interest. Disclaimer. Publisher's note: Copernicus Publications remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. Acknowledgements. The authors greatly acknowledge the financial support provided by ESA through various projects including the following: SMOSnet International Soil Moisture Network (grant no.
4000102722/10/NL/FF/fk) and Operations (grant no. 3-13185/NL/FF/fk), IDEAS+ (grant no. TVUK/AG/18/02082), QA4EO (grant no. TPZV/UK/AG/19/02321), and QA4SM (grant no. 4000135204/21//I-BG). Additional funding for the methodological advances has been received from the EU FP7 EartH2Observe project (grant no. 603608), the EU H2020 GROW project (grant no. 690199), and the QA4SM project funded by the Austrian Space Applications Programme 14 (grant no. 866004). Network funding has been provided by TWIGA (grant no. 776691). Andrej Varlagin has been supported by the Russian Foundation for Basic Research (grant no. 19-04-01234-a). Alexander Gruber also received funding (grant nos. FWO-1224320N and FWO-1530019N). We acknowledge the endorsement of various international bodies, including CEOS, WCRP GEWEX, GCOS, GTN-H, and GEO.
Global validation results of surface soil moisture of ERA and ESA CCI soil moisture against ISMN (masked and unmasked for quality flags) for the period 2001-2019. The results for ESA CCI were produced with the QA4SM validation service (https://qa4sm. eu/, last access: 28 October 2021). The results are accessible at https://doi. org/10. 5281/zenodo. 4288919 (Aberer, 2020a) (absolute values -no flags),.
The authors greatly acknowledge the financial support provided by ESA through various projects including the following: SMOSnet International Soil Moisture Network (grant no. 4000102722/10/NL/FF/fk) and Operations (grant no. 3-13185/NL/FF/fk), IDEAS+ (grant no. TVUK/AG/18/02082), QA4EO (grant no. TPZV/UK/AG/19/02321), and QA4SM (grant no. 4000135204/21//I-BG). Additional funding for the methodological advances has been received from the EU FP7 EartH2Observe project (grant no. 603608), the EU H2020 GROW project (grant no. 690199), and the QA4SM project funded by the Austrian Space Applications Programme 14 (grant no. 866004). Network funding has been provided by TWIGA (grant no. 776691). AndrejVarlagin has been supported by the Russian Foundation for Basic Research (grant no. 19-04-01234-a).
36 0. 35 0. 06 https://doi. org/10. 5281/zenodo. 4288921(Aberer,2020b)(absolutevalues-withflags),https://doi. org/10. 5281/zenodo. 4288915(Aberer,2020c)(anomalies-no flags), and https://doi. org/10. 5281/zenodo. 4288913(Aberer,2020d) (anomalies -with flags). Note: ubRMSD is the unbiased root mean squared difference.
The Climate Change Initiative of the European Space Agency (ESA CCI) uses data from the ISMN to assess, each year, the quality of new soil moisture climate data record releases and their improvements with respect to forerunner versions(Gruber et al. , 2019b; Dorigo et al. , 2015). Within ESA CCI, ISMN data were also used to quantify the spatial representativeness of ESA CCI satellite and climate model data sets (Nicolai- Shaw et al. , 2015b). The validation is systematically performed through the use of QA4SM (see below) and the results and validation settings are published (e. g. https://doi. org/10. 5281/zenodo. 4120205; Scanlon, 2020). -C3S produces authoritative, quality-assured climate data records of soil moisture and other essential climate variables (ECVs). The satellite soil mois- ture products produced within C3S are routinely updated every 10 d with the latest available satel- lite observations.
g. https://doi. org/10. 5281/zenodo. 4736927;Preimesberger,2021). -CopernicusGlobalLand Service (CGLS) produces, within 1-2 d after satellite overpass, soil moisture data sets from Sentinel-1 and from a combination of Sentinel-1 and ASCAT (SCATSAR;Bauer- Marschallinger et al. , 2018, 2019). The latter propagates the surface observations to deeper layers by means of the so-called soil water index (SWI). Both products are provided at 1 km resolution for Europe. ISMN data are used for validating moisture in surface and deeper soil layers.
Ru_CFR network includes two stations located on the territory of the Central Forest Reserve (CFR), Tver region, Russia. Since 2015, half-hourly continuous measurements of soil moisture have been carried out. Both stations provide measurements of soil moisture at four different depths and measurements of soil temperature, air temperature, and precipitation. A50 RUSWET-AGRO, RUSWET-GRASS, and RUSWET-VALDAI The three historical RUSWET networks were agricultural prediction campaigns conducted by the State Hydrologi- cal Institute of the former Soviet Union within the area of present-day Russia. Measurements were taken from 1952 until 2002 and initially distributed through the Global Soil Moisture Data Bank. Altogether, the networks operated 337 sites at which soil moisture was measured 3 times per month via gravimetric sampling.
Climate change over High Mountain Asia (HMA, including the Tibetan Plateau) is investigated over the period 1979-2014 and in future projections following the four Shared Socioeconomic Pathways: SSP1-2. 6, SSP2-4. 5, SSP3-7. 0 and SSP5-8. 5. The skill of 26 Coupled Model Intercomparison Project phase 6 (CMIP6) models is estimated for near-surface air temperature, snow cover extent and total precipitation, and 10 of them are used to describe their projections until 2100. Similarly to previous CMIP models, this new generation of general circulation models (GCMs) shows a mean cold bias over this area reaching -1. 9 [-8. 2 to 2. 9] • C (90 % confidence interval) in comparison with the Climate Research Unit (CRU) observational dataset, associated with a snow cover mean overestimation of 12 % [-13 % to 43 %], corresponding to a relative bias of 52 % [-53 % to 183 %] in comparison with the NOAA Climate Data Record (CDR) satellite dataset.