dataesr/scientific-paragraphs-categorization huggingface dataset:
Over time, many other countries also established at least some air quality monitoring sites with ozone monitors. Often, the data from these networks are now available in near-realtime (see for example https:// aqicn. org/map/world/). However, the rapid reporting of these data precedes the required quality control, which is needed for the purposes of TOAR. Access to qualitycontrolled data from the networks' archives is more difficult due to either restrictive government regulations or the lack of supra-regional data archives. In some world regions, multi-national networks or databases have been installed that complement national monitoring and data provision. Examples are the East Asia (Acid Deposition) Network (EANET) or the European Environment Agency Airbase system. Unfortunately, as described in section 5, the data from these multi-national archives are not always fully consistent with the original data reported at the regional or national level.
The TOAR database maintains these "duplicate" data records as individual data series and applies a merging procedure to select the most appropriate data for analysis (see section 5). The focus of current air quality monitoring networks generally lies in urban and suburban areas. However, in the context of acid deposition monitoring, ozone monitoring stations were also established in rural areas. These stations allow for observations of regional baseline concentrations and attribution of high ozone episodes. Other surface ozone observations, typically of shorter duration, have been made during field campaigns or in support of ozone impact studies on forest or agricultural vegetation. Finally, a few programs have recorded multi-year ozone measurements from mobile platforms such as railway trains (for example  TROICA: Oberlander et al.
, 2002; Pankratova et al. , 2011) , or ships (EU project APICE:  Velchev et al. , 2011). For establishing the TOAR database of surface ozone concentrations we focused on data from stationary platforms, with hourly time resolution, and time series that are longer than 2 years. The vast majority of measurements were made with the UV absorption technique (see  Tarasick et al. , 2017, this issue). Passive sampling data were not considered due to the low time resolution. These choices were made in order to allow a globally uniform calculation of ozone metrics for the analyses in TOAR-Metrics  (Lefohn et al. , 2017, this issue) , TOAR-Health, TOAR-Vegetation, and TOAR-Climate. Furthermore, these criteria allow characterization of at least some inter-annual variability, and to assess the robustness of the derived ozone metrics at each site.
In order to achieve a well-defined data set and allow for some quality assurance of the ozone data (see section 5), the database was closed for new submissions in July 2016. Only corrections to existing data sets were accepted after this date. The most recent measurements that entered the TOAR analyses are from 2015, although most of the results presented throughout TOAR do not extend beyond 2014. In Europe, many datasets included in TOAR do not extend beyond the year 2012 due to changes in the Airbase data reporting system, which coincided with the build-up of the TOAR database and prevented inclusion of more recent Airbase data before closure of data submissions. Data availability and accessibility of existing long-term surface ozone observations varies considerably among the countries and multi-national networks.
Some networks maintain comprehensive, well-managed databases and allow open access through ftp, web downloads or interoperable web services. Such data were readily retrieved and included in the TOAR database. In cases where open access is not available or language problems prevented us from accessing or interpreting data directly, we tried to negotiate access to long-term archived data, which was particularly successful for data from Japan, and South Korea. In other regions, data collection and harmonisation remains fragmented and many different data providers must be addressed individually in order to obtain data. Major efforts were undertaken especially in Australia, South Africa, and South America to collect the available data and make them accessible to TOAR. In some countries concerns about misuse or misinterpretation of the data prevent local authorities from openly sharing information, especially in the form of time-resolved hourly concentration values.
Also, large requests for hourly data can impose a substantial work load on agency staff and they may not be in a position to engage in the necessary reprocessing of historic data if this is not a direct part of their mandate. In the case of research data, not all scientists who are involved in ozone measurements are fully supportive of the TOAR open data policy for different reasons. This may have prevented them in some cases from freely sharing their data with the TOAR database curators. We would like to note in this context that the TOAR database contains special provisions to restrict open access to the ozone metrics at a limited number of sites if this is requested by the data provider. In such cases, only the aggregated metrics data products (see section 6) are freely available. However, it is important that these metrics are derived consistently with the metrics at all other stations in the database, so only hourly data are entered into the database.
The primary intention of  Sofen et al. (2016b)  was again to provide surface observations for the evaluation of global chemistry models, and therefore their main products are gridded data sets containing spatial averages of monthly mean or other statistically aggregated ozone concentrations. The present work benefitted from the cooperation with E. Sofen on the identification of data format and data quality issues (see discussion in sections 4 and 5). Going beyond previous ozone data collection efforts, the TOAR initiative has worked closely with data providers from around the world in order to increase coverage beyond the regions where data are easily available from open data archives. Through the creation of regional working groups and the efforts of the many co-authors on this article, it has been possible to integrate the hourly ozone data from 9,690 stations, many of which had never been available for internationally coordinated research.
Some of these data, for example from Australia and Japan, date back to the early 1980s and thus fill important gaps in our knowledge about global surface ozone during that period. Table  1  lists the ozone monitoring networks and data sources of the TOAR database, and Table  2  provides a summary of the database holdings grouped by region. Regions are labelled as defined by the Task Force on Hemispheric Transport of Air Pollution (TFHTAP;  Koffi et al. , 2016). Large gaps remain in the global coverage of surface ozone measurements. Very few data exist in Africa, Central America, Central Asia, South Asia, and the Middle East. Where data are available from these regions, the time series are often short and sometimes appear to have some data quality issues. Figure  1  shows the temporal evolution of the number of ozone data records in the database by region.
In general the data coverage has increased over time in all world regions. The dropoff in the most recent years is mostly due to delays in the final processing and validation of current data. In North America, most data originate from AQS and NAPS (Table  1 ). The earliest data from AQS that are included in the TOAR database are from 1980, the earliest records from NAPS are from 1974. As we did not have NAPS data after 2013, the coverage in North America drops slightly during the final two years. In Europe, the majority of data are from Airbase with the earliest records from Great Britain dating back to 1973. Since Airbase changed their reporting system and data format for data after 2012, we could not include more recent data in the TOAR database. In East Asia, the earliest data are from Japan, dating back to 1976.
Data from South Korea have been made available to us beginning in 2000. The TOAR database also includes data from 26 Chinese stations including 15 in Hong Kong, some of which date back to 1990. Unfortunately, we have not been able to obtain ozone data from the vast Chinese air quality network which commenced in 2012. Similarly, data from only 7 Taiwanese stations were included while at least 50 stations currently report ozone concentrations  (AQICN, 2017). The earliest data from Australia and New Zealand date back to 1978, while the majority of measurements in this region commenced around 1992. In Mexico and the Caribbean the network density has steadily increased since the mid-1980s. There are no other data from Central America. Data coverage decreases after 2010 in the TOAR database due to the fact that we obtained these data from individual researchers and not from official agencies and their analyses did not always include the most recent years.
The earliest officially available data from Argentina and Chile are from 1994 and 1997, respectively, while ozone data from Brazil are available since 1998. Data classified as oceanic in the database are mostly from coastal sites and they are labelled OCN only because of inaccuracies in the global gridded map that was used to assign the TFHTAP region code to each station. True oceanic sites are American Samoa (GAW), Sable Island (NAPS, OTHER), Ieodo Ocean Research Station (OTHER), Ogasawara (EANET), and Minamitorishima (GAW). We note that several other island sites are not included in the OCN region. For example, Amsterdam Island (OTHER) belongs to region SAF, and Bermuda (GAW) belongs to Middle and Central America in spite of their remote locations (this has been coded in the TFHTAP gridded file).
In future versions of the database, a better designation of island sites would be desirable. The earliest data from Africa and the Middle East are from Cape Point, South Africa (GAW), which began in 1983. Assekrem, Algeria (GAW), and Amsterdam Island (OTHER) commenced in 1997 and 1995, respectively. Beginning in 2000, data from up to 20 stations are also available from Tehran, Iran. Unfortunately, these seem to have some data quality issues. From Southern Asia, 7 data sets could be obtained from India, and 1 data set from Nepal. From South East Asia, 5 data sets are from Indonesia, 4 from Thailand, 3 from Malaysia, and 1 from Vietnam. Central Asia has one data set from the station Issyk-Kul in Kyrgyzstan (GAW), and the RBU region contains data from 8 stations in Russia, 1 in Armenia, and 1 in Poland (misclassified because it is located on the border with Belarus).
Access to TOAR data All TOAR data except for the original hourly time series are freely available for scientific and policy use. However, we want to stress that, in spite of close collaboration between the TOAR database curators and most data providers, the TOAR data products do not constitute official data and are therefore not to be used in analyses with legal implications such as exceedance monitoring. 1  The actual TOAR database with the hourly observations continues to develop (for example, we recently included ozone precursor and meteorological data for European stations), while the pre-compiled data products, which have been used in other parts of the assessment, represent a frozen snapshot in order to guarantee reproducibility of results. For access to the pre-compiled TOAR data products including present-day monthly, seasonal, summertime, and annual data, trend datasets, and gridded datasets, we refer the reader the data publication on PANGAEA (https://doi.
org/10. 1594/PANGAEA. 876108). The metrics files are provided as simple csv files. Gridded data products intended for model evaluation purposes are provided as netcdf files. More detailed descriptions of the file formats and variables are provided in the Supplemental Material 1. The PANGAEA data portal also contains an extensive collection of plots similar to those presented in TOAR-Health, TOAR-Vegetation, and TOAR-Climate. The live database can be accessed via the interactive web interface of the Jülich Open Web Services Interface (JOIN; https://join. fz-juelich. de), and through the Representational State Transfer (REST) services which are also provided by JOIN. Details are provided in the JOIN user guide (Supplemental Material 2). The JOIN web interface allows for easy access to metadata and data from individual measurement sites.
It includes several filtering options to select stations based on various metadata criteria (faceted search). Hourly, daily, monthly, seasonal, summertime, and annual data can easily be visualized as time series and, with the exception of hourly data, are also available for download as text files. The user can also generate comprehensive data summary plots, which contain a time series including data capture information, average annual, weekly, and diurnal cycles with distinction between night and day or season, respectively, frequency distributions, and trend information (see example in Figure  2 below ). For instructions how to generate such plots, see Supplement 2. Via the REST interface of JOIN, most information from the TOAR database is also available in an interoperable way. This allows for software access to TOAR data without the requirement to log into the JOIN web interface.
Data collection The vast majority of data in the TOAR database were obtained from a few well-managed networks which maintain their own databases and apply their own quality control measures to varying degrees (see  Sofen et al. , 2016b  for a discussion on typical data and metadata issues). In each case we established contact with the database managers and individual data providers and asked for the best available data collection and metadata information. In many cases, this included extensive discussions of metadata and data quality issues with the data providers and archive managers. As noted above, in some regions the TOAR regional working groups and other individuals invested considerable effort to collect and harmonize ozone data sets and submit them to the database. A simple template in ASCII format was designed and shared for these submissions.
Reported metadata and their quality control The TOAR data submission template requested detailed metadata information, including, but not limited to the station location and altitude, station type ('traffic', 'industry', or 'background'), station type of area ('urban', 'suburban', 'rural', or 'remote'), and details about the measurement method, the data set PI, and the contributing organisation. Not all of this information is always available, and as of yet there is no standardized vocabulary applied to many of these metadata fields, limiting the ability to search for specific types of data in the database. For example, there are more than 50 different terms used to describe the most common ultraviolet absorption measurement method; in this case the information has been standardized. We performed extensive checking of station metadata, primarily with respect to the station location.
In several hundred cases we tried to manually verify the station location through Google maps (Google, 2017), and often found either incorrect or imprecise coordinates which were then corrected. Often, these checks were prompted by detecting different coordinate values for identical stations in different data repositories. In extreme cases, the station locations differed by more than 30 km from each other (for example for Somerton, GB0044R between EMEP and Airbase). In most cases the differences were on the order of a few hundred meters or less. One means of checking station coordinates was to compare the reported station altitudes with the topographic elevations from Google maps. This not only revealed many cases with highly inaccurate station elevations (for example, many of the older AQS sites apparently had altitudes given in feet instead of meters, or they reported altitude above ground), but could also be used to correct the reported location of the site, in particular in mountainous terrain.
For example, the reported station altitude of the Indian station Mt. Abu was 1680 m asl, while Google maps returned an altitude of only 1180 m at the reported coordinate values of 24. 6°N, 72. 7°E. The station building could be visually identified on the Google maps satellite image at the more precise location 24. 653056°N, 72. 779167°E. Using these coordinates to retrieve a new Google maps altitude yielded 1663 m asl, which is very close to the reported station altitude. Note, however, that differences in station altitude can also occur for sites where sampling occurs on tall towers. Some of these were identified and documented in the station_comments attribute, but we likely missed several other towers, because information on inlet heights is generally not available. The coordinate corrections were also sent to the data providers for verification.
More recently, we have begun documenting all coordinate corrections by introducing additional database fields. For example, three different station altitude values are maintained (station_reported_ alt, station_google_alt, and station_etopo_alt, see Table  3  below), and we use a station_alt_flag to document which of these is regarded as the most trustworthy piece of information. This value is then returned as station_alt. Similarly, we have begun to document the confidence we have in the station coordinates that are saved in the database (for details see the description of the database layout in Supplemental Material 2). Despite these efforts many issues remain with respect to inaccuracies in the station information, and this may limit the applicability of these data for impact assessments.
For example, while modern web services and GIS applications would make it feasible to relate air pollutant concentrations to the distance of pollution sources such as roads or industrial plantations, such analyses will produce erroneous results if the station coordinates are not maintained with high accuracy. Furthermore, many impact studies also require information about the sampling height of the measurement. This information is, unfortunately, currently not included in the TOAR database, because there are too few data sets that report this site characteristic.
Gridded metadata In order to improve the characterization of stations and their environment, and to allow for globally consistent data aggregation by site-specific criteria, we obtained several high-resolution global gridded data products. As described above these additional metadata are also used to quality control the metadata information that is provided with the original data. Table  3  provides an overview of the station metadata added to the data submissions through extraction from global gridded data products. Additional information can be found in the description of the database layout in Supplemental Material 2. The global gridded products are provided in different resolutions, are valid for different years, and may contain errors themselves. It is therefore not advisable to rely on one individual piece of metadata for selecting or aggregating surface ozone data.
For example, we tried to correlate several "pollution indicators" (nighttime lights, population density, NO x emissions, and NO 2 tropospheric column densities), and found large scatter among these variables even though they are qualitatively consistent (Figure  3 ). Correlation coefficients r range from 0. 46 to 0. 83. The lowest correlation is found between nighttime lights and log(NO 2 columns), while the highest correlation is calculated for the pair of nighttime lights and log(population density). Nevertheless, taken together these indicators allow us to make some clear distinctions between more and less pollution-impacted environments, and with their help it has been possible for the first time to develop a globally applicable, robust station classification scheme. Table  4  lists the criteria that were applied to mark all stations in the TOAR database as "urban", "rural, low elevation", "rural, high elevation", or "unclassified".
The limited information on data quality that is available in current surface ozone data sets precludes a meaningful and systematic use of metadata for the identification of potential data inaccuracies. Furthermore, such metadata would not protect against data processing and data submission errors, of which we found many in the data that were made available to TOAR. We did not keep track of all individual data errors that were identified. A rough, subjective estimate is that more than 95% of the hourly ozone values do not show any obvious quality issues. Most of the questionable data concern complete time series from some fifty sites where reported ozone mole fractions are frequently interrupted and the values, diurnal and seasonal patterns simply don't match any expectation. These series are easily identified and excluded from further analysis.
About 1% of the remaining data show questionable or erroneous features during some parts of the time series. This can be individual outlier values or, for example, calibration shifts which happen throughout one year or part of a year. Even if these cases are only 1%   The prefix "station_" is omitted from the variable names in the rule expressions for clarity. For details of the selected variables, see Table  3. Population density is reported in km -2 , nightlight is an integer index, omi_no2_column is given in 10 15 molec. cm -2 , and altitudes are in m. of the data, several hundred time series are affected, and without accounting for such data problems, the TOAR statistics and trend analyses would lead to wrong results. The most prevalent, obvious errors are readily exposed through visible inspection of time series, or from database queries searching for negative or anomalously high concentration values.
Due to the sheer amount of data we had to process and screen, not all discovered data quality issues were systematically logged. However, in many cases erroneous or questionable data were flagged, and the data providers were contacted. Clearly, for the future development of the TOAR database, it would be desirable to develop algorithms to pre-screen the data and alert the station operators about potentially flawed time series. While we have indeed begun such developments, there has not been enough time to conduct sufficient testing, to integrate such tests into a quality control system, and to optimize the tests in order to reduce false positive alarms and potential misses to the extent possible. A particular data quality issue concerns very small or negative concentration values. Small negative concentrations can arise in ozone measurements due to the measurement principle of UV absorption instruments.
However, assuming appropriate equipment with adequate signal to noise ratios and detection limits, and carefully maintained and calibrated instruments we would not expect mixing ratios lower than -1 or -2 ppbv in an hourly mean value (see  Galbally and Schultz, 2013). Yet, at some stations mixing ratios of -5 ppbv or even lower were reported. These are likely associated with calibration errors and cannot easily be corrected, because it is unclear if the data should be shifted or truncated. Hence, such data have to be accepted as is (they are not flagged in the TOAR database), but the user should be aware that the data uncertainty will be large. It should be noted that individual data centers or data providers handle negative concentrations differently. Without detailed knowledge of their procedures, the resulting uncertainties of the data can be only poorly assessed.