DOI: 10.5281/zenodo.13799955
title: Chemical Occurence Data Curation Results with CleanGeoStreamR

description: About

This dataset contains curated chemical occurrence records from the NORMAN network, focusing on measurements in European surface waters. The curation process ensures harmonized spatial metadata, standardized units, and consistent substance identifiers to support reproducible environmental assessments and mixture risk analyses.

Data processing and quality control were performed using the CleanGeoStreamR R package, which provides automated workflows for geospatial verification, coordinate normalization, and detailed documentation of data sources and processing steps.

A detailed description of the curation workflow, including methodological rationale and software implementation, is available in the peer-reviewed publication Automated Curation of Spatial Metadata in Environmental Monitoring Data (DOI: 10.1016/j.ecoinf.2025.103038).

Access

Access level: This dataset is openly available without restriction under the Creative Commons Attribution 4.0 International (CC BY 4.0) license.

Provenance and Versioning

All curation steps were performed using documented workflows in the CleanGeoStreamR R package. The extended dataset (02_empodat_unfiltered_tagged.csv) includes a column named ModificationFLAGS, which documents the specific curation and standardization steps that were applied to each record (e.g., unit harmonization, coordinate correction, site merging, removal of duplicate entries). This information is retained to provide full traceability during quality control and reproducibility assessments. Future updates will be published as new Zenodo versions under the same concept DOI.

File Information

The dataset is provided in tabular CSV format along with accompanying XML metadata files. The CSV files contain curated chemical occurrence data, coordinate reference tables, and comparison/debug tables used during spatial quality control. File sizes vary based on the number of records and sites included in each table.

Core Data Files



01_chemical_data.csv: Primary curated chemical occurrence dataset after harmonization and quality control.

02_empodat_unfiltered_tagged.csv: Extended dataset including all available observations with location tags and curation attributes.

02_df_coords.csv: Reference table providing verified geographic coordinates for monitoring locations.

02_locations0.csv: Intermediate site-location table used in coordinate resolution workflows.

02_IDX_map.csv: Large index map linking occurrence records to harmonized site identifiers (mapping table).


Metadata Files



01_chemical_data_metadata.xml: Metadata describing fields, units, and processing context for the curated dataset.

02_empodat_unfiltered_tagged_metadata.xml: Metadata describing fields and processing context for the extended dataset.






Quality Control / Debugging Files



comparison_of_rows_Case_1–7.csv: Comparison reports used during duplicate, overlap, and site-merge evaluations.

false_entries_debug.csv: Records identified as inconsistent during spatial or identifier checks.

false_entries_debug_without_NA_neighbour_info.csv: QC subset excluding records lacking neighbor coordinate information.

02_chemical_data_duplicated.csv: Records flagged as duplicates during identifier harmonization.






Measured Variables

The core dataset contains harmonized information on sampling events, substances, and geographic context. The main variable groups are:



Norman_SusDat_ID: Substance identifier from the NORMAN SusDat reference database.

Substance_name: Common or systematic name of the chemical associated with the Norman_SusDat_ID.

lon, lat: Geographic coordinates of the sampling location, expressed in decimal degrees (WGS84).

country: Country in which the sampling site is located.

station_name_n: Standardized monitoring station name after name harmonization and site-merging procedures.

river_basin_name_n: Name of the river basin associated with the sampling location.

water_body_name_n: Name of the specific water body at the sampling site.

concentration_value: Measured concentration of the substance in the sampled water.

concentration_unit: Reporting unit of the concentration value.

time_point: Date of sampling

ModificationFLAGS: Internal curation and quality control indicators summarizing applied corrections, such as unit conversions, coordinate adjustments, duplicate merging, or metadata inference steps.


Curation Workflow Summary

The dataset was curated using the CleanGeoStreamR R package, which standardizes geographic metadata, harmonizes substance identifiers, and consolidates monitoring locations across data sources. Coordinates were validated and corrected where necessary, measurement units were aligned to common reference units (typically µg/L), and duplicate or inconsistent records were resolved using spatial proximity, identifier concordance, and temporal information. Metadata from source systems was integrated into a unified schema to support consistent interpretation of sampling locations, substances, and concentration measurements.

Domain Metadata Standards

Geospatial and environmental descriptors in this dataset can be mapped to concepts in the Environment Ontology (ENVO) where needed, for example when annotating surface water and river basin features. Chemical identifiers are provided using the NORMAN SusDat reference system, which can be cross-referenced to external identifier resources such as ChEBI when required. These links are not applied automatically here, but the dataset structure allows such mappings to be established in downstream workflows.