D1.14: Updated data management plan
Creators: name: Peters-von Gehlen, Karsten
Publisher: name: Zenodo
Year: 2024

description: About this document

This report reviews the data management implementation, listing the type and sources of data produced, handled, and stored until month 30, and provides information on respective repositories and access conditions. If content is updated in this reviewed version of the DMP, it is preceded by “UPDATE March 2024” and is formatted in italic font.

Work package in charge

WP1

Executive summary

This Deliverable details how all data in nextGEMS is handled and how the project addresses compliance with the FAIR data principles. Further, every work package has contributed information regarding the research data workflows applied to proceed towards the project goals. This also applies to the processing of personal data which is related to the ethics deliverable D11.1. All information contained in this DMP is considered preliminary, and the DMP will be updated as nextGEMS progresses.

Conclusion & Results

In this DMP, we have detailed the approach of nextGEMS to comply with the FAIR data principles from the perspective of a project creating very large amounts of numerical model simulation output, which becomes obsolete with every new step in model development. Making all data available according to the FAIR principles is technically and organisationally not possible - nextGEMS thus focuses on selected project outcomes to be made available according to the FAIR principles. All necessary information needed to handle and process data created in nextGEMS is documented in https://easy.gems.dkrz.de and is not contained in this DMP. That website is maintained by nextGEMS project scientists.

Further, we have collected information pertaining to project data from every work package. This information consists of contact details, data produced and reused, data dependencies between work packages, approaches to project-internal and global data-sharing and dissemination, and data archiving approaches. Due to the early stage of the nextGEMS project, all information given in this DMP is considered as preliminary and will be updated as the project progresses, i.e., the DMP is appropriately treated as a “living document”.

UPDATE March 2024: Over the course of the nextGEMS project, the handling of data has undergone a significant evolution. At the beginning of the project, the simulation output data was handled and accessed as was common practice over the last decade or two, i.e., data available on regular grids, made available as netcdf files and accessed via their path on the parallel file system at DKRZ. By M30 of the project, simulation output is provided in small chunks in zarr format on HEALPix grid and accessed via intake catalogs using python notebooks. The efficiency of data access and processing has increased by orders of magnitude, and the methods have been thoroughly tested in the nextGEMS hackathons. The online resource https://easy.gems.dkrz.de has proven to be essential for the success of the projects’ data handling and management. The data handling methods developed in nextGEMS provide blueprints for rethinking and redesigning the way large-volume Earth System Model output data are managed.