#  Storage and Accessibility of Data

The intention of the consortium is to grant full and open access to all data
being collected, processed and generated within the ExaHyPE project, not only
during the grant period but also beyond that. For this reason, we implemented
the following technical infrastructure which is subject to further extension.

We set up a project website,  _http://exahype.eu/_ ,  for dissemination of all
project activity.

Furthermore, we set up a YouTube-channel for _exahype@gmail.com_ for the
dissemination of project videos. It is accessible under

_https://www.youtube.com/channel/UCKRM7I8tB6MxidxCuvn3FCA_

The source code of the ExaHyPE engine is under version control in the GIT
repository _git@gitlab.lrz.de:gi26det/ExaHyPE.git_ , which is accessible only
to the developers and members of the ExaHyPE project. Stable releases
including the full source code are published in a second repository located at
_https://github.com/exahype/exahype_ ,  which is open-access. This two-level
publication of source code gives us the possibility to keep the main-
development in a core team and to build up a user community based on stable
production versions of the code. Nevertheless, any code developed within the
project will be made available in the public repository at GitHub.com.

The whole simulation pipeline of the project is monitored with the constant
integration tool Jenkins  _https://jenkins-ci.org/_ ,  which runs benchmark
simulations and test cases on a nightly basis. Using this pipeline, we produce
and publish the documentation of the source code, the profiling of runtime
behaviour and a guidebook (i.e. a user documentation).

For very large datasets, such as simulation results of grand challenge
simulations, it is not feasible to host them in a repository or provide the
download via a project website. For such datasets we are developing an access
and archiving structure together with our associated partner, the Leibniz
Supercomputing Centre (LRZ)  _https://www.lrz.de/_ .  We will elaborate more
on this topic in later versions of this document, as soon as we have such
large datasets at hand and more progress in the development of the technical
infrastructure is made. In these versions also the potential additional costs
for long-term storage will be targeted.

The consortium plans for a backup period of 10 years. As technical
infrastructure is hosted at LRZ, this is ensured and gives full accessibility
of all data, including archiving for long term availability of the data.
Details can be found at the respective pages of the LRZ under
_https://www.lrz.de/_ .

In Chapters 4 to 6, we give a description for every dataset including the used
standards. However, for established tools or already published data, we
refrain from replicating these. In particular, we will not collect data of the
following kind:

Tools that are not developed by ExaHyPE: tools, libraries, operation systems,
compilers or visualisation tools, as we do consider this neither feasible nor
useful.

Data that is already publicly accessible: geo-information databases or
material descriptions, etc.; this data will be explicitly referenced.

#  Dataset Description – ExaHyPE Engine

**Data set reference and name:**

ExaHyPE Engine

**Origin of the dataset:**

_Generated_

The code of the ExaHyPE engine is written by the project members from scratch.
It uses and extends functionality of existing research codes and in this sense
it is based on previous code development of the partners. Namely these are the
_Peano_ software developed by Weinzierl et al. and the PDE solver _pdesol_ by
Dumbser et al.

**Type of the dataset:**

_Source code_

The ExaHyPE engine consists of the source code itself and a set of developed
pre- and post-processors which support the creation of user-specific
applications and the evaluation of the simulated results.

**Level of open access:**

The access to the ExaHyPE engine is partially open and partially confined as
explained in the dataset description.

**Ethical considerations for this dataset:**

No ethical considerations have to be taken for the ExaHyPE engine.

**Dataset description:**

The ExaHyPE Engine is the core software project developed by the consortium
members. The software is accessible to the team of developers through an
access-protected GitLab repository and to the open public through an open-
access GitHub repository. This distinction is only a difference in time and is
to streamline the development process. Any development of the ExaHyPE engine
will be released to the public-accessible repository.

**Standards:**

The main programming languages used for the ExaHyPE Engine are C++, FORTRAN,
JAVA and Python. Parallel programming models include standard MPI and OpenMP,
as well as Intel Thread Building Blocks. Standard version control (git) is
applied to the source code.

Input and configuration files are tailored text formats readable by users. As
the files are text files, standard version control (git) is applied.

Output data will be stored in VTK, HDF5, and Tecplot standard.

**Data sharing:**

The ExaHyPE engine is provided under the modified BSD license for free use, as
specified in the grant agreement. The reference is given by the following
header in every source file:

//

// This file is part of the ExaHyPE project.

//

// (C) http://exahype.eu

//

// The project has received funding from the European Union’s Horizon // 2020
research and innovation programme under grant agreement // No 671698\. For
copyrights and licensing, please consult the webpage. //

**Archiving and preservation:**

All data are stored in the repository and therefore follow the archiving and
preservation procedures outlined in Chapter 3.

#  Dataset Description – Applications from Geophysics

**Data set reference and name:**

Geophysical seismic wave simulations

**Origin of the dataset:**

_Collected_ and _processed_ data – see data set description for details.

**Type of the dataset:**

Geophysical _input data_ , _configurations_ and _simulation results_ – see
data set description for details.

**Level of open access:**

All data is available without restrictions to the members of the project,
unless it stems from sources that do not allow redistribution. Scientific
results (in form of post-processed simulation output) and simulation
configurations (e.g. including boundary conditions) will be presented in open-
access journals and made openly accessible. Input data, e.g. in form of
detailed subsurface material properties and geometries, have varying levels of
open access from publicly available to restricted, depending on their origin.

**Ethical considerations for this dataset:**

No ethical considerations have to be taken for this dataset.

**Dataset description:**

Input:

  * Subsurface material properties determining elastic and non-elastic seismic wave propagation (e.g. 

  

wave speeds): Available mostly in public geophysical community repositories or
from restricted scientific publications or scientific collaboration partners.

  * Subsurface geometry properties describing material interfaces, fault planes and geological structures: Available partly from geophysical community repositories (publicly) or will be generated by LMU Munich based on scientific publications and collaborations (restricted). 

  

  * Surface topography and bathymetry data giving high-resolution elevation of earth or planetary surface: Available from geophysical community repositories (publicly). 

  

  * Location and observed ground shaking of seismic stations during real earthquakes: Available from geophysical community repositories (publicly). 

  

Configuration:

  * Boundary conditions: Will be made publicly available upon publication. 

  

  * Frictional descriptions - analytic, empirical relationships describing frictional failure on a fault: Will be made publicly available. 

  

  * Initial parameterization of stress and strength state of the modelling domain: Will be made publicly available upon publication. 

  

Output:

  * Wave-field output: Large scale spatial-temporal output of all elastic and inelastic quantities which are solved for during the simulation of seismic wave propagation. Will be made publicly available upon publication in post-processed form. 

  

  * Fault output: Large scale spatial-temporal output of all frictional quantities which are solved for on earthquake fault during simulations incorporating dynamic rupture. Will be made publicly available upon publication in post-processed form. 

  

  * Synthetic seismograms at chosen locations: Post-processed ground-shaking time series allowing for comparison and analysis using observational seismological methods. Will be made fully publicly available upon publication. 

  

All output data will be generated via software produced within the project or
via software that is proprietary but of which we have copyright access. In
addition, publicly available dedicated software may be employed for the
analysis of the dataset from simulations.

We do not plan on purchasing any kind of data.

**Standards:**

Input, configuration and output data will be processed and published in
formats according to existing standards in the geophysical community. We will
try to define and advocate for suitable future research standards in case they
are not available.

We aim on making our simulations fully reproducible by providing computational
metadata (compiler type, compilation flags, source-code tree structure,
information on supercomputers infrastructure employed for producing the data,
information on the software employed in the analysis of the data).

The data will be stored in the most compact form possible using well-known
protocols such as hdf5 or VTK.

**Data sharing:**

The sharing of the Geophysical dataset will follow the data-sharing policy of
the ExaHyPE project.

Input, configuration and output datasets, which are not already available to
the public from other sources, will be made publicly available upon scientific
publication of geophysical simulations.

The geophysical parts of the ExaHyPE engine (source code) employed to produce
the scientific data will be made publicly available upon scientific
publication of the regarding simulation and not later than one year after the
end of the project.

**Archiving and preservation:**

The archiving and preservation policy of the geophysical dataset will follow
that of the ExaHyPE project.

We will not archive data which is publicly available from other sources.

#  Dataset Description – Applications from Astrophysics

**Data set reference and name:**

Astrophysical simulations of merging binary neutron stars

**Origin of the dataset:**

_Generated, collected_ and _processed_ data – see data set description for
details.

**Type of the dataset:**

Astrophysical _input data_ , _configurations_ and _simulation results_ – see
data set description for details.

**Level of open access:**

All the data is available without restrictions to the members of the project,
while the scientific results will be presented in open-access journals or on
publicly open preprint archives.

**Ethical considerations for this dataset:**

No ethical considerations have to be taken for this dataset.

**Dataset description:**

The data that will be generated and collected will refer to the evolution of
primitive hydrodynamical and

MHD quantities, either in the form of scalar quantities (density, energy,
etc.) or in the form of vectorial quantities (electromagnetic fields, etc.),
or in the form of tensor quantities (metric tensor, extrinsic curvature,
etc.).

A subset of this data will be employed to produce figures in scientific
publications and will be stored in specific folders dedicated to the various
publications.

All of the data will be generated via software produced within the project or
via software that is proprietary but of which we have copyright access. In
addition, publicly available dedicated software may be employed for the
analysis of the dataset from simulations.

**Standards:**

The data will be stored in the most compact form possible using well-known
protocols such as hdf5 or VTK.

Different datasets will be stored with precise timestamps and with all the
metadata information that is needed to reproduce the results of the
simulations. Such metadata includes: compiler type, compilation flags, source-
code tree structure, information on supercomputers infrastructure employed for
producing the data, information on the software employed in the analysis of
the data.

**Data sharing:**

The sharing of the Astro dataset will follow the data-sharing philosophy of
the ExaHyPE project.

More specifically:

  * all the useful data produced and collected in the simulations will be made available publicly in its most compact and yet useful form. 

  

  * the source code employed to produce the scientific data will be made publicly available as soon as its release will not endanger the academic prospects of the personnel employed in the ExaHyPE project (in particular student and postdocs) and after a proper scientific exploitation of the code has been made. At any rate, all of the produced software will be made publicly available no longer that one year after the end of the project. 

  

**Archiving and preservation (including storage and backup):**

The archiving and preservation policy of the Astro dataset will follow that of
the ExaHyPE project.

#  Degree of Progress

All activities regarding Data Management are currently proceeding as planned
and no major issues have been identified.