#  Executive Summary

The MaX Centre of Excellence aims at supporting the needs of all the
stakeholders involved in the field of materials modeling, simulation and
design by providing new instruments and services in the form of data, codes,
expertise and turnkey solutions to efficiently address the crucial challenges
of novel materials development in the exascale computing era. This document
provides a description of the strategies and solutions adopted within the MaX
CoE to establish a high level materials’ informatics framework to curate,
preserve and share all the data produced by the flagship codes. The core
technology behind this objective is the AiiDA code, a python infrastructure
designed to support different codes through a plugin interface, allow for an
automated design and implementation of complex workflows and task tracking and
able to store the full provenance of each object in a tailored database. AiiDA
parses the input and output files and runs the calculations on high
performance computing platforms, stores the data using uniform formats based
on python dictionaries and preserve the full provenance in the form of a
Directed Acyclic Graph (DAG). AiiDA also enables a social ecosystem where the
simulation workflows and results can be openly shared.

#  Introduction

2.1 About this document

This document is deliverable D3.1 of the MaX project and briefly describes the
types of data produced in the project, standards used for data and how the
data is being curated, preserved and shared. This is a living document and
will be updated continuously in the course of the project. The acronyms used
are summarised in the glossary at the end.

#  Description of the data

Within this project, multiple open-source computer codes including Fleur,
SIESTA, Quantum ESPRESSO, YAMBO, etc. for electronic-structure calculations
and materials modeling are being developed (see Table 1). The materials
informatics framework AiiDA has been designed to support different codes, and
plugins are already available or are being currently developed to support all
codes of this Centre of Excellence (CoE). AiiDA promotes advanced programming
models through Python abstraction layers to disseminate advanced
functionalities to arbitrary quantum engines (i.e. simulation codes). It
provides a model of automatic data generation and storage, to guarantee
provenance, preservation, reproducibility and reuse. This platform will be
used to organise and coordinate thousands of simulations, searching for
optimal properties and performance by acquiring a variety of heterogeneous
microscopic data from the calculations. It should ideally allow for an
automated design and implementation of complex workflows and task tracking,
based on a scripting interface for job creation and submission. The results
will feed a database of structures and properties that will in turn drive
further simulations. The data thus generated will be used for instance for
data-mining and machine-learning, or to build classical neural networks to
further ramp up the time and length scales accessible to numerical modeling.

Table 1: The primary codes that will be used as part of this project.  
  
<table>  
<tr>  
<th>

Name

</th>  
<th>

License

</th>  
<th>

Main Developers

</th> </tr>  
<tr>  
<td>

FLEUR [1, 2]

</td>  
<td>

GNU-GPL

</td>  
<td>

Jülich

</td> </tr>  
<tr>  
<td>

QUANTUM ESPRESSO [3, 4]

</td>  
<td>

GNU-GPL

</td>  
<td>

SISSA, EPFL, CIN

</td> </tr>  
<tr>  
<td>

SIESTA [5, 6]

</td>  
<td>

SIESTA License

</td>  
<td>

ICN2, BSC

</td> </tr>  
<tr>  
<td>

YAMBO [7, 8]

</td>  
<td>

GNU-GPL

</td>  
<td>

CNR, CIN

</td> </tr>  
<tr>  
<td>

AIIDA [9, 10]

</td>  
<td>

MIT-BSD, EPFL License

</td>  
<td>

EPFL, Robert Bosch

</td> </tr>  
<tr>  
<td>

I-PI [11, 12]

</td>  
<td>

GNU-GPL/MIT-BSD

</td>  
<td>

EPFL

</td> </tr> </table>

3.1 Types of data

AiiDA provides automated solutions and various plugins for computer codes
without a need for tuning code specific parameters. It stores the
calculations, their inputs and their results (either parsed, extracted from
Extensible Markup Language (XML) outputs or from text files with the
appropriate dictionaries) in a database and its associated file repository.
This data is generated from open-source electronic-structure material
simulations codes that encompass key technologies such as all-electron,
pseudopotentials, and localised basis sets, density-functional theory, time-
dependent density-functional theory, and many-body perturbation theory,
multiscale/multiphysics modelling with a focus on quantum mechanics/molecular
mechanics, solvation and electrochemistry, thermal/electrical transport, and
complex magnetic properties. The key codes are described in Table 1.

We are also using the Crystallography Open Database (COD) [13] and Theoretical
Crystallography Open Database (TCOD) [14] external open access databases of
crystal structures for organic and inorganic compounds to obtain the input
atomic coordinates of crystalline materials.

3.2 Format and scale of the data

AiiDA parses the input and output files mostly stored as text or XML and runs
the calculations/codes on high performance computing platforms. The full
provenance of each data object (inputs, outputs, calculations) is
automatically stored in database in a format that enables the simulation
results to be fully reproduced. The database has an associated repository with
text and binary (machine-independent) files. We have used a uniform format to
define the main raw and analysed data irrespective of the different plugins
(e.g. Quantum ESPRESSO). These formats contain data in dictionary format,
exportable for instance to plain JavaScript Object Notation (JSON) (for
example, StructureData, ParameterData data types in AiiDA store metadata in
python dictionaries within a database). Currently we are using the PostgreSQL
[15] open-source relational database to store our data. It contains around 1M+
records but over the course of the next year and a half it is expected to grow
to 100M+ records (with 10 TB+ occupied disk space). Currently we use
applications like Jmol, Visual Molecular Dynamics (VMD), PyMOL, VESTA,
XCrySDen and Blender to visualise 2D and 3D structures, and Matplotlib,
Gnuplot and Mathematica for plotting the data.

AiiDA provides a social ecosystem where the simulation results, materials and
provenance data and scientific workflows can be shared. It provides plugins to
import crystal structures from many common formats and directly from external
databases such as Inorganic Crystal Structure Database (ICSD) [16] or COD. It
has also COD and TCOD exporters to export data to these external databases.
This allows us to share data easily and also ensures their long-term
preservation.

#  Data collection/generation

4.1 Methodologies for data collection/generation

Data for the project will be created and collected by using the AiiDA
framework for the management of the simulations. AiiDA plugins and workflows
are being written for different simulation codes in order to support the
simulation with at least the codes used within this Centre of Excellence, but
also to support other codes available in the community. By using AiiDA, the
full provenance of all calculations is preserved from initial inputs to final
outputs, as well as all steps along the way, in the form of a Directed Acyclic
Graph (DAG). This allows any output data to be retrospectively checked for
quality if there are questions about how it was generated. Workflows on the
other hand provide a means of proactive quality assurance whereby a series of
steps is designed and implemented by a domain expert and packaged as a
workflow. A workflow can then be executed by experts and non-experts alike,
with internal checks and heuristics that attempt to ensure the quality of data
with respect to convergence and other relevant simulation parameters.
Furthermore, by having a standard way of running particular calculations, it
becomes much easier to compare and validate results.

Raw inputs and outputs from computer simulations codes will be stored directly
so that they may be re-parsed or manually inspected if necessary. Otherwise
data will be stored as standard, code-independent objects in the AiiDA
framework (e.g. crystal structures, band structures, pseudopotentials, _k_
-point paths, etc) allowing easy querying and manipulation of results from a
variety of simulation software packages.

All naming of these input and output files are handled internally by AiiDA and
files can be retrieved for particular calculations by either issuing a query
to match specific search criteria or directly by using the Universally unique
identifier (UUID) of a known simulation.

4.2 Data quality and standards

As mentioned previously the combination of persistent provenance and workflows
will be used in combination to maintain consistency and quality. Our
provenance model also acts as a form of documentation storing all the steps
that lead to any result in the database.

One aspect of the project involves the dissemination of a library of, so
called, pseudopotentials which contain information about the quantum
mechanical properties of the outermost electrons (those relevant in chemical
bonds) for the elements of the periodic table. Currently there may be many
different pseudopotentials for each element, however this makes it difficult
to compare calculations, and worse, some pseudopotentials are not accurate
enough to give reliable results for some of the calculations they are being
used in. By providing a standard set of pseudopotentials that have been
thoroughly tested we alleviate both these problems. We are also involved in
working on ontologies in collaboration with the TCOD team. During this
collaboration, we have implemented exporters for calculations managed using
AiiDA to the domain-specific ontology that is being built within the TCOD
project, so that also calculation results (such as crystal energies or atomic
forces) can be stored in a code-independent format.

#  Data management, documentation and curation

5.1 Managing, storing and curating data.

In AiiDA, all data (calculations, their inputs and their outputs) generated by
running highthroughput simulations on local or remote servers is naturally
stored on those computers. Moreover, relevant inputs and outputs are persisted
in the AiiDA repository, composed both of a folder-like structure and of a
database. For the latter, we use PostgreSQL, a powerful open source object-
relational database. The format for storing data (depending on the specific
type of data) is defined by the specific AiiDA data plugins, described in
detail in the code documentation. The data format of common objects (crystal
structures, band structures, force constants, etc.) is the same for all
objects of the same type, even if generated by different computer codes, to
facilitate data exchange, queries, and the bridging of different simulation
tools. Moreover, each data format is accompanied by data import and export
functions from/to standard formats (for instance Crystallographic Information
File (CIF) files for crystal structures). Further export functions can be
added transparently and will be developed during the project. Every data
object is a node in the DAG where links between nodes keeps track of the data
provenance (who generated the data, with which parameters, etc.), allowing for
easy regeneration of the same data with the same inputs. Moreover, beside
common metadata (user/owner, creation and last modification date, etc.) any
further metadata can be attached to any node of the database (data and
calculations). Also, AiiDA provides data sharing capabilities, both to share
portions of the calculations database with selected groups of users and
collaborators, and to export the data to public repositories.

Currently the full database and part of the file repository (small files) are
stored on a server at École Polytechnique Fédérale de Lausanne, Switzerland
(EPFL) while the remaining part of the file repository is stored on a Centro
Svizzero di Calcolo Scientifico Swiss National Supercomputing Centre,
Switzerland (CSCS) server. The policy defining what a large file is depends on
the application and is defined within the AiiDA workflows used to generate the
data. On the EPFL server, there are backup scripts running every day
performing a full backup of database and an incremental backup of the file
repository. The data stored at CSCS server is also backed up daily.

5.2 Metadata standards and data documentation

There are several key pieces of simulation software that will be used for this
project as described in Table 1. Typically a simulation is ran by supplying
one or more input files which along with the primary data of interest (be it a
configuration of atoms in space, the electronic structure, a material
property, etc) will typically produce auxiliary data (metadata) which can vary
greatly software to software. To interface software with AiiDA, a plugin is
written that converts AiiDA nodes (used as input) to the actual input files
required by the code, and parses outputs allowing these to be stored in the
database in a standard way, such that it can later be queried using the AiiDA
Application Programming Interface (API).

The AiiDA code itself can be considered to be _data_ in this context. AiiDA is
fully documented both in the form of descriptions of functions that make up
the API and as guides describing steps such as the installation procedure,
configuring users, setting up codes, etc. The documentation is shipped with
the code and can also be found online at  http://aiida-
core.readthedocs.org/en/latest/.

5.3 Data preservation strategy and standards

As part of the MARVEL National Centre of Competence in Research (NCCR) project
[17], funded by the Swiss National Science Foundation (SNSF), a large storage
allocation is being purchased at CSCS which will be used to store and retain
large files until at least 2018. Later, data will be handled by until 2026
(see timeline in fig. 1). In the meantime we expect to obtain other funding
opportunities for further future preservation of the data. This applies also
to all the data on the  materialscloud.org  platform that is being developed
at EPFL, where consortium members will be free to share data publicly and
privately (with selected collaborators) under the condition that data be made
publicly available within approximately a year of depositing.

The data retained, being generated with AiiDA, will include full provenance of
all simulations carried out. Some large output files may not be preserved if
they are judged to be easy to reproduce and unlikely to be needed after the
simulation has completed. This policy is code-dependent and sensible defaults
are defined within each AiiDA plugin, but can be easily changed by the user or
group who runs the simulations and generates the data.

Figure 1: MaX project timeline, compared with the Swiss Marvel project.

#  Data security and confidentiality

6.1 Formal information/data security standards

AiiDA adopts a distributed approach whereby an AiiDA instance (the code plus
the associated database) can be hosted on an individual’s machine, a group
server or a national or international server. Instances within a group should
be managed and secured by the group itself or an appointed administrator. We
provide a means of sharing results either with collaborators or the public at
large via the  materialscloud.org  website which will be running a full AiiDA
instance. In this case, EPFL and CSCS (for the AiiDA repository and the large
files, respectively) will be responsible for maintaining data security.

Data transport and access will be carried out over secure communication
channels, i.e. Secure Shell (SSH), and access to the database will be
restricted to authorised users only. The AiiDA database does not store users’
private SSH keys and therefore any possible compromise of the database does
not lead to a security breach that extends beyond the data stored in the
database itself.

6.2 Main risks to data security

Access to data and the execution of simulations is typically initiated by
opening an SSH connection. This protocol itself is considered to be highly
secure and is widely used. Also, SSH keys are used to connect rather than the
password. Moreover, these keys are not stored in the AiiDA database; instead,
AiiDA uses the keys of the Linux user under which the AiiDA daemon is running,
therefore there is no additional security risk with respect to standard SSH
connections. In any case, should a private key be obtained by an individual
other than the authorised user there are typically system logs that keep track
of all access, and the specific SSH key can be disabled to stop further
activity. In addition, AiiDA, keeps an extensive log of the activities carried
out which can be examined retrospectively in such a case.

#  Data sharing and access

7.1 Suitability for sharing

The data generated in this project is highly suitable for sharing. Given that
a simulation may take many hundreds (if not thousands) of Central Processing
Unit (CPU) hours it is beneficial to the community to be able to access these
without having to recompute them. In addition to raw data there will be a
curated section of the  materialscloud.org  that will contain results
condensed from many simulations in a form that gives an overview of particular
property or area of interest.

7.2 Discovery by potential users of the research data

Data will be discoverable by the following means:

  * The  materialscloud.org  website which will host a public facing frontend enabling access to publicly shared results. 

  

  * A private section of the website will allow dissemination with selected (authorised) collaborators. However users will be obliged to make this data publicly available approximately a year after depositing it. 

  

  * Publications will contain references to the database (including UUIDs) indicating where the data used for that study can be found. 

  

  * Publications that use results from the AiiDA repository will be encouraged to cite the paper describing the software infrastructure. 

  

7.3 Governance of access

The ultimate decision about sharing the data will lie with the PI and the
author of the data, however in general it is expected that all data from
published work within this Centre of Excellence will be made publicly
available. We are in the process of preparing a data sharing policy for the
data added to the  materialscloud.org  web portal, that will require users to
make their data available under a Creative Commons license after a maximum
period, which is likely to be one year.

The core of the AiiDA APIs (“aiida_core”) is released under an open-source MIT
license and is available to download for free. The full code, including the
full content of “aiida_core”, plus a set of useful additional plugins, are
distributed as the “aiida_epfl” package, available free of charge for academic
users. To download, users have to register with their academic email. The main
terms of the “aiida_epfl” license are available at
http://theossrv1.epfl.ch/aiida_download/.  Finally, most of the material
simulation codes are open source. As described in the proposal, also the
remaining ones are in the process of introducing a multiple-licensing system
to comply with the open-source requirements.

#  Relevant institutional, departmental or study policies on data sharing and
data security

Data will be generated by different institutions, and the data policies of the
respective institutions will apply. For data shared on  materialscloud.org,
the policies of EPFL and CSCS will apply (as the data is expected to be stored
at these two institutions. In particular, EPFL provides a combined document
“Directive concerning research integrity and good scientific practice at EPFL
(LEX 3.3.2)”  [18] for all data policies. The data policies from CSCS are
instead explained in the document  “Data_Storage_policy_V2.pdf”  [19].