#  Executive Summary

The MaX Centre of Excellence aims at supporting the needs of all the
stakeholders involved in the field of materials modeling, simulation and
design by providing new instruments and services in the form of data, codes,
expertise and turnkey solutions to efficiently address the crucial challenges
of novel materials development in the exascale computing era. This document
provides a description of the strategies and solutions adopted within the MaX
CoE to establish a high level materials’ informatics framework to curate,
preserve and share all the data produced by the flagship codes. The core
technology behind this objective is the AiiDA code, a python infrastructure
designed to support different codes through a plugin interface, allow for an
automated design and implementation of complex workflows and task tracking and
able to store the full provenance of each object in a tailored database. AiiDA
parses the input and output files and runs the calculations on high
performance computing platforms, stores the data using uniform formats based
on python dictionaries and preserve the full provenance in the form of a
Directed Acyclic Graph (DAG). AiiDA also enables a social ecosystem where the
simulation workflows and results can be openly shared. Both these aspects have
been strengthened in the last version of the code. On one hand with the update
of the AiiDA plugin and workflow systems on the other, with the development of
the AiiDA REST (REpresentation Stateless Transfer) Api which also constitutes
the backbone of the  materialscloud.org  portal and finally with the
implementation of various exporters and converters to the most commonly used
data formats and ontologies.

#  Introduction

2.1 About this document

This document is deliverable D3.4 of the MaX project, it is the updated
version of data management plan already designed and delivered in D3.1[1] at
month 6. The documents describes the types of the data produced in the
project, standards used for data and how the data is being curated, preserved
and shared. The basic types of data and the way they are organized by AiiDA
remains essentially unchanged with respect to D3.1[1] and they are hereafter
reported in a similar way. Important developments have been carried out to
ease the sharing of data and of the computational protocols used to generate
them. Such developments include a critical revision of the AiiDA plugin system
and the implementation of a REST Api compliant with the OPTIMaDe protocol.

#  Description of the data

Within this project, various open-source first principle simulation codes such
as Fleur, SIESTA, Quantum ESPRESSO and YAMBO are being developed (see table
1). The materials informatics framework AiiDA has been designed to allow the
support of many different codes through a plugin interface. Plugins for the
main codes within by this Center of Excellence (CoE) are already available as
described in deliverable Table 1: The primary codes that will be used as part
of this project.  
  
<table>  
<tr>  
<th>

Name

</th>  
<th>

License

</th>  
<th>

Main Developers

</th> </tr>  
<tr>  
<td>

Fleur [3, 4]

</td>  
<td>

GNU-GPL

</td>  
<td>

Jülich

</td> </tr>  
<tr>  
<td>

Quantum ESPRESSO [5, 6]

</td>  
<td>

GNU-GPL

</td>  
<td>

SISSA, EPFL, CIN

</td> </tr>  
<tr>  
<td>

SIESTA [7, 8]

</td>  
<td>

SIESTA License

</td>  
<td>

ICN2, BSC

</td> </tr>  
<tr>  
<td>

YAMBO [9, 10]

</td>  
<td>

GNU-GPL

</td>  
<td>

CNR, CIN

</td> </tr>  
<tr>  
<td>

AiiDA [11, 12]

</td>  
<td>

MIT-BSD, EPFL License

</td>  
<td>

EPFL, Robert Bosch

</td> </tr>  
<tr>  
<td>

i-PI [13, 14]

</td>  
<td>

GNU-GPL/MIT-BSD

</td>  
<td>

EPFL

</td> </tr> </table>

D3.2[2] of this project. A general overview of the present status of the
plugins can be found at  http://www.aiida.net/plugins/  together with the
contact of the reference person for each plugin and a link to the main
repository. The page collects also numerous other plugins, for codes outside
this CoE, which are being currently developed as part of other collaborations
or by individual contributors. AiiDA promotes advanced programming models
leveraging python abstraction layers to disseminate advanced functionalities
to arbitrary quantum engines (i.e. simulation codes). It provides a model of
automatic data generation and storage, to guarantee provenance, preservation,
reproducibility and reuse. This platform is used to organize and coordinate
thousands of simulations, it allows to acquire and store a variety of
heterogeneous microscopic data from the calculations that can be subsequently
queried for the desired material properties. Details on the calculation
execution such as the parallelization scheme and execution time are also
retained to empower performance optimization. Furthermore AiiDA allows for an
automated design and implementation of complex workflows and task tracking,
based on a scripting interface for job creation and submission. The inputs,
results and computational procedures at each step of the workflows are
collected and stored in a database. All plugins, workflows and the data
produced through them are thought and designed to be openly shared as
discussed afterward.

3.1 Types of data

AiiDA provides automated solutions and various plugins for computer codes
without a need for tuning code specific parameters. It stores the
calculations, their inputs and their results (either parsed, extracted from
Extensible Markup Language (XML) outputs or from text files with the
appropriate dictionaries) in a database and its associated file repository.
This data is generated from open-source electronicstructure material
simulations codes that encompass key technologies such as allelectron,
pseudopotentials, and localised basis sets, density-functional theory,
timedependent density-functional theory, and many-body perturbation theory,
multiscale/multiphysics modelling with a focus on quantum mechanics/molecular
mechanics, solvation and electrochemistry, thermal/electrical transport, and
complex magnetic properties. The specific data to be stored in the database
within the input and output nodes of a calculation and the files to be
retrieved and stored in a local repository are determined by each code plugin.
This follows the specific characteristic of each simulations code and the
physics of the problem under study. The design of each calculation node is
documented by the plugin developers, see for example  http://aiida-
core.readthedocs.io/en/stable/plugins/ quantumespresso/pw.html.  AiiDA also
allows to use data from external open access databases of crystal structures
for organic and inorganic compounds such as Crystallography Open Database
(COD) [15] ,Theoretical Crystallography Open Database (TCOD) [16] and
Inorganic Crystal Structure Database (ICSD) [17] to obtain the input atomic
coordinates of crystalline materials. The platform also offers the
possibility, at the workflow level, to copy large files (for example charge
densities) to a data storage facility for later reuse and to save into the
database a symbolic link to such remote folder.

3.2 Format and scale of the data

AiiDA parses the input and output files mostly stored as text or XML and runs
the calculations/codes on high performance computing platforms. The full
provenance of each data object (inputs, outputs, calculations) is
automatically stored in database in a format that enables the simulation
results to be fully reproduced. The database has an associated repository with
text and binary (machine-independent) files. We have developed uniform formats
to define the most common raw and analyzed data irrespective of the different
plugins. These standards contain data in dictionary format, exportable for
instance to plain JavaScript Object Notation (JSON) (for example,
StructureData, ParameterData data types in AiiDA store metadata in python
dictionaries within a database). AiiDA however allows the flexibility to
define new data structures and formats which might be strongly code depended.
Currently we are using the PostgreSQL [18] open-source relational database to
store our data. Overall they presently contains around 15M+ records and over
the course of the next year it is expected to grow to 100M+ records (with 10
TB+ occupied disk space). Currently we use applications like Jmol, Visual
Molecular Dynamics (VMD), PyMOL, VESTA, XCrySDen and Blender to visualise 2D
and 3D structures, and Matplotlib, Gnuplot and Mathematica for plotting the
data.

#  Data collection/generation

4.1 Methodologies for data collection/generation

Data for the project are created and collected by using the AiiDA framework
for the management of the simulations. AiiDA plugins and workflows have been
written for different simulation codes in order to support the simulation with
at least the codes used within this Center of Excellence, but also to support
other codes available in the community (See  http://www.aiida.net/plugins/)  .
By using AiiDA, the full provenance of all calculations is preserved from
initial inputs to final outputs, as well as all steps along the way, in the
form of a Directed Acyclic Graph (DAG). This allows any output data to be
retrospectively checked for quality if there are questions about how it was
generated. Workflows on the other hand provide a means of proactive quality
assurance whereby a series of steps is designed and implemented by a domain
expert and packaged as a workflow. A workflow can then be executed by experts
and non-experts alike, with internal checks and heuristics that attempt to
ensure the quality of data with respect to convergence and other relevant
simulation parameters. Furthermore, by having a standard way of running
particular calculations, it becomes much easier to compare and validate
results.

Raw inputs and outputs from computer simulations codes are be stored directly
so that they may be re-parsed or manually inspected if necessary. Otherwise
data are be stored as standard, code-independent objects in the AiiDA
framework (e.g. crystal structures, band structures, pseudopotentials, _k_
-point paths, etc) allowing easy querying and manipulation of results from a
variety of simulation software packages.

All naming of these input and output files are handled internally by AiiDA and
files can be retrieved for particular calculations by either issuing a query
to match specific search criteria or directly by using the Universally unique
identifier (UUID) of a known simulation.

4.2 Data quality and standards

As mentioned previously the combination of persistent provenance and workflows
is used in combination to maintain consistency and quality. Our provenance
model also acts as a form of documentation storing all the steps that lead to
any result in the database.

One aspect of the project involves the dissemination of a library of, so
called, pseudopotentials which contain information about the quantum
mechanical properties of the outermost electrons (those relevant in chemical
bonds) for the elements of the periodic table. Currently there may be many
different pseudopotentials for each element, however this makes it difficult
to compare calculations, and worse, some pseudopotentials are not accurate
enough to give reliable results for some of the calculations they are being
used in. By providing a standard set of pseudopotentials that have been
thoroughly tested we alleviate both these problems. We are also involved in
working on ontologies in collaboration with the TCOD team. During this
collaboration, we have implemented exporters for calculations managed using
AiiDA to the domain-specific ontology that is being built within the TCOD
project, so that also calculation results (such as crystal energies or atomic
forces) can be stored in a code-independent format.

#  Data management, documentation and curation

5.1 Managing, storing and curating data

In AiiDA, all data (calculations, their inputs and their outputs) generated by
running high-throughput simulations on local or remote servers is naturally
stored on those computers. Moreover, relevant inputs and outputs are persisted
in the AiiDA repository, composed both of a folder-like structure and of a
database. For the latter, we use PostgreSQL, a powerful open source object-
relational database. The format for storing data (depending on the specific
type of data) is defined by the specific AiiDA data plugins, described in
detail in the code documentation. The data format of common objects (crystal
structures, band structures, force constants, etc.) is the same for all
objects of the same type, even if generated by different computer codes, to
facilitate data exchange, queries, and the bridging of different simulation
tools. Moreover, each data format is accompanied by data import and export
functions from/to standard formats (for example Crystallographic Information
File (CIF) files for crystal structures) Importer and exporter for commonly
used formats such as ASE (Atomic Simulation Environment) and PyMatgen format
have been developed. Other import/export capabilities can be transparently
added through upon necessity. Every data object is a node in the DAG where
links between nodes keeps track of the data provenance (who generated the
data, with which parameters, etc.), allowing for easy regeneration of the same
data with the same inputs. Moreover, beside common metadata (user/owner,
creation and last modification date, etc.) any further metadata can be
attached to any node of the database (data and calculations). Also, AiiDA
provides data sharing capabilities, both to share portions of the calculations
database with selected groups of users and collaborators, and to export the
data to public repositories as described in more details in D3.3[19].

Currently the full database and part of the file repository (small files) are
stored on a server at École Polytechnique Fédérale de Lausanne, Switzerland
(EPFL) while the remaining part of the file repository is stored on a Centro
Svizzero di Calcolo Scientifico - Swiss National Supercomputing Centre,
Switzerland (CSCS) server. The policy defining what a large file is depends on
the application and is defined within the AiiDA workflows used to generate the
data. On the EPFL server, there are backup scripts running every day
performing a full backup of database and an incremental backup of the file
repository. The data stored at CSCS server is also backed up daily.

5.2 Metadata standards and data documentation

There are several key pieces of simulation software that will be used for this
project as described in table 1. Typically a simulation is ran by supplying
one or more input files which along with the primary data of interest (be it a
configuration of atoms in space, the electronic structure, a material
property, etc) will typically produce auxiliary data (metadata) which can vary
greatly from software to software. To interface software with AiiDA, a plugin
is written that converts AiiDA nodes (used as input) to the actual input files
required by the code, and parses outputs allowing these to be stored in the
database in a standard way, such that it can later be queried using the AiiDA
Application Programming Interface (API). The AiiDA code itself can be
considered to be _data_ in this context. AiiDA is fully documented both in the
form of descriptions of functions that make up the API and as guides
describing steps such as the installation procedure, configuring users,
setting up codes, etc. The documentation is shipped with the code and can also
be found online at  http:// aiida-core.readthedocs.org/en/latest/.  To ease
the deployment procedure a dockerfile [20] for AiiDA installation with default
parameters has been realized (See
https://bitbucket.org/aiida_team/aiida_core_docker)  this method will be also
extended to each code with a plugin interface with AiiDA in order to readily
install a working AiiDA and simulation code instance.

Figure 1: MaX project timeline, compared with the Swiss Marvel project.

5.3 Data preservation strategy and standards

As part of the MARVEL National Centre of Competence in Research (NCCR) project
[21], funded by the Swiss National Science Foundation (SNSF), a large storage
allocation has been purchased at CSCS which will be used to store and retain
large files until at least 2018. It is foreseen that this will be extended
until 2026 (see timeline in fig. 1). In the meantime we expect to obtain other
funding opportunities to enable further preservation of the data. This applies
also to all the data on the  materialscloud.org  platform that is being
developed at EPFL, where consortium members will be free to share data
publicly and privately (with selected collaborators) under the condition that
data be made publicly available within approximately a year of depositing. The
Materials Cloud platform is expected to be fully operative in April 2017.
Entire AiiDA graphs or part of them could be directly shared through the
Materials Cloud portal.

The data retained, being generated with AiiDA, will include full provenance of
all simulations carried out. Some large output files may not be preserved if
they are judged to be easy to reproduce and unlikely to be needed after the
simulation has completed. This policy is code-dependent and sensible defaults
are defined within each AiiDA plugin, but can easily be changed by the user or
group who runs the simulations and generates the data.

Together the Materials Cloud and AiiDA aim to enable researchers to trivially
adopt aspects of the Findable, Accessible, Interoperable and Re-usable (FAIR)
principles as laid out by  Force11.  Any calculation ran through AiiDA is
automatically interoperable (we define an open standard and provide tools to
convert data to other libraries and are adding Open Databases Integration for
Materials Design (OPTiMaDe) compatibility) and re-usable (by virtue of the
provenance tracking and workflows). The Materials Cloud on the other hand
ensures that the data is both findable and accessible. Section 7 outlines
further details.

#  Data security and confidentiality

6.1 Formal information/data security standards

AiiDA adopts a distributed approach whereby an AiiDA instance (the code plus
the associated database) can be hosted on an individual’s machine, a group
server or a national or international server. Instances within a group should
be managed and secured by the group itself or an appointed administrator. We
provide a means of sharing results either with collaborators or the public at
large via the  materialscloud.org  website which will be running a full AiiDA
instance. In this case, EPFL and CSCS (for the AiiDA repository and the large
files, respectively) will be responsible for maintaining data security.

Data transport and access will be carried out over secure communication
channels, i.e. Secure Shell (SSH), and access to the database will be
restricted to authorised users only. The AiiDA database does not store users’
private SSH keys and therefore any possible compromise of the database does
not lead to a security breach that extends beyond the data stored in the
database itself.

6.2 Main risks to data security

Access to data and the execution of simulations is typically initiated by
opening an SSH connection. This protocol itself is considered to be highly
secure and is widely used. Also, SSH keys are used to connect rather than the
password. Moreover, these keys are not stored in the AiiDA database; instead,
AiiDA uses the keys of the Linux user under which the AiiDA daemon is running,
therefore there is no additional security risk with respect to standard SSH
connections. In any case, should a private key be obtained by an individual
other than the authorised user there are typically system logs that keep track
of all access, and the specific SSH key can be disabled to stop further
activity. In addition, AiiDA, keeps an extensive log of the activities carried
out which can be examined retrospectively in such a case.

#  Data sharing and access

7.1 Suitability for sharing

The data generated in this project is highly suitable for sharing. Given that
a simulation may take many hundreds (if not thousands) of Central Processing
Unit (CPU) hours it is beneficial to the community to be able to access these
without having to recompute them. In addition to raw data there will be a
curated section of the  materialscloud.org,  which uses directly AiiDA as a
backend, that will contain results condensed from many simulations in a form
that gives an overview of particular property or area of interest.

AiiDA provides a social ecosystem where the simulation results, materials and
provenance data can be shared. It provides plugins to import crystal
structures from many common formats and directly from external databases such
as ICSD [17] or COD. It has also COD and TCOD exporters to export data to
these external databases. Moreover it is fully interoperable with commonly
used data formats for crystal structures such as XSF[22], ASE[23] and
Pymatgen[24].

7.2 Discovery by potential users of the research data

Data will be discoverable by the following means:

  * The  materialscloud.org  website which will host a public facing frontend enabling access to publicly shared results. 

  

  * A private section of the website will allow dissemination with selected (authorised) collaborators. However users will be obliged to make this data publicly available approximately a year after depositing it. 

  

  * Publications will contain references to the database (including UUIDs) indicating where the data used for that study can be found. 

  

  * Publications that use results from the AiiDA repository will be encouraged to cite the paper describing the software infrastructure. 

  

7.3 Data access and reusability

##  7.3.1 REST interface and Optimade support

A key feature to enable large-scale data access is represented by the
development of the AiiDA REST (REpresentation Stateless Transfer) Api, an
interface that assigns Universal Resource Identifiers (URIs) to the objects
stored in an AiiDA instance, thus making them obtainable via HTTP requests in
a programmatic manner. The REST Api can be coupled to
authentication/authorization modules so that the AiiDA administrator can
choose the degree of accessibility of the resources. Moreover the AiiDA
developers signed-on to provide a URI syntax compliant with the standards
defined in the OPTIMaDe ( "Open Databases Integration for Materials Design)
protocol to let the users interrogate heterogeneous databases with the same
syntax. Finally, the AiiDA REST Api can be used as an essential building block
for on-line services that expose a repository of data persisted in AiiDA. A
paramount example is  materialscloud.org  whose back-end largely relies on the
AiiDA REST Api.

##  7.3.2 Sharing of plugins workflows

Point A1.1 of the FAIR principles requires that “the protocol is open, free,
and universally implementable”. This is facilitated with a flexible plugins
system being developed that enables developers to extend AiiDA to support
their own codes and formats and share this functionality with the community
via the Materials Cloud. Similarly, workflows that encode scientific expertise
on how to carry out a series of steps to arrive at a result can be
disseminated via the Materials Cloud. A critically revised version of the
workflow engine and of the plugin system distribution, has been implemented in
the latest release AiiDA v0.8 and described in details in D3.3[19]. This
simplifies both the sharing and reuse of these two high level code components.

7.4 Governance of access

The ultimate decision about sharing the data will lie with the PI and the
author of the data, however in general it is expected that all data from
published work within this Center of Excellence will be made publicly
available. We are in the process of preparing a data sharing policy for the
data added to the  materialscloud.org  web portal, that will require users to
make their data available under a Creative Commons license after a maximum
period, which is likely to be one year.

The core of the AiiDA APIs (“aiida_core”) is released under an open-source MIT
license and is available to download for free. The full code, including the
full content of “aiida_core”, plus a set of useful additional plugins, are
distributed as the “aiida_epfl” package, available free of charge for academic
users. To download, users have to register with their academic email. The main
terms of the “aiida_epfl” license are available at
http://theossrv1.epfl.ch/aiida_download/.  Finally, most of the material
simulation codes are open source. As described in the proposal, also the
remaining ones are in the process of introducing a multiple-licensing system
to comply with the open-source requirements.

#  Relevant institutional, departmental or study policies on data sharing and
data security

Data will be generated by different institutions, and the data policies of the
respective institutions will apply. For data shared on  materialscloud.org,
the policies of EPFL and CSCS will apply (as the data is expected to be stored
at these two institutions. In particular, EPFL provides a combined document
“Directive concerning research integrity and good scientific practice at EPFL
(LEX 3.3.2)”  [25] for all data policies. The data policies from CSCS are
instead explained in the document  “Data_Storage_policy_V2.pdf”  [26].