The implementation of a Data Policy can start only with the availability of a
metadata catalogue software. Such a software manages the metadata of raw and
derived data taken at experiment facilities (i.e. partners of CALIPSOplus). In
a metadata catalogue different types of metadata are saved: 1) _administrative
metadata:_ data management lifecycle, ownership, file catalog and 2)
_scientific metadata:_ describing the sample, the beamline and experiment as
well as parameters relevant for data analysis.

A data catalogue software enables management of the data lifecycle, i.e. from
data aquisition to data analysis and eventual deletion of the data. The data
can be linked to proposals and samples, to publications (DOI, PID) and can be
migrated to and from longterm storage on tape.

A metadata catalogue helps keeping track of the data provenance (i.e. the
steps leading to the final results) and it allows to check scientific
integrity (checksum of data). It allows to find data based on the metadata
(i.e. the users own data and handles open access to data). In the long term:
metadata catalogues will help to automate standardised analysis workflows and
support the standardisation of data formats.

_Has your facility some e-infrastructures like metadata catalogue software in
place?_  
  
<table>  
<tr>  
<th>

</th>  
<th>

**CALIPSOplus partners**

</th>  
<th>

**Country**

</th>  
<th>

**Has your facility some e-infrastructures like Metadata Catagogue Software in
place?**

</th> </tr>  
<tr>  
<td>

1

</td>  
<td>

HELMHOLTZ-ZENTRUM DRESDENROSSENDORF EV Germany

</td>  
<td>

Germany

</td>  
<td>

Not planned yet

</td> </tr>  
<tr>  
<td>

2

</td>  
<td>

ANKARA UNIVERSITESI

</td>  
<td>

Turkey

</td>  
<td>

Not planned yet

</td> </tr>  
<tr>  
<td>

3

</td>  
<td>

AARHUS UNIVERSITET

</td>  
<td>

Denmark

</td>  
<td>

Not planned yet

</td> </tr>  
<tr>  
<td>

4

</td>  
<td>

ALBA - CONSORCIO PARA LA

CONSTRUCCION EQUIPAMIENTO Y

EXPLOTACION DEL LABORATORIO DE

LUZ DE SINCROTRON

</td>  
<td>

Spain

</td>  
<td>

Alba offers as well a portal for remote access to data from the experimental
team authenticated with the proposal ID. Alba is working on the implementation
of ICAT metadata catalogues for the beamlines. This will be progressively
implemented as well as the whole data policy on the beamlines. The plan is to
have the first prototype working on a beamline by 2018. In parallel ALBA is
also working in other specific macromolecular metadata laboratory information
management systems (ISPyB)

</td> </tr>  
<tr>  
<td>

5

</td>  
<td>

CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE

</td>  
<td>

France

</td>  
<td>

Not planned yet

</td> </tr>  
<tr>  
<td>

6

</td>  
<td>

STIFTUNG DEUTSCHES ELEKTRONENSYNCHROTRON DESY

</td>  
<td>

Germany

</td>  
<td>

ICAT metadata catalogue

</td> </tr>  
<tr>  
<td>

7

</td>  
<td>

DIAMOND LIGHT SOURCE LIMITED

</td>  
<td>

United Kingdom

</td>  
<td>

ICAT metadata catalogue

</td> </tr>  
<tr>  
<td>

8

</td>  
<td>

ELETTRA - SINCROTRONE TRIESTE SCPA

</td>  
<td>

Italy

</td>  
<td>

ICAT metadata catalogue planned

  * We collect different type of data. Each beamline has its own data acquisition system. 

  

  * Most of the beamline acquisition systems are implemented using the Tango control system. Data acquired are saved in the storage system in an area called scratch. 

  

  * Tango, Labview 

  

</td> </tr>  
<tr>  
<td>

9

</td>  
<td>

EUXFEL - EUROPEAN X-RAY FREE-

ELECTRON LASER FACILITY GMBH

</td>  
<td>

Germany

</td>  
<td>

Planned

</td> </tr>  
<tr>  
<td>

10

</td>  
<td>

HELMHOLTZ-ZENTRUM BERLIN FUR MATERIALIEN UND ENERGIE GMBH

</td>  
<td>

Germany

</td>  
<td>

We use a hierarchical storage management system for the central storage and
long time archival of the data. The bulk of the data will be on tape. We use
ICAT as the metadata catalogue and user portal for the access to the data.

</td> </tr>  
<tr>  
<td>

11

</td>  
<td>

ISTITUTO NAZIONALE DI FISICA NUCLEARE

</td>  
<td>

Italy

</td>  
<td>

Not yet in place

</td> </tr>  
<tr>  
<td>

12

</td>  
<td>

ESRF - INSTALLATION EUROPEENNE DE RAYONNEMENT SYNCHROTRON

</td>  
<td>

France

</td>  
<td>

ICAT metadata catalogue

</td> </tr>  
<tr>  
<td>

13

</td>  
<td>

KIT - KARLSRUHER INSTITUT FUER TECHNOLOGIE

</td>  
<td>

Germany

</td>  
<td>

Not planned yet

</td> </tr>  
<tr>  
<td>

14

</td>  
<td>

LUNDS UNIVERSITET

</td>  
<td>

Sweden

</td>  
<td>

MELANI

</td> </tr>  
<tr>  
<td>

15

</td>  
<td>

PSI - PAUL SCHERRER INSTITUT

</td>  
<td>

Switzerland

</td>  
<td>

MELANI

</td> </tr>  
<tr>  
<td>

16

</td>  
<td>

FELIX - STICHTING KATHOLIEKE UNIVERSITEIT

</td>  
<td>

Netherlands

</td>  
<td>

Not yet in place

</td> </tr>  
<tr>  
<td>

17

</td>  
<td>

SESAME - SYNCHROTRON-LIGHT FOR

EXPERIMENTAL SCIENCE AND

APPLICATIONS IN THE MIDDLE EAST

</td>  
<td>

Jordan

</td>  
<td>

Not in user operation yet

</td> </tr>  
<tr>  
<td>

18

</td>  
<td>

Société Civile Synchrotron SOLEIL

</td>  
<td>

France

</td>  
<td>

Not yet in place

</td> </tr>  
<tr>  
<td>

19

</td>  
<td>

SOLARIS - UNIWERSYTET JAGIELLONSKI

</td>  
<td>

Poland

</td>  
<td>

Not yet in place

</td> </tr> </table>

Table 3. Implementation of metadata catalogue software

_How data curation is regulated at your facility?_  
  
<table>  
<tr>  
<th>

</th>  
<th>

**CALIPSOplus partners**

</th>  
<th>

**Country**

</th>  
<th>

**How data curation is regulated at your facility?**

</th> </tr>  
<tr>  
<td>

1

</td>  
<td>

HELMHOLTZ-ZENTRUM DRESDENROSSENDORF EV Germany

</td>  
<td>

Germany

</td>  
<td>

\-

</td> </tr> </table>  
<table>  
<tr>  
<th>

2

</th>  
<th>

ANKARA UNIVERSITESI

</th>  
<th>

Turkey

</th>  
<th>

\-

</th> </tr>  
<tr>  
<td>

3

</td>  
<td>

AARHUS UNIVERSITET

</td>  
<td>

Denmark

</td>  
<td>

The curation of data is carried out by the beam line scientists. All of the
original data files are kept here at the facility, but users are given copies
of all their data for analysis and publication.

</td> </tr>  
<tr>  
<td>

4

</td>  
<td>

ALBA - CONSORCIO PARA LA

CONSTRUCCION EQUIPAMIENTO Y

EXPLOTACION DEL LABORATORIO DE

LUZ DE SINCROTRON

</td>  
<td>

Spain

</td>  
<td>

Access to raw data and the associated metadata obtained from a public access
experiment is restricted to the experimental team for 3 years. After this
embargo period, the data can be made publicly available. These data are
remotely accessible by the research group. Data preprocessing and analysis is
depending on the Beamline partially done at the Alba premises and completed by
the researchers at their home institutes. This means that the data repository
contains always raw data and in some cases processed and curated data.

</td> </tr>  
<tr>  
<td>

5

</td>  
<td>

CENTRE NATIONAL DE LA RECHERCHE SCIENTIFIQUE

</td>  
<td>

France

</td>  
<td>

\-

</td> </tr>  
<tr>  
<td>

6

</td>  
<td>

STIFTUNG DEUTSCHES ELEKTRONENSYNCHROTRON DESY

</td>  
<td>

Germany

</td>  
<td>

Long term archiving in tape library (up to 10 years)

</td> </tr>  
<tr>  
<td>

7

</td>  
<td>

DIAMOND LIGHT SOURCE LIMITED

</td>  
<td>

United Kingdom

</td>  
<td>

\-

</td> </tr>  
<tr>  
<td>

8

</td>  
<td>

ELETTRA - SINCROTRONE TRIESTE SCPA

</td>  
<td>

Italy

</td>  
<td>

  * The users decide if the data acquired are of good quality and if the case they transform them into datasets. Datasets are saved in an area of the storage system called online where they can be processed, accessed. In the long term the idea is to move the data from the online area to the offline data that in principle can be remote. 

  

  * Preferred data format is HDF5. 

  

</td> </tr>  
<tr>  
<td>

9

</td>  
<td>

EUXFEL - EUROPEAN X-RAY FREE-

ELECTRON LASER FACILITY GMBH

</td>  
<td>

Germany

</td>  
<td>

Long term archiving in tape library (up to 10 years) • Preferred data format
is HDF5.

</td> </tr>  
<tr>  
<td>

10

</td>  
<td>

HELMHOLTZ-ZENTRUM BERLIN FUR MATERIALIEN UND ENERGIE GMBH

</td>  
<td>

Germany

</td>  
<td>

We are currently implementing the Data Policy. Data curation is still work in
progress.

</td> </tr>  
<tr>  
<td>

11

</td>  
<td>

ISTITUTO NAZIONALE DI FISICA NUCLEARE

</td>  
<td>

Italy

</td>  
<td>

We backup data on dedicated external Hard Drives

</td> </tr>  
<tr>  
<td>

12

</td>  
<td>

ESRF - INSTALLATION EUROPEENNE DE RAYONNEMENT SYNCHROTRON

</td>  
<td>

France

</td>  
<td>

The deadline for implementing the data policy on all ESRF beamlines is in
2020. At the moment 11 beamlines are connected to the metadata catalogue (6
are inprogress), data archiving is work in progress at 17 beamlines. This is
long term archiving in tape library (data are being archived for 10 years in
tape archive)

</td> </tr>  
<tr>  
<td>

13

</td>  
<td>

KIT - KARLSRUHER INSTITUT FUER TECHNOLOGIE

</td>  
<td>

Germany

</td>  
<td>

\-

</td> </tr>  
<tr>  
<td>

14

</td>  
<td>

LUNDS UNIVERSITET

</td>  
<td>

Sweden

</td>  
<td>

Not yet in place

</td> </tr>  
<tr>  
<td>

15

</td>  
<td>

PSI - PAUL SCHERRER INSTITUT

</td>  
<td>

Switzerland

</td>  
<td>

  * Data Analysis as a service DaaS project: focusses on offline data analysis and large offline disk storage. Finishes end of October 2017 

  

  * Petabyte Archive: focuses on enabling the data flows to and from a longterm data storage at HPC CSCS/Lugano. Finishes end of 2017 (data storage up to 10 years) 

  

  * Data Curation Project, collaboration with ESS, focusses on data catalog and data analysis automation. Started 2017 and will last until end of 2019. Enabled to add dedicated manpower for data curation tasks. 

  

</td> </tr>  
<tr>  
<td>

16

</td>  
<td>

FELIX - STICHTING KATHOLIEKE UNIVERSITEIT

</td>  
<td>

Netherlands

</td>  
<td>

The experimental data are stored on a facility server; maintainenance and
back-up is organized centrally by the Radboud University IT department. We are
currently exploring and evaluating the possibilities to use local (Radboud)
and a national (e.g. DANS) repositories.

</td> </tr>  
<tr>  
<td>

17

</td>  
<td>

SESAME - SYNCHROTRON-LIGHT FOR

EXPERIMENTAL SCIENCE AND

APPLICATIONS IN THE MIDDLE EAST

</td>  
<td>

Jordan

</td>  
<td>

Not in user operation yet

</td> </tr>  
<tr>  
<td>

18

</td>  
<td>

Société Civile Synchrotron SOLEIL

</td>  
<td>

France

</td>  
<td>

Not yet in place

</td> </tr>  
<tr>  
<td>

19

</td>  
<td>

SOLARIS - UNIWERSYTET JAGIELLONSKI

</td>  
<td>

Poland

</td>  
<td>

Not yet in place

</td> </tr> </table>

Table 4. Data curation at different facilities.

**Implementation of Data Management Plan for CALIPSOplus**

As is clear from the survey 11 of 19 partners, by the end of 2018 will have a
data policy in place based on the PaNdata data policy (Deliverable D2.1. of
PaNdata Europe FP7 project in 2011). The PaNData data policy frame work
defines (long term) goals concerning data storage, life cycle management, data
access and ownership. Implementation of PSI data policy needs a metadata
catalogue. This implies that the implementation of the data policies can only
start with the availability of metadata catalogue software. Some facilities
use the iCAT software (see Table 2) that was developed within the PaNdata ODI
FP7 project, others are currently developing a new metadata catalogue software
called MELANI (PSI, ESS, MAXIV) that is developed within the Data Analysis as
A Service (DAAS) project of PSI. Implementation of the Data Policies is done
step by step (i.e. role out from beamline to beamline) at all facilities. This
stepwise process implies that a DMP will only be complete once these processes
at the different facilities have been finished.

**FAIR Data management at a glance: DMP components to be covered**

  1. **Data summary:** Within this project **data** are **collected** during the experiments at the facilities. The data are collected by users that received transnational access money from the CALIPSOplus project. Most data are collected in the HDF5 or Nexus **format** . Data are open access after the embargo period of 3-5 years and can be reused by third parties. Data origin is from the experimental station of the CALIPSoplus partner large scale facilities. 

  

  2. The two types of metadata catalogues planned to be used by the CALIPSOplus partners (ICAT and MELANI; see Table 3) make data findable, accessable, interoparable and reusable **(FAIR data),** with standards for metadata creation. 

  

  

  2. 2.The timeline in which **data** become **accessible** is defined by the embargo period (3-5 years) in the specific facility data policy (see example of data policy in ANNEX 1). 

  

  

2.3 The **interoperability of the data** is guaranteed by the metadata
catalogue softwares in place.

  3. The implementation of the data policies at the different facilities and with this the putting in place of metadata catalogue software is a just started and an ongoing process. Therefore the **allocation of recources** , i.e. the cost of making our data FAIR and the costs for long term preservation of data will be descibed in detail in the next update of our DMP which is due in month 19 of the project. 

  

  

**Conclusion**

The stepwise implementation of the Data Policies (i.e. role out from beamline
to beamline) at all facilities implies that a DMP will only be complete once
these processes at the different facilities have been finished. The present
version of the DMP reflects the status as it is now for the different partners
of CALIPSOplus. In the next update of the DMP in month 19, a clear progress in
role out of this process will be visible.