<figure><img src="image_1.png" /></figure>

HAL
open science

FINAL ANALYSIS REPORT ON USE OF IVOA STANDARDS FOR FAIR ESFRI AND COMMUNITY DATA AND BEST STEWARDSHIP PRACTICES FOR VALUE-ADDED DATA

Mark G. Allen, Stefania Amodeo, François Bonnarel, Françoise Genova,
Hendrik Heinl, Marco Molinaro, Martino Romaniello

► To cite this version:

Mark G. Allen, Stefania Amodeo, François Bonnarel, Françoise Genova, Hendrik Heinl, et al. FINAL ANALYSIS REPORT ON USE OF IVOA STANDARDS FOR FAIR ESFRI AND COMMUNITY DATA AND BEST STEWARDSHIP PRACTICES FOR VALUE-ADDED DATA. D4.8, ESCAPE project report D4.8. 2022. hal-05201986

HAL Id: hal-05201986

https://hal.science/hal-05201986v1

Submitted on 6 Aug 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

Public Domain
<figure><img src="image_2.png" /></figure>

<figure><img src="image_3.png" /></figure>

ESCAPE

European Science Cluster of Astronomy &
Particle physics ESFRI research Infrastructures

<figure><img src="image_4.png" /></figure>

Project Title European Science Cluster of Astronomy & Particle physics ESFRI research Infrastructure

Project Acronym ESCAPE

Grant Agreement No 824064

Instrument Research and Innovation Action (RIA)

Topic Connecting ESFRI infrastructures through Cluster projects (INFRA-EOSC-4-2018)

Start Date of Project 04.02.2019

Duration of Project 48 Months

Project Website www.projectescape.eu

## D4.8 - FINAL ANALYSIS REPORT ON USE OF IVOA STANDARDS FOR FAIR ESFRI AND COMMUNITY DATA AND BEST STEWARDSHIP PRACTICES FOR VALUE-ADDED DATA

Work Package WP4, Connecting ESFRI projects to EOSC through VO

Lead Author (Org) M. ALLEN (CNRS/ObAS)

Contributing Author(s) (Org) S. AMODEO (CNRS/ObAS), F. BONNAREL (CNRS/ObAS), F. GENOVA (CNRS/ObAS), H. HEINL (CNRS/ObAS), M. MOLINARO (INAF), M. ROMANIELLO (ESO)

Due Date 30.09.2022, M40

Date 15.11.2022

Version 1.0

### Dissemination Level

<figure><img src="image_5.png" /></figure>

PU: Public

<figure><img src="image_6.png" /></figure>

PP: Restricted to other programme participants (including the Commission)

<figure><img src="image_7.png" /></figure>

RE: Restricted to a group specified by the consortium (including the Commission)

<figure><img src="image_8.png" /></figure>

CO: Confidential, only for members of the consortium (including the Commission)
## Versioning and contribution history

<table><thead><tr><td>Version</td><td>Date</td><td>Authors</td><td>Notes</td></tr></thead><tbody><tr><td>0.1</td><td>04/09/202</td><td>M. Allen (CNRS/ObAS)</td><td>Draft outline proposed to co-authors. This starting point is delayed due to effort being applied to finalising D4.7.</td></tr><tr><td>0.2</td><td>22/09/2022</td><td>F. BONNAREL (CNRS/ObAS), F. GENOVA (CNRS/ObAS), M. MOLINARO (INAF), M. ROMANIELLO (ESO), S. AMODEO (CNRS/ObAS)</td><td>Co-authors' comments included.</td></tr><tr><td>0.3-0.7</td><td>04/11/2022</td><td></td><td>Results presented in 'ESCAPE to Future' event integrated into draft.</td></tr><tr><td>0.8</td><td>06/11/2022</td><td></td><td>Text checked by co-authors.</td></tr><tr><td>0.9</td><td>08/11/2022</td><td></td><td>Co-author changes / additions implemented.</td></tr><tr><td>0.95</td><td>09/11/2022</td><td></td><td>Version sent to all WP4 partners.</td></tr><tr><td>1.0</td><td>15/11/2022</td><td></td><td>Version 1.0 submitted to Project Manager.</td></tr></tbody></table>

## Disclaimer

ESCAPE - The European Science Cluster of Astronomy & Particle Physics ESFRI Research Infrastructures has received funding from the European Union's Horizon 2020 research and innovation programme under the Grant Agreement n° 824064.
Table of contents

<table><tr><td>Table of contents</td><td>3</td></tr><tr><td>Acronym list</td><td>4</td></tr><tr><td>Project Summary</td><td>8</td></tr><tr><td>Executive Summary</td><td>9</td></tr><tr><td>1. Introduction</td><td>10</td></tr><tr><td>2. FAIR practices in astronomy – The astronomical Virtual Observatory</td><td>11</td></tr><tr><td>2.1. The science drivers of astronomical data sharing</td><td>11</td></tr><tr><td>2.2. Update on FAIR principles put into action in astronomy</td><td>12</td></tr><tr><td>3. Methodology and work plan</td><td>16</td></tr><tr><td>3.1 Technology Forums</td><td>16</td></tr><tr><td>3.2 ESCAPE participation in the International Virtual Observatory Alliance (IVOA)</td><td>17</td></tr><tr><td>3.3 European Data Provider Forum and Hands-on Workshop</td><td>17</td></tr><tr><td>4. Results</td><td>19</td></tr><tr><td>4.1 Results on the development and update of common interoperability standards</td><td>19</td></tr><tr><td>4.2 Highlight results for interoperable visualization techniques and tools</td><td>21</td></tr><tr><td>4.3 Highlight results for data provenance</td><td>22</td></tr><tr><td>4.4 Highlight results for gravitational wave follow-up observations</td><td>23</td></tr><tr><td>4.5 Highlight results for radio astronomy services</td><td>24</td></tr><tr><td>4.6 Highlight results for solar physics data services</td><td>25</td></tr><tr><td>5. Analysis</td><td>27</td></tr><tr><td>5.1 Analysis of IVOA standards and FAIR concepts</td><td>27</td></tr><tr><td>5.2. Best Practices for Data Stewardship</td><td>31</td></tr><tr><td>6. Conclusion</td><td>33</td></tr><tr><td>Appendix 1 – Dedicated meetings for VO framework developments for ESFRI and RIs</td><td>34</td></tr><tr><td>Appendix 2 – IVOA Interoperability Meeting – Milestones</td><td>36</td></tr></table>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>
Acronym list

AAI: Authentication and Authorisation Infrastructure

ADASS: Astronomical Data Analysis Software and Systems

ADQL: Astronomical Data Query Language

ALMA: Atacama Large Millimetre Array

ALTA: Aptertif Long Term Archive at Astron

ASTERICS: Astronomy ESFRI & Research Infrastructure Cluster

ASTRON: the Netherlands Institute for Radio Astronomy

ASTRONET: A comprehensive long-term planning for development of European astronomy

B2FIND: EUDAT metadata service and discovery portal

CERN: European Organization for Nuclear Research

CEVO: Connecting ESFRI projects to EOSC through the Virtual Observatory framework (ESCAPE Work Package 4)

Characterisation: Characterisation Data Model

CNRS: Centre National de la Recherche Scientifique

CNRS-ObAS: CNRS - Observatoire Astronomique de Strasbourg

CTA: Cherenkov Telescope Array

CTAO: Cherenkov Telescope Array Observatory

D#: Deliverable number

DaCHS: Data Center Helper Suite for many VO protocols and standards developed at UHEI

DADI: Data Access, Discovery and Interoperability (ASTERICS Work Package 4)

DataLink: Dta access protocol for cross service resource linking

ECAP: Erlangen Center for Astroparticle Physics

EGO-Virgo: European Gravitational-Wave Observatory

ELT: Extremely Large Telescope (was E-ELT)

EOSC: European Open Science Cloud

EPN: Europlanet Society

EPN-TAP: Europlanet Table Access Protocol

ESCAPE: European Science Cluster of Astronomy & Particle physics ESFRI research infrastructures

ESFRI: European Strategy Forum on Research Infrastructures

ERIC: European Research Infrastructure Consortium

ESASky: Application to access and visualise astronomical data
ESO: European Southern Observatory

EST: European Solar Telescope ASTRONET

EUDAT: European DATA (e-infrastructure)

EVN: The European VLBI Network

EVN Data Archive: Data Archive containing correlated data associated with EVN observations

FAIR: Findable, Accessible, Interoperable, Reusable or Facility for Antiproton and Ion Research

FAIRsFAIR: Fostering Fair Data Practices in Europe

FAIRSharing: A curated, informative and educational resource on data and metadata standards, inter-related to databases and data policies.

FITS: Flexible Image Transport System

FREQSYS: Metadata for frequency/wavelength in VOTable

HDF5: Hierarchical Data Format, version 5

HEALPix: Hierarchical Equal Area isoLatitude Pixelisation of a 2-sphere

HiPS: Hierarchical Progressive Survey, based on HEALPix

INAF: Istituto Nazionale di Astrofisica

INFN: Instituto Nacional de Fisica Nucleare

INTA: Instituto Nacional de Tecnica Aeroespacial

IPDA: International Planetary Data Alliance

IR: Infra-Red

IRAM: Institute de Radioastronomie Millimetrique

IVOA: International Virtual Observatory Alliance

Jhelioviewer: visualization software for solar image data

JIVE: Joint Institute for VLBI ERIC

KAGRA: Kamioka Gravitational Wave Detector

KIS: Kiepenheuer Institut für SonnenPhysik

KM3NeT: A multi-km3 sized Neutrino Telescope

LIGO: Laser Interferometer Gravitational-Wave Observatory

LIGO-VIRGO-KAGRA: Scientific collaboration between LIGO, VIRGO and KAGRA

LOFAR: Low Frequency Array

LUPM: Laboratoire Univers et Particules de Montpellier

MS#: Milestone number

MOC: Multi-Order Coverage

mocopy: Python library to create and manipulate MOCs
NASA: National Aeronautics and Space Administration

NenuFAR: New extension in Nançay upgrading LOFAR

OAI-PMH (Open Archives Initiative Protocol for Metadata Harvesting)

ObsCore: Observation Core Component Data Model

ObsParis: Observatoire de Paris

ObsTAP: Observation Core Component Data Model using the Table Access Protocol

OSSR: Open-source scientific Software and Service Repository, ESCAPE WP3

ProvDM: Provenance Data Model to enable traceability of data products.

ProvTAP: Serialization of the Provenance Data Model with TAP

RDF: Resource Description Framework

RI: Research Infrastructure

ROB: Royal Observatory of Belgium

SAMP: Simple Application Messaging Protocol

SIA: Simple Image Access

SKA: Square Kilometre Array

SKAO: SKA Observatory

SODA: Server-side Operations for Data Access

SOLARNET: H2020 Programme for Integrating High Resolution Solar Physics

StandardsRegExt: Extension of the resource schema to standards

STMOC: Space Time MOC

TAP: Table Access Protocol

TAPRegExt: Extension of the resource schema to TAP resources

TIMESYS: Metadata for time in VOTable

UCDs: Unified Content Descriptors

UEDIN: University of Edinburgh

UHEI: Ruprecht-Karls-Universität Heidelberg

USET: Uccle Solar Equatorial Table

UV: Ultraviolet

UWS: Universal Worker Service

VAMDC: Virtual Atomic and Molecular Data Centre

Virgo: Virgo Collaboration of the gravitational-wave detector hosted by EGO-Virgo

VLBI: Very-long-baseline interferometry

VO: Virtual Observatory
**VO oli Data Service:** Extension of the IVOA Resource Metadata to describe data collections and the
services to access them

**VOEvent:** IVOA standard to represent, transmit, publish and archive transient celestial events.

**VOLLT:** JAVA library implementing the IVOA standards TAP/ADQL and UWS developed at CNRS-ObAS

**VO Resource:** XML encoding schema for resource metadata
**VOEvent:** Virtual Observatory Event

**VO Table:** Virtual Observatory Table format standard

**W3C:** World Wide Web Consortium

**WebGL:** Web Graphics Library

**WG:** Working Group

**XML:** eXtensible Markup Language
## Project Summary

ESCAPE (European Science Cluster of Astronomy & Particle physics ESFRI research infrastructures) addresses the Open Science challenges shared by ESFRI facilities (SKA, CTA, KM3Net, EST, ELT, HL-LHC, FAIR) as well as other pan-European research infrastructures (CERN, ESO, JIVE) in astronomy and particle physics. ESCAPE actions are focused on developing solutions for the large data sets handled by the ESFRI facilities. These solutions shall: i) connect ESFRI projects to EOSC ensuring integration of data and tools; ii) foster common approaches to implement open-data stewardship; iii) establish interoperability within EOSC as an integrated multi-messenger facility for fundamental science. To accomplish these objectives ESCAPE aims to unite astrophysics and particle physics communities with proven expertise in computing and data management by setting up a data infrastructure beyond the current state-of-the-art in support of the FAIR principles. These joint efforts are expected result into a data-lake infrastructure as cloud open-science analysis facility linked with the EOSC. ESCAPE supports already existing infrastructure such as astronomy Virtual Observatory to connect with the EOSC. With the commitment from various ESFRI projects in the cluster, ESCAPE will develop and integrate the EOSC catalogue with a dedicated catalogue of open source analysis software. This catalogue will provide researchers across the disciplines with new software tools and services developed by astronomy and particle physics community. Through this catalogue ESCAPE will strive to cater researchers with consistent access to an integrated open-science platform for data-analysis workflows. As a result, a large community "foundation" approach for cross-fertilization and continuous development will be strengthened. ESCAPE has the ambition to be a flagship for scientific and societal impact that the EOSC can deliver.

<figure><img src="image_11.png" /></figure>

<figure><img src="image_12.png" /></figure>
# Executive Summary

This document reports the final version of the analysis of the use of IVOA standards for FAIR ESFRI and community data, and also describes a number of best practices for data stewardship that have been identified throughout this work. The analysis has been done as a part of Task 4.2 "Implementation of the FAIR principles for ESFRI data through the Virtual Observatory" and Task 4.3 "Adding value to trusted content in astronomy archives" of ESCAPE Work Package 4 "Connecting ESFRI projects to EOSC through VO framework" (CEVO).

The Virtual Observatory (VO) is the international data sharing framework which enables astronomy data to be findable, accessible, interoperable and re-usable. The International Virtual Observatory Alliance (IVOA) is an organisation that debates and agrees the technical standards that are needed to make the VO possible. Over the course of the ESCAPE project a program of work has been pursued to advance the definition and adoption of common open IVOA standards for interoperability based on requirements of the ESFRIs and other Research Infrastructures of pan-European interest for their use of FAIR practices using the VO framework. We describe the evolution of FAIR practices in astronomy and how these are put into practice by VO standards, tools and services.

The methodology applied is to bring together the partners who have expertise in VO technologies and development of standards, with partners from the ESFRIs and other RIs who have expertise in their data. In this way the ESFRIs and RIs become engaged with the definition of the interoperability standards by being actors in the IVOA.

The outcomes of the work include the definition of new and updated standards, the implementation of new services, and also the development of new tools. These results are tabulated in the report with identification of the relevance to the various ESFRIs and RIs and also to the different thematic areas of astronomy, astroparticle physics and solar physics. These standards can be used by the whole astronomical community world-wide, well beyond the infrastructures participating in CEVO.

We analyse the use of the VO standards for the FAIRisation of astronomical data, and identify key elements of data stewardship for their implementation and use to support Open Science. We conclude that strong engagement with the communities of astronomy data providers, the service and tool developers, and the community of scientific users (in particular early career researchers) is essential for the coordination of efforts for acceleration of Open Science in astronomy.

This final version of the analysis builds on the intermediate results presented in D4.2 and has been prepared as a major update of the initial analysis.
# 1. Introduction

Work Package 4 of ESCAPE, “Connecting ESFRI projects to EOSC through VO framework” (CEVO), has pursued a program of work to make the seamless connection of ESFRI and other astronomy and astroparticle research infrastructures to the EOSC through the Virtual Observatory (VO) framework. The high-level objective of Task 4.2 “Implementation of the FAIR principles<fnref n="1" /> for ESFRI data through the Virtual Observatory” is the definition and adoption of common open VO standards for interoperability based on ESFRI requirements. The connection with EOSC is ensured via Task 4.1 which has concluded its final report in D4.7. The standards can be used by the whole astronomical community world-wide, well beyond the infrastructures participating in CEVO, and are appropriate standards to be considered for the EOSC Interoperability Framework.

The Virtual Observatory is a framework of open standards for making astronomy data FAIR – Findable, Accessible, Interoperable, Reusable. It is an established and operational interoperability framework that has proven to be a great success for many aspects of the interoperability and FAIRness of astronomy data. It is an essential component of the astronomy data landscape, as has been strongly stressed in the ASTRONET Infrastructure Roadmaps since its first publication in 2008 and in the 2014 Infrastructure Roadmap update<fnref n="2" />, and in the new ASTRONET Roadmap currently under consultation in 2022³. The VO framework essentially enables the international astronomy data providers, in particular ground- and space-based observatories to publish their data using the IVOA standards. Compliant scientific tools and services then enable the discovery, access and (re-)use of the data by the whole astronomy research community. A description of the recently updated Virtual Observatory architecture and an update of the status of FAIR practices in astronomy is provided in Section 2.

The inclusion of the data of the ESFRI facilities from astronomy and astroparticle physics in the VO was already advanced in a number of areas thanks to the collaboration between ESFRI pathfinders and European VO teams in the ASTERICS Data Access, Discovery and Interoperability (DADI) Work Package since 2015. The CEVO objectives have built on these developments to make the seamless connection of ESFRI and other astronomy research infrastructures to the EOSC through the VO. Task 4.2 refines and further pursues implementation of FAIR principles for astronomy data via the use and development of common standards for interoperability including the extension of the VO to communities<fnref n="4" /> who were new to VO technologies at the beginning of the project.

The methodology applied throughout the project was set up in D4.1, the “Detailed WP4 Project Plan” describes the activities of the three WP4 tasks. This report supersedes the intermediate analysis that was presented in D4.2 (March 2020) and is fully updated with the results and analysis that have been done over the course of the project. This report draws on some information provided in periodic technical reports and milestone reports, and is designed to be self-contained as one of the final products of the project.

<figure><img src="image_13.png" /></figure>

<figure><img src="image_14.png" /></figure>

<footnote n="1">¹ Wilkinson, M., Dumontier, M., Aalbersberg, I. et al. The FAIR Guiding Principles for scientific data management and stewardship. *Sci Data* **3**, 160018 (2016) doi:10.1038/sdata.2016.18 (https://www.nature.com/articles/sdata201618)</footnote>

<footnote n="2">² https://www.astronet-eu.org/archives</footnote>

<footnote n="3">³ https://www.astronet-eu.org</footnote>

<footnote n="4">⁴ Neutrino astrophysics (for KM3NeT) and solar physics (for EST).</footnote>
The methodology and work is described in Section 3. The results are described in Section 4, and the analysis is presented in Section 5. Conclusions and future steps are presented in Section 6.

## 2. FAIR practices in astronomy – The astronomical Virtual Observatory

Astronomy has been a pioneer of Open Data sharing, and remains at the forefront. International agreement on a data format, FITS, and on standards allowing users to find, access and interoperate data, the so-called Virtual Observatory standards, provides a common data sharing framework open to all and enables the development of interoperable tools to access and use data.

FITS was first published in 1981 (Wells, Greisen & Harten<fnref n="5" />). The International Virtual Observatory Alliance<fnref n="6" /> (IVOA) was created in 2002 to define and maintain the astronomical interoperability standards. Its processes are adapted from the W3C ones. Data producers have been providing their data in the Virtual Observatory, and the community has been able to find, access, interoperate and reuse astronomical data years before FAIR was defined in the foundational Nature paper in 2016 (Wilkinson et al<fnref n="7" />). In 2022 more than 128 "authorities" from all around the world, large agencies as well as smaller teams, have declared at least one service in the VO registry of resources. VO-enabled data services provided by the ground and space-based observatory archives as well as by value-added data repositories, together with VO-enabled interoperable tools providing access to data, are used by the community in their daily research work.

### 2.1. The science drivers of astronomical data sharing

Astronomy research infrastructures are most well known as the ground and space-based telescopes, but one can consider the *data as a whole* is also one of the discipline's research infrastructures, which is used by the community in their everyday work. Data are at the core of the disciplinary scientific approach, which requires to be able to find, access, interoperate and reuse them. Combining observations from different instruments is essential to understanding the physical phenomena at work in astronomical objects. *Multi-wavelength astronomy* where observations across multiple electromagnetic wavebands are used is now the norm and is providing a significant fraction of the publications<fnref n="8" />. We have now entered an era of *multi-messenger astronomy* as spectacularly demonstrated by the first identification of the astronomical object at the origin of a gravitational wave event GW170817 and the identification of blazar TXS 0506+956 as the first source of non-stellar cosmic neutrinos. The fact that many astronomical objects are *variable* on a wide range of time scales, their brightness or other characteristics changing with time, or that they have measurable motions on the celestial sphere, is also an important scientific driver to preserve and share data on the long term. As in many other disciplines, researchers also want to compare models with observations. Moreover, maximising the scientific return of expensive instruments is another motivator for reusing data for scientific objectives other than those for which they were initially collected. Open access to existing data is also a key factor when planning for new observations or new facilities.

<figure><img src="image_15.png" /></figure>

<figure><img src="image_16.png" /></figure>

<footnote n="5">⁵ 1981A&AS...44..363W</footnote>

<footnote n="6">⁶ http://ivoa.net</footnote>

<footnote n="7">⁷ https://doi.org/10.1038/sdata.2016.18</footnote>

<footnote n="8">⁸ A few years ago, about half of the ESO publications used data from one or more of the other major observatories.</footnote>
A major characteristic of astronomy is the availability of data in almost all observatories and missions, usually after a short proprietary period (in general 1-2 years) for the data obtained in competitive calls for proposals for observation time. The metadata are generally publicly available during the proprietary period, but data is released when it ends. This provides for an ideal landscape for promoting data sharing.

## 2.2. Update on FAIR principles put into action in astronomy

The list of the FAIR guiding principles from Williamson et al. is given in Table 1 for reference. Figure 1 is a schema of the Virtual Observatory architecture which has been updated to a new version 2.0⁹ in 2021 with the input of an ESCAPE member (M. Molinaro INAF) in the IVOA Technical Coordination Group. The original IVOA architecture<fnref n="10" /> published in 2010 pre-dated the FAIR principles. The new version identifies the direct mapping of its main functionalities and how they relate to FAIR.

**To be Findable:**

F1. (metaicie are assigned a globally unique and persistent identifier

F2. data are described with rich metadata (defined by R1 below)

F3. metadata clearly and explicitly include the identifier of the data it describes

F4. (meta)data are registered or indexed in a searchable resource

**To be Accessible:**

A1. (meta)data are retrievable by their identifier using a standardized communications protocol

A1.1. the protocol is free, open and universally implementable

A1.2. the protocol allows for an authentication and authorization procedure, where necessary

A2. metadata are accessible, even when the data are no longer available

**To be Interoperable:**

I1. (meta)data use a formal, accessible, shared, and broadly applicable language for knowledge representation

I2. (meta)data uses vocabularies that follow FAIR principles

I3. (meta)data include qualified references to other (meta)data

**To be reusable:**

R1. (meta)data are richly described with a plurality of accurate and relevant attributes

R1.1. (meta)data are released with a clear and accessible data usage license

R1.2. (meta)data are associated with data provenance

R1.3. (meta)data meet domain relevant community standards

<caption>**Table 1:** The FAIR Guiding Principles, as defined by Wilkinson et al. 2016</caption>

<figure><img src="image_17.png" /></figure>

<figure><img src="image_18.png" /></figure>

<footnote n="9">⁹ https://ivoa.net/documents/IVOAArchitecture/20211101/index.html</footnote>

<footnote n="10">¹⁰ https://ivoa.net/documents/Notes/IVOAArchitecture/index.html</footnote>
Recent analyses of FAIR in astronomy have been published by IVOA Australian colleagues O'Toole et al. 2022<sup>11</sup> and by ESCAPE colleagues Molinaro et al. 2021<sup>12</sup>, where in each case Interoperability is identified as the core aspect for astronomy.

The IVOA standards<sup>13</sup> are defined by Working Groups which tackle different aspects of data sharing (currently Applications, Data Access Layer, Data Models, Grid & Web Services, Registry of Resources, Semantics). In fact, the IVOA standards enable users to find, access and interoperate data. Reusability is mostly in the hands of the data providers. For observational data, FITS integrates data and metadata, in particular about the observation (telescope, instrument, observation field of view, instrument parameters, etc.). Data providers declare their resources in the IVOA Registry of Resources and implement an "interoperability layer" on top of their data holdings to interface them with the Virtual Observatory using the VO protocols.

The list of VO standards is available and maintained on the IVOA web site at http://www.ivoa.net/documents/. One should note here that the IVOA standards are Recommendations endorsed by the IVOA Executive Board after extensive discussion by the community and following a defined process. The two words "standard" and "recommendation" will be used indifferently in the following to refer to them.

As shown in Figure 1, the different facets of FAIR in the astronomy practices can be mapped onto the
architecture of the standards defined by the different IVOA Working Groups:

* Findable: Registry of Resources (and Data Access Protocols)

* Accessible: Data Access Protocols

* Interoperable: Data Models, Semantics

* Reusability: Figure 1 refers to the implementation of reusability by the data providers in the resource layer. Going back to the FAIR Guiding Principles, all the IVOA standards ("community standards") are relevant, in particular those which define metadata. Provenance is specifically cited in R1.2.

The Applications Working Group activities concerning tools and services are relevant to the four facets of FAIR.

<footnote n="11"><sup>11</sup> https://ui.adsabs.harvard.edu/abs/2022arXiv220310710O/abstract</footnote>

<footnote n="12"><sup>12</sup> https://ui.adsabs.harvard.edu/abs/2021arXiv211114468M/abstract</footnote>

<footnote n="13"><sup>13</sup> http://www.ivoa.net/documents/</footnote>
<figure><img src="image_19.png" /><figcaption>**Figure 1:** The VO architecture, and how it enables users to find, access and interoperate data.</figcaption></figure>

The Virtual Observatory is an open, global data sharing framework, with no central point. Anyone can declare a service in a VO Registry of Resources, and the different registries harvest each other regularly to provide a complete and uniform view of VO-enabled data sources. Anyone can develop and share a tool to access these data, and the VO provides standards to make the tools interoperable. An example of VO tool is displayed in Figure 2: the Aladin interactive sky atlas developed by the Centre de Données astronomiques de Strasbourg<fnref n="14" /> and used in the CEVO training events. One can visualise all the positions in the sky for which observatory archives or data bases provide data in the image field. If the data is an image, Aladin can visualize the image and compare and combine it with others. If the data is for instance a spectrum, it can seamlessly link by a simple click to a tool which will display the spectrum.

The Virtual Observatory framework is operational, and is constantly being evolved and extended to take into account the needs of the astrophysics community, in particular to take into account new sub-domains of astrophysics. It was expanded to include astroparticle physics in the ASTERICS<fnref n="15" /> Cluster, and this is taken much further in ESCAPE. Furthermore ESCAPE has made significant

<footnote n="14">14 http://cds.unistra.fr/</footnote>

<footnote n="15">15 https://www.asterics2020.eu/</footnote>
## 3. Methodology and work plan

CEVO brings together ESCAPE partners who have expertise in the VO framework, with partners who are connected to the ESFRI projects and other research infrastructures. The VO expertise is provided by the partners: CNRS-ObAS, INAF, INTA, UEDIN, UHEI who have built up the Euro-VO alliance of national European projects since 2002. ObsParis is also a VO expert and it brings a special link between VO and the Cherenkov Telescope Array Observatory (CTAO). The ESFRIs and other research infrastructures who are directly involved in CEVO are: the ESFRI projects the European Solar Telescope (EST), and the cubic-kilometre-sized Neutrino Telescope (KM3NeT) and the ESFRI landmark projects the Cherenkov Telescope Array Observatory (CTAO), the Extremely Large Telescope (ELT) and the Square Kilometre Array (SKA). In addition to the ELT, the pan-European International Organization European Southern Observatory (ESO) brings other world-class established astronomical observatories (e.g. ALMA, the La Silla Paranal Observatory). Additionally the research infrastructures (RI) European Gravitational-Wave Observatory (EGO-Virgo) and the Joint Institute for VLBI ERIC (JIVE) are also participating directly in the work package. ASTRON also brings the SKA pathfinder LOFAR, a distributed facility that it coordinates. The various ESFRIs and other RIs are grouped into domain specific areas, Solar Physics, Radio data, Event-based data, UV/Optical/IR data.

The Detailed Work Plan D4.1 lists the activities which have been guided the work, in particular the sub-tasks of Task 4.2:

* **"Gathering of requirements from ESFRIs for their use of the VO framework and its connection to EOSC"** provided the starting point. It involved a series of in-depth discussions with the ESFRIs and the other Research Infrastructures (RI) participating in CEVO, in dedicated meetings and videoconferences with each RI or RI group. The list of the meetings and visits organised to gather ESFRI and RI requirements is provided in Appendix 1.

* **"Update and definition of standards based on the requirements and priorities, and representation of ESFRI interests in the global VO framework"**, is the core of the work. It has involved a large effort to the update and definition of IVOA standards based on the ESFRI and RI priorities, and includes the work performed in the relevant IVOA Working Groups and Interest Groups.

### 3.1 Technology Forums

The methodology for coordinating the work was to gather all of the WP4 partners in Technology Forum events to report progress and to serve as preparation for the bi-annual IVOA interoperability meetings. Table 2 below lists the Technology Forum events that have been run during the project. Each of these events involved in-depth discussions of on domain/ESFRI-RI needs, VO usage and specific standards, and also the so-called "hack-a-thon" sessions for hands-on problem solving and development. These events also provided a venue for the reporting of results so the collected sets of presentations (available at the links in Table 2) represents a rich resource of experiences of implementing and developing interoperable standards and tools.

<figure><img src="image_20.png" /></figure>

<figure><img src="image_21.png" /></figure>
presentation from ESO on their use of IVOA standards from high level requirements to implementation of selected standards, going through analysis of constraints, evolution of existing archive infrastructure, selection of databases, DBMSes integration and maintenance in the operational environment, using off-the-shelf components, costs (FTEs), obsolescence. The workshop topics covered tools for implementation and operations of services using the IVOA standards: TAP, DaCHS, VOLLT, MOC and HIPS. All of the materials for this training event are available on the event page, and a report<fnref n="28" /> was produced.

<figure><img src="image_22.png" /></figure>

<figure><img src="image_23.png" /></figure>

<footnote n="28">²⁸ MS24 report - https://cloud Secure2020.de/index.php/s/GVkD88VXA88rzRh</footnote>
## 4. Results

Here we present an overview of the results of the work that has been done toward advancing the use of common standards to enable FAIR data. The methods, as described above, were designed to enable the various ESFRI/RIs to become engaged with the entire process of development and implementation of the standards. Below we summarise the level of engagement that has been achieved at the European and international levels, and the outcomes in terms of the new and updated interoperability standards that have been advanced during the project. We provide a number of highlighted results to demonstrate the diverse range of implementations in the areas of: data access and visualisation, systems for managing data provenance (in particular for high energy physics), multi-messenger astronomy, radio astronomy and solar physics.

The successful engagement of ESFRI/RIs in these processes for building the common standards at the international level is evident in the high number of interactions that have occurred at the IVOA interoperability meetings. The WP4 partners have made between 15 and 30 contributions at these milestone events (as tabulated in Appendix 2). This involvement has been followed by actions within the various IVOA Working Groups for the development and update of interoperability standards.

### 4.1 Results on the development and update of common interoperability standards.

The relevant standards to which WP4 partners have contributed to are presented below in Table 3. The table indicates the status of the standards, a short description of the standard as well as an explanation of the relevance of the standard for the ESFRI/RIs in ESCAPE.

<table><thead><tr><td>Name (long name)/ Status / Link</td><td>Description</td><td>Relevance for ESCAPE</td></tr></thead><tbody><tr><td>MOC 2.0 (Multi-order coverage map) Approved IVOA standard Link</td><td>The Multi-Order Coverage map standard enables the indexation of data based on the spatial coverage of the data on the sky, and also the temporal coverage of the data (characterised by the observation time and sampling). It is based on the HEALPix tessellation and related to the HiPS system widely used for data access to very large image surveys.</td><td>Relevant to observational data from all ESFRI/RIs. Provides the basis for innovative techniques developed in ESCAPE for combination of multi-messenger data, including data discovery, access and visualization.<br/>e.g. MOCs are used on Services of Radio Astronomy and to describe the probability maps of Gravitational Waves (LIGO/VIRGO),</td></tr><tr><td>PhotDM 1.1 (Photometry Data Model) Approved IVOA standard Link</td><td>Describes photometry filters, photometric systems, magnitude systems, zero points and its interrelation with the other IVOA data models.</td><td>The PhotDM is relevant for a wide range of observational data from ESFRI/RIs.</td></tr></tbody></table>

<figure><img src="image_24.png" /></figure>

<figure><img src="image_25.png" /></figure>
<table><tr><td>DataLink 1.1<br>IVOA Working Draft<br>Link</td><td>DataLink is a data access protocol with the purpose to provide a mechanism to link resources found via one service to resources provided by other services.</td><td>This generalised mechanism is widely applicable for linking related data sets, and is relevant for all astronomy data services. This upgrade of the DataLink 1.0 (LINK) (which is heavily used for example in the ESO archive) was identified in ESCAPE as a priority for interoperability.<br>e.g. DataLink services are used by ESO, ASTRON and JIVE.</td></tr><tr><td>ProvDM<br>(Provenance Data<br>Model)<br>Approved IVOA<br>standard<br>Link</td><td>Provides a standard for how provenance information can be modelled, stored and exchanged. It enables traceability of data products.</td><td>Tracking the provenance of data products is identified as relevant to all ESFRI/RIs. In particular it has been a priority for CTA, KM3NeT, with strong interest of ESO and SKA.</td></tr><tr><td>ProvTAP 1.0<br>IVOA internal draft</td><td>A TAP/ADQL extension for Provenance Data Model which would enable the querying of TAP services for Provenance information.</td><td>Test implementations of the Provenance Data model have been relevant in particular to CTA and KM3NeT.</td></tr><tr><td>ObsCore Data<br>Model extension for<br>visibility data<br>(Observational Core<br>Data Model)<br>IVOA Note in<br>progress<br>Github Link</td><td>The ObsCore data model standard describes the core metadata about an astronomical observation. It is an extensible model designed to accommodate new types of data. This extension is to include radio astronomy interferometry data (also referred to as 'visibilities').</td><td>This extension is identified as a priority for JIVE and ASTRON for the publication of visibility data.</td></tr><tr><td>SODA 1.1<br>(Server Operations<br>for Data Access)<br>IVOA Working Draft</td><td>The SODA web service interface defines a RESTful web service for performing server-side operations and processing on data before transfer.</td><td>Relevant to ESFRI/RI data services that provide 'cut-out' extractions from large data sets. The updates of SODA 1.0 (Link) have been driven by ESCAPE priorities in particular for Radio Astronomy for the cut-out DataLink services that are expected for large radio data sets.<br>(e.g. SODA services provided by ESO on integral field data, and expected for SKA)</td></tr><tr><td>SIAP 2.1<br>(Simple Image<br>Access Protocol)</td><td>A protocol that supports queries for images available in a given data collection near a given position on the sky.</td><td>The update of the widely implemented SIAP services is a priority across all ESFRI/RIs that provide image data.</td></tr></table>
<table><caption>**Table 3:** IVOA standards contributed to by CEVO</caption><tr><td>IVOA Working Draft</td><td></td><td>(e.g. SIAP services run at ESO, and many VO partner services)</td></tr><tr><td>TimeSeries discovery and access<br>IVOA Note in preparation</td><td>This is an IVOA note on how to annotate light curve data. This is in particular important for all future large surveys that have multiple epochs.</td><td>This standard is identified as relevant for time series data, and will be applicable to Time Domain and multi-messenger astrophysics.<br>(e.g. applications are expected for Rubin/LSST, SKA, ESO and many more)</td></tr><tr><td>Execution Planner – design outline<br>IVOA Note<br>LINK</td><td>A developmental activity to define an 'Execution Planner' web service interface to provide a simple way to discover and access computing services.</td><td>This priority emerged from cross-work-package efforts in ESCAPE between WP2, WP4, WP5 on the topic of interoperability of science analysis platforms. It is relevant to all ESFRI/RIs and projects using science platforms.</td></tr></table>

## 4.2 Highlight results for interoperable visualization techniques and tools

Embeddable visualization using the IVOA HiPS standard and the Aladin Lite visualiser has been identified as a common need across many of the ESCAPE partners (ALMA, EGO/Virgo, ESO, SKA, ASTRON) as well as external interests (e.g. it is used in ESASky<fnref n="29" />). This led to the priority to update Aladin Lite to use WebGL technologies and significant effort has been applied to this over the course of the project. Aladin Lite v3 prototypes have been demonstrated at the ADASS conferences (2020, 2022) with the release date set for January 2023. It includes new features: support of multiple projections (Aitoff, Mollweide, Orthographic, Mercator), support of FITS tiles, display of FITS images, heatmaps visualisation of catalogue data, improved rendering pipeline and coordinates grids. Figure 3 shows Aladin Lite v3 displaying the ESO VVV DR4 survey data, and the MeerKat (SKA pathfinder telescope) dataset on the galactic centre. The interactive zoomable display makes use of the IVOA standardized HiPS format using the newly implemented FITS version that enables the full dynamic range of the data to be used. Aladin Lite v3 is expected to be implemented in various ESFRI/RI systems wherever embedded visualization is needed. It is already being used in the SKA Science Regional Centre visualization prototyping activities.

<footnote n="29">²⁹ https://sky.esa.int</footnote>
<figure><img src="image_26.png" /><figcaption>**Figure 3:** Aladin Lite v3 visualizer that was developed in ESCAPE, displaying the ESO VVV DR4 survey data (base layer), and the MeerKat (SKA pathfinder telescope, overlay layer) dataset on the galactic centre.</figcaption></figure>

## 4.3 Highlight results for data provenance

The Provenance Data Model was identified as a high priority for ESCAPE WP4 partners (in particular CTA and KM3NeT related partners). This subject has reached a level of maturity where the IVOA standard was approved<fnref n="30" /> in April 2020, then a dedicated ESCAPE CEVO workshop<fnref n="31" /> was held on 7-8 September 2020, and ESCAPE CEVO partners led a community “Birds of a Feather - BoF” discussion session during the ADASS conference in November 2020. The workshop was focused on the use of Provenance DM information to explore the traceability of data products, find contact information and acknowledge data providers. Providing provenance information allows a user to assess the quality and reliability of the products, and is a step toward supporting reproducibility. The workshop gathered 28 participants including ESCAPE partners ASTRON, CNRS-ObAS, CTAO-GMBH, ECAP-FAU (WP3), INAF, JIVE, Obs-Paris. This workshop contained presentations on technical solutions, demonstrations, hands-on sessions and discussions, and was used to update and collect the requirements of ESFRI projects in order to build the road map of future developments. The results and feedback of the workshop were brought to the international level at the November 2020 IVOA interoperability meeting (MS23) by M. Servillat (Obs Paris) in a presentation on “*Provenance activities in the European project ESCAPE*”. Furthermore, the workshop and BoF discussion were also complemented by a specific meeting on Provenance standards with (July 9, 2021) (SKAO, Obs-Paris,

<figure><img src="image_27.png" /></figure>

<figure><img src="image_28.png" /></figure>

<footnote n="30">³⁰ https://www.ivoa.net/documents/ProvenanceDM/20200411/index.html</footnote>

<footnote n="31">³¹ https://indico.in2p3.fr/event/21913/</footnote>
CNRS-ObAS and external partner LUPM). Another external workshop was also held on "Provenance in Practice"<fnref n="32" /> in the French community, with contributions from ESCAPE partners. The work has also been presented and published at the "International Provenance and Annotation Workshop" (19-22 July 2021³³) in a paper "Towards a Provenance Management System for Astronomical Observatories"<fnref n="34" />.

4.4 Highlight results for gravitational wave follow-up observations

Sky-spatial and temporal coverage systems for indexing of astronomy data have also been identified as a high-level priority for fast data access and management of complex sky regions. This work has been driven by requirements for gravitational wave follow-up campaigns connected to EGO/Virgo. An important result is the development of the IVOA MOC 2.0 standard which provides a "Multi-Order Coverage" map based on the HEALPix tessellation, with the major upgrade that the MOC 2.0 standard now also provides a hierarchical system for describing the coverage in time. This standard now provides a general capability for being able to search and cross-match astronomy data sets based on their sky coverage and their temporal coverage. The standard is led by CNRS-ObAS with contributions from EGO-Virgo (INFN) and also from external partners including the Vera C. Rubin Observatory, and NASA. The new capabilities were applied by EGO-Virgo and resulted in a journal paper "Multi Order Coverage data structure to plan multi-messenger observations"<fnref n="35" />. The paper includes an interactive python notebook and on-line video tutorial materials, both of which were used in the ESCAPE "Science with interoperable data" school (D4.6). Code libraries for the use of the MOC standard have been produced by CNRS-ObAS, namely the 'MOC Lib Rust'<fnref n="36" /> that has been onboarded to the ESCAPE Open Source Software Repository (OSSR). Figure 4 shows the ESCAPE work that has been integrated into the LIGO-VIRGO-KAGRA Public Alert User Guide for sky map identification and visualizations of the gravitation wave credible regions, and its use in the interactive Aladin application.

<figure><img src="image_29.png" /></figure>

<figure><img src="image_30.png" /></figure>

<footnote n="32">³² https://indico.obspm.fr/event/1267/</footnote>

<footnote n="33">³³ https://link.springer.com/conference/ipaw</footnote>

<footnote n="34">³⁴ M. Servillat et al. 2021 : https://link.springer.com/chapter/10.1007/978-3-030-80960-7_20 (Open Access: https://arxiv.org/pdf/2109.07751.pdf)</footnote>

<footnote n="35">³⁵ Greco et al. 2022, accepted in Astronomy and Computing : https://doi.org/10.1016/j.ascom.2022.100547</footnote>

<footnote n="36">³⁶ https://zenodo.org/record/7078879#.Y2AQCC3L3kl</footnote>
**Figure 4:** EGO/VIRGO use of IVOA standards and tools for Sky Map visualization of the credible regions for Gravitational Wave events.

## 4.5 Highlight results for radio astronomy services

The ESFRI/RIs in WP4 that are related to radio and millimetre wavelength astronomy are SKA, JIVE, and ALMA, with LOFAR being closely connected to ASTRON. ESCAPE WP4 has facilitated a new level of integration of radio astronomy into the VO. A new IVOA Interest Group<fnref n="37" /> has been created with a CNRS-ObAS member as the vice-chair, and led to the publication of an IVOA note on "Radio astronomy in the VO: services implementation review", published in November 2021, which includes input developed in ESCAPE in particular for ALMA, ASTRON, JIVE as well as a multiple other European radio telescopes (Nançay, NenuFAR, INAF radio telescopes). Based on requirements identified in WP4 a proposal (November 2021) was made to plan an extension to the IVOA Observation Core Data Model (ObsCore) to take into account radio visibility data. The support for the use of VO standards in radio astronomy within the WP4 has led to the implementation of new and improved services for data from JIVE and ASTRON. A "VO service for the European VLBI Network" has been opened by JIVE, and was presented by M. Kettenis at the ESCAPE European Data provider Forum<fnref n="38" /> (M24). The ASTRON ALTA (Apertif Long-Term Archive) has started publishing timing/transient data in the VO and provides a number of services based on the DACHS publishing suite (from the UHEI partner). This work was also presented to the IVOA community in May 2021 (MS 25). The Table Access Protocol (TAP) services from ASTRON have also been used as an example of access to VO data in the WP5 analysis platform prototype. Other work for the use of the IVOA HiPS standard has been demonstrated for LOFAR data enabling it to be visualised in the Aladin client, and initial discussions have been held with SKAO and WP2 partners to explore the use case of a HiPS node in the Data Lake storage system. The ESCAPE work on radio astronomy has also been highlighted to the wider astronomy community at the European Astronomical Society meeting in a presentation on "Radio Astronomy in the Virtual Observatory: Bringing radio data to researchers in the spirit of Open Science", by CNRS-ObAS and ASTRON partners.

One of the examples is the implementation of a JIVE service to access interferometric data sets using VO standards. This service runs on top of the existing archive services. It is based on the DaCHS service software from the UHEI partner, where a custom resource description for JIVE has been implemented. The DaCHS system makes it easy for the service to be registered in the VO registry, and the new EVN Data Archive service is now visible in the all VO tools that are able to consume the IVOA Registry. Figure 5 shows the results of an ADQL query posted to the TAP interface of the JIVE service.

CEVO radio astronomy activities have been presented in June 2021 in the radioastronomy session of the European Astronomical Society meeting. Following a decision of EAS, the content of this session is the heart of a book "Data Intensive Radioastronomy" which will be published in 2023 by Springer. ESCAPE members are in charge of Virtual Observatory dedicated chapters in this book.

<figure><img src="image_31.png" /></figure>

<figure><img src="image_32.png" /></figure>

<footnote n="37">³⁷ IVOA Radio Astronomy IG wiki pages: https://wiki.ivoa.net/twiki/bin/view/IVOA/IvoaRadio</footnote>

<footnote n="38">³⁸ https://indico.in2p3.fr/event/23987/</footnote>
<figure><img src="image_33.png" /><figcaption>Figure 5: The Table Access Protocol (TAP) service hosted at JIVE.</figcaption></figure>

## 4.6 Highlight results for solar physics data services

The emphasis of the activities related to the European Solar Telescope (EST) has been on the application of IVOA standards for solar physics data, and setting up the basis for interoperability between the astronomy Virtual Observatory and the SOLARNET virtual observatory<fnref n="39" />. The highlight results include the installation of an IVOA standard TAP service for the sunspot catalogue of the Uccle Solar Equatorial Table (USET) facility at Royal Observatory of Belgium. This was achieved using the DaCHS VO publishing tool with the help of the UHEI partner. A catalogue of coronal hole detection is in the last step of being finalized as a second TAP service, and will include provenance information as prescribed by the IVOA provenance data model. Furthermore, a TAP client has been implemented in the Space Weather Helioviewer (SWHV) (which is a particular branch of JHelioviewer<fnref n="40" />). Figure 6 shows the USET images imported into JHelioviewer that has been enabled by this work. Interoperability between the TAP service and the JHelioviewer solar visualization tool has been implemented using the IVOA standardized messaging protocol called SAMP.

<footnote n="39">³⁹ http://solarnet.oma.be/</footnote>

<footnote n="40">⁴⁰ https://www.jhelioviewer.org</footnote>
<figure><img src="image_34.png" /><figcaption>**Figure 6:** Data from the Uccle Solar Equatorial Table (USET) facility at Royal Observatory of Belgium interoperably imported into the JHelioviewer solar visualization tool.</figcaption></figure>
# 5. Analysis

The work described above enables us to analyse the IVOA standards in terms of their use for making astronomical data FAIR. As explained in Section 2 and shown in Figure 1, the different facets of FAIR in the astronomy practices can be roughly aligned with the standards defined by the different IVOA Working Groups. The Working Groups define standards which are generally applicable, and others aimed at specific kinds of data. The standards that have been identified as relevant for the the ESFRIs and RIs is given below, classified following the FAIR concepts.

## 5.1 Analysis of IVOA standards and FAIR concepts

### Findability

The Registry of Resources defined by the Working Group with the same name is the core of the VO framework: it lists the available resources and provides a machine-readable link to each of them and essential metadata<fnref n="41" />. As described in D4.7, the Registry is also included in EUDAT B2FIND, providing the Virtual Observatory a first level of visibility in the EOSC. The Registry implements VOResource<fnref n="42" />, an XML encoding Schema for resource metadata. It uses IVOA Identifiers<fnref n="43" /> for globally unique identifiers for the resources within the VO. Integration of the VO Registry in EOSC is identified as a path toward connection of astronomy resources in EOSC, but the high level 'machine readable' capabilities of VO Registries are not as yet included in the EOSC catalogues so at present this integration can only be of a limited nature. The VO Registry can be seen as an example of a domain catalogue, and efforts should continue to be made for demonstrating this to EOSC catalogue developers.

The VO Registry is extensible by design in order to accommodate changes in the types of resources. Extensions of the resource schema are defined for specific kinds of resources, such as the VO standards themselves (StandardsRegExt<fnref n="44" />), collections and services (VOService<fnref n="45" />), or the Table Access Protocol described below (TAPRegExt<fnref n="46" />).

All the data resources provided by the ESFRIs and RIs that are intended for wide use in the astronomy community should be listed in the VO Registry. For future evolutions it will be important that VOResource and the metadata applicability be checked and extensions defined if necessary.

### Accessibility

The Data Access Layer standards provide protocols for remote access to different kinds of data. Most of the protocols are built on the basis of standardised Data Models to ensure interoperability. One of the most important standards is the Table Access Protocol (TAP)<fnref n="47" /> which defines a service protocol for accessing general table data, including astronomical catalogues as well as general database tables.

<figure><img src="image_35.png" /></figure>

<figure><img src="image_36.png" /></figure>

<footnote n="41">⁴¹ http://www.ivoa.net/documents/RM/index.html</footnote>

<footnote n="42">⁴² http://www.ivoa.net/Documents/VOResource/index.html</footnote>

<footnote n="43">⁴³ http://www.ivoa.net/Documents/IVOAIdentifiers/index.html</footnote>

<footnote n="44">⁴⁴ http://www.ivoa.net/documents/StandardsRegExt/</footnote>

<footnote n="45">⁴⁵ http://www.ivoa.net/documents/VOService/index.html</footnote>

<footnote n="46">⁴⁶ http://www.ivoa.net/documents/TAPRegExt/index.html</footnote>

<footnote n="47">⁴⁷ http://www.ivoa.net/Documents/TAP/</footnote>
The TAP standard can be used in a variety of different contexts. It supports multiple query languages, including queries specified using the *Astronomical Data Query Language (ADQL)*^48^. Relational database schema are made interoperable using the "TAP schema" formalism. TAP is designed to be very general and has wide applicability to tabular data services. There is also a related standard adapted to the Solar System community where the Europlanet (EPN) projects have standardized their TAP schema into the so-called *EPN-core*, which is operated as *EPN-TAP*^49^ and which is now also an IVOA standard^50^. The success of the EPN-TAP implementation for Solar Physics data (Section 4.6 above) shows how the TAP-based approach enables interoperability across astronomy-planetary science and solar physics.

Another early standard, *Simple Cone Search*^51^, defines a simple query protocol for retrieving records from a catalog of astronomical sources. The query describes sky position and an angular distance, defining a cone on the sky. Other recommendations address in particular the *Simple Image Access protocol (SIA)*^52^, which enables discovery and access to, and retrieval of datasets including retrieval of 2-D images as well as cubes of three or more dimensions and sparse data such as "event lists". *DataLink*^53^ describes the linking of data discovery metadata to access to the data itself, further detailed metadata, related resources, and to services that perform operations on the data. At a lower level, the *Server-side Operations for Data Access*^54^ (*SODA*) web service capabilities can act upon data files, performing various kinds of operations and applying functions to the data, including cutout and data extraction.

The *Observation Locator Table Access Protocol*^55^ and of the *Object Visibility Simple Access Protocol*^56^ are also of interest for the ESFRIs and RIs. Observation localisation is required for post-alert follow-up observations.

The *Grid & Web Services Working Group* defines the IVOA *Single-Sign-On Profile*^57^ and *Credential Delegation Protocol*^58^, which have to be assessed with respect to the EOSC work on AAI. The *Universal Worker Service Pattern*^59^ UWS defines how to manage asynchronous execution of jobs on a service.

The Data Access standards provide a wide array of techniques that are already implemented in many of the ESFRI/RIs. The maintenance and extension of these is necessary for the continued growth and operation on the VO framework.

<figure><img src="image_37.png" /></figure>

<footnote>^48 http://www.ivoa.net/Documents/ADQL/index.html</footnote>

<footnote>^49 Erard, S., Cecconi, B., Le Sidaner, P., et al., The EPN-TAP protocol for the Planetary Science Virtual Observatory, Astronomy and Computing, 7-8, 52, https://doi.org/10.1016/j.ascom.2014.07.008</footnote>

<footnote>^50 https://ivoa.net/documents/EPNTAP/20220822/index.html</footnote>

<footnote>^51 http://www.ivoa.net/Documents/latest/ConeSearch.html</footnote>

<footnote>^52 http://www.ivoa.net/Documents/SIA/</footnote>

<footnote>^53 http://www.ivoa.net/documents/DataLink/index.html</footnote>

<footnote>^54 http://www.ivoa.net/documents/SODA/index.html</footnote>

<footnote>^55 http://www.ivoa.net/documents/ObsLocTAP/index.html</footnote>

<footnote>^56 http://www.ivoa.net/documents/ObjVisSAP/index.html</footnote>

<footnote>^57 http://www.ivoa.net/Documents/SSO/index.html</footnote>

<footnote>^58 http://www.ivoa.net/Documents/CredentialDelegation/</footnote>

<footnote>^59 http://www.ivoa.net/documents/UWS/index.html</footnote>
## Interoperability

The Application Working Group defines and maintains VOTable<fnref n="60" />, the first standard defined for the Virtual Observatory. VOTable is an XML based standard for the interchange of data defined as a set of tables, and is widely used throughout the VO and is integrated into many other standards. Its elements are updated when required to fulfil additional requirements. The Apps Working Group also maintains the Simple Application Messaging Protocol<fnref n="61" /> SAMP, which is used to interoperate VO-enabled applications via standardised messages for various types of action.

It also maintains the standards that define the Hierarchical Progressive Survey<fnref n="62" /> (HiPS) system. HiPS is a multi-resolution representation of data sets that has many innovative applications for very large image surveys and catalogues. HiPS enables progressive zooming into image surveys where the data at the required resolution can be streamed in real-time for visualisation applications. The related Multi-Order Coverage map method<fnref n="63" /> (MOC) has been standardised to specify the coverage of arbitrary sky regions. The MOC standard has been extended (by ESCAPE authors) to deal with time coverage (STMOC), which will be of high interest for all the ESFRIs and RIs with interests in the Time Domain and Alerts. This is a particularly innovative development that provides a practical indexing method for astronomy data in space and time, which may also be developed for energy/frequency/velocity axes. The implementation of time into the mocpy, the Python library to create and manage MOCs, and its support for the multi-order format provided by LIGO and Virgo is one of the early impacts of this effort, along with the improved MOC visualisation of gravitational wave events in the Virgo-GW skymap<fnref n="64" />. The inclusion of an element to define metadata for time coordinates (TIMESYS) in the last version of VOTable is also relevant for the ESFRI and RI needs. A possible further evolution to fulfil would be to define wavelength/frequency information (FREQSYS).

The Semantics Working Group is in charge of IVOA vocabularies, with a document proposed on practices related to the use of RDF-based vocabularies in the VO, Vocabularies in the VO<fnref n="65" />, a recommendation on Units in the Virtual Observatory<fnref n="66" />, and the IVOA standard to define Unified Content Descriptors<fnref n="67" /> (UCD), a controlled vocabulary for describing astronomical data quantities. The UCD standard, one of the first defined by the IVOA, has been one of the essential assets to ensure interoperability in the VO. The list of terms<fnref n="68" /> evolves following the process defined in the recommendation on Maintenance of the list of UCD words<fnref n="69" />. ESFRI and RI should continue to participate in the definition of vocabularies for their needs.

The Data Modelling Working Group provides models of the different kinds of entities for which a common description is useful to enable interoperability. Two standards are of particular interest in ESCAPE context, one being used at the data centre level, the other one at the dataset level. The

<figure><img src="image_38.png" /></figure>

<figure><img src="image_39.png" /></figure>

<footnote n="60">⁶⁰ http://www.ivoa.net/Documents/VOTable/</footnote>

<footnote n="61">⁶¹ http://www.ivoa.net/documents/SAMP/index.html</footnote>

<footnote n="62">⁶² http://www.ivoa.net/documents/HiPS/index.html</footnote>

<footnote n="63">⁶³ http://www.ivoa.net/documents/MOC/index.html</footnote>

<footnote n="64">⁶⁴ http://www.virgo-gw.eu/skymap.html</footnote>

<footnote n="65">⁶⁵ http://www.ivoa.net/documents/Vocabularies/20190905/</footnote>

<footnote n="66">⁶⁶ http://www.ivoa.net/documents/VOUnits/index.html</footnote>

<footnote n="67">⁶⁷ http://www.ivoa.net/Documents/latest/UCD.html</footnote>

<footnote n="68">⁶⁸ http://www.ivoa.net/Documents/UCD1+/index.html</footnote>

<footnote n="69">⁶⁹ http://www.ivoa.net/Documents/UCDlistMaintenance/index.html</footnote>
ObsCore<fnref n="70" /> recommendation defines the core components of an observation data model that are necessary to perform data discovery when querying data centers for astronomical observations of interest, and how they are implemented in the Table Access Protocol (ObsTAP). EPN-core (see above) is an extension of the ObsCore table for planetary data that has also been now used for Solar Physics. In the IVOA Radio Astronomy Interest Group, ESCAPE partners have worked together with international observatories and interferometric networks to define an extension of the ObsCore data model to support this type of data is necessary and what parameters would be needed to accomodate this. The Characterisation<fnref n="71" /> data model defines the high level metadata necessary to describe the physical parameter space of observed or simulated astronomical data sets which can be used to derive a structured description of any relevant data and thus to facilitate its discovery and scientific interpretation. These two models have to be assessed to ensure that they will be able to fulfil the requirements of the ESFRIs and RIs.

The Provenance data model, which is also under the remit of the Data Modeling WG, will be discussed
in the Reusability section.

Reusability

For reusability, all the IVOA standards ("community standards", FAIR principle R1.3) are relevant, in
particular those which define metadata (schemata and/or artefacts).

Data formats are also essential components for reusability and interoperability. The widespread
usage of the FITS format is a key asset in the domain for astronomy. Other formats, such as HDF5<sup>72</sup>,
are used by some of the ESFRIs. It is important to assess how to ensure reusability (and
interoperability) when these formats are used. The radio-astronomy community has been using FITS
files in various flavors for storing and distributing visibility data but is moving away from this. In
recent years, the MeasurementSet<sup>73</sup> has established itself as a new standard that is used by many
new instruments and supported by a wide variety of data processing software. CTA discusses the
usage of HDF5 for low level data and FITS for higher level.

Provenance is specifically cited among the principles (FAIR principle R1.2). Many IVOA standards, as
well as the FITS data format, include provenance information. The IVOA Provenance Data Model<sup>74</sup>
describes how provenance information can be modeled, stored and exchanged within the
astronomical community in a standardized way, following the definition of provenance as proposed
by the W3C. The model was developed under the leadership of the Paris Observatory CTA team in
ASTERICS and ESCAPE. A TAP implementation of the Provenance standard, ProvTAP, has been
proposed. Provenance is a domain in which CEVO is enabling synergies between CTA and KM3NeT.

FAIR principle R1.1, which requires that the (metaicie) data is released with a clear and accessible data
usage licence, can be an issue for many astronomical data resources: astronomical data is in general
openly available, often after a proprietary period (in general 1 year) during which the metadata are
in general public. Data and metadata often do not have an explicit usage licence, and usage rights
rely mostly on disciplinary ethics: cite the origin of data when data is cited. The IVOA Data Curation

<figure><img src="image_40.png" /></figure>

<figure><img src="image_41.png" /></figure>

<footnote n="70">70 http://www.ivoa.net/documents/ObsCore/index.html</footnote>

<footnote n="71">⁷¹ http://www.ivoa.net/Documents/latest/CharacterisationDM.html</footnote>

<footnote n="72">⁷² https://support.hdfgroup.org/HDF5/whatishdf5.html</footnote>

<footnote n="73">⁷³ https://casa.nrao.edu/casadocs/casa-5.1.0/reference-material/measurement-set</footnote>

<footnote n="74">⁷⁴ http://www.ivoa.net/documents/ProvenanceDM/index.html</footnote>
and Preservation Interest Group is an appropriate venue to discuss usage licences for astronomical data.

Also relevant to Reusability is the VOEvent<fnref n="75" /> standard, which specifies how to represent, transmit, publish, and archive information about transient celestial events. In fact, a timely follow-up is crucial for the RIs dealing with alerts, including those involved in CEVO. VOEvent is already used e.g. for gravitational wave alerts and by the neutrino experiments which are CTA pathfinders.

## 5.2. Best Practices for Data Stewardship

The ESCAPE project brought partners from the Astronomy and Particle Physics ESFRI infrastructures together to address their common challenges, including the support Open Science via the use of the FAIR principles. From the ESCAPE experience we identify a number of best practices that relate to data stewardship. These have been explored for in a paper (Allen & Amodeo et al. 2021⁷⁶) that was presented by ESCAPE partners at the "Library and Information Services in Astronomy IX" conference in 2021⁷⁷. We summarize the main findings below and extend them to include connections between training activities and data stewardship.

One of the concepts that has been reinforced by the work done in WP4 is that it is essential to have disciplinary level standards, and that these standards need to include or allow interfaces with wider interdisciplinary standards and frameworks. Disciplinary level standards are necessary because there are specific types of data and information with specialized characteristics within a given domain, that must be managed by experts in order to maintain scientific integrity of the data and services. In astrophysics, this includes the disciplinary level metadata for semantic content (IVOA standard UCDs), and also the systems for sky coordinates and indexing (e.g. World Coordinate Systems, and the IVOA HiPS/MOC standards), as well as for characterising astronomical observations (such as the IVOA Observational Core Data Model) to name just a few examples which have been relevant in this work. In practice, we find that it is critical for the parties that will implement the standards to be involved in their definition and maintenance, which has implications for the data stewardship roles that are needed to support open science in astronomy infrastructures.

With the creation of Open Science frameworks such as EOSC in Europe, and in particular the inclusion of existing infrastructures into these frameworks, the astrophysics standards and data services can be seen as part of the systems that have a wider scope of data sharing across all areas of science. This requires that the astronomy standards be compatible at some level with more generic standards. This is already the case with the IVOA Resource Registry which uses the Dublin Core metadata and OAI-PMH (Open Archives Initiative Protocol for Metadata Harvesting), and enables the harvesting of records by EUDAT/B2FIND in readiness for inclusion in EOSC catalogues. Also, the IVOA standards definitions themselves which are included in the FAIRSharing database<fnref n="78" /> of standards. Detailed interdisciplinary science will demand higher levels of cross-discipline data compatibility, but it can

<figure><img src="image_42.png" /></figure>

<figure><img src="image_43.png" /></figure>

<footnote n="75">⁷⁵ http://www.ivoa.net/documents/VOEvent/index.html</footnote>

<footnote n="76">⁷⁶ https://hal.archives-ouvertes.fr/hal-03372190v1</footnote>

<footnote n="77">⁷⁷ https://lisa9.org</footnote>

<footnote n="78">⁷⁸ https://fairsharing.org/standards/?q=IVOA</footnote>
**already be considered as good practice to establish the relationship of disciplinary standards to those in the wider scientific landscape.**

In terms of data stewardship, it requires that scientists, software engineers, data publication experts and librarians be aware of the wider context of their work. For example, in these various roles, it will be important to use interoperable metadata and to benefit from systems/tools/services that can be made common across domains. This is challenging because it requires a level of participation in activities that extend beyond the traditional boundaries of a discipline. The EOSC initiative is promoting widespread uptake of Open Science, and projects such as FAIRsFAIR<sup>79</sup> are fostering FAIR data practices, so there is a rapidly growing set of training resources for training which can be of great value. **In ESCAPE WP4 we have found that it is good practice to make links between the training activities and data stewardship.**

The main training activities conducted by WP4 included two “Science with interoperable data” schools (D4.3 And D4.6) which were aimed at early career researchers, and the “European Data Provider Forum – Hands-on workshop” (MS 25) aimed at data providing projects or infrastructures. While the schools focused on enabling participants to use the interoperable systems for their own research, we found that introducing them to how the systems work, in particular how standards have been implemented to make the data interoperable, was also very beneficial. As such the participants were informed how, as authors, they can facilitate the stewardship of their published data – by correctly attaching the appropriate metadata and by using the standards (for example for specification of physical units, or providing data in standardised formats), and **by following the recently published astronomy community guidelines for “Best Practices for Data Publication in the Astronomical Literature”<sup>80</sup>**.

The training event for data providing projects or infrastructures also enabled us to identify that best practices can be very effectively promoted by the **use of leading examples from operational infrastructures**. Sharing the experience of adopting standards, and explaining the various challenges, enables others to benefit from the many years of experience that have been built up by infrastructures that have already gone through this process. The training event itself focused on exchange of solutions and best practices. One of the best examples was the detailed presentation by the ESO partner on the experience of implementing IVOA standards in the ESO Science Archive, and the many lessons learned. This presentation has been adapted and re-used to reach a wider audience in a dedicated ESCAPE Webinar<sup>81</sup> in June 2022.

Data stewardship is a topic that is addressed in a variety of ways in research infrastructures. In the context of ESCAPE we have investigated the ways in which value can be added to the content of telescope archives by providing additional capabilities beyond the usual search functions. In Task 4.3 we explored the application of techniques from the field of Machine Learning to astronomical archive

<figure><img src="image_44.png" /></figure>

<figure><img src="image_45.png" /></figure>

<footnote n="79"><sup>79</sup> https://www.fairsfair.eu</footnote>

<footnote n="80"><sup>80</sup> https://ui.adsabs.harvard.edu/abs/2022ApJS..260...5C/abstract</footnote>

<footnote n="81"><sup>81</sup> https://projectescape.eu/events/escape-vo-enabling-international-data-interoperability-one-single-virtual-astronomy-facility</footnote>
search capabilities. The results have been published the D4.5⁸² report where we demonstrate the viability of enabling archival searches based on data similarity, in which users upload a data item they are interested in, and the system returns similar assets. In terms of data stewardship, this activity showed that additional expertise can be brought into a data archive environment to bring new levels of functionality to existing systems. While this activity was an exploratory exercise, we can recognise that the role of data stewards in the future may widen to include different specialisations such as artificial intelligence and other technologies, and we expect that **the boundaries between data stewards and data scientists will evolve to have a lot of overlap as new capabilities for open science are realised.**

## 6. Conclusion

The achievements in the ESCAPE project for the task on 'Implementation of the FAIR principles for ESFRI data through the Virtual Observatory' show that the IVOA standards are an effective way to make astronomical data findable, accessible, interoperable and reusable. The results presented here include improvements to interoperability for all of the ESFRI/RIs involved, and serve as examples of how disciplinary standards can be put into practice in services and tools to enable Open Science.

ESCAPE has provided essential input to the IVOA ensuring the European interests have been taken into account in the international standardisation for the scientific domain of astronomy. We have enabled the ESFRI and RI requirements to help drive the evolutions of the IVOA data sharing framework.

The immediate activities that follow from the success of the ESCAPE project include the continued implementation of the standards and tools. The work on spatial and temporal indexing of data has applicability across all of the astronomy infrastructures, and the goal is to foster innovative use of this system for multi-messenger and time-domain astronomy. Implementation of the visualization tools built in ESCAPE are expected in various archives and data services, for example the ESO and ESA archives, and a wide range of data access interfaces and science platforms. In terms of standards, one of the clear next steps is to define the standards necessary for science platforms and their implementation is various kinds of virtual research environments.

Coordination of the European representation in the IVOA for the development of the VO is crucial for its sustainability. ESCAPE has extended the network of cooperating infrastructures who use and contribute to the VO, and the achievements described above will be a long-lasting legacy of ESCAPE provided that the framework can be well maintained. As such an essential future goal is to maintain the coordination of VO activities in Europe to meet the scientific and technical needs.

<figure><img src="image_46.png" /></figure>

<figure><img src="image_47.png" /></figure>

<footnote n="82">⁸² https://projectescape.eu/deliverables-and-reports/delivrable-45-release-prototype-machine-learning-enabled-archive-services</footnote>
# Appendix 1 – Dedicated meetings for VO framework developments for ESFRI and RIs.

Meetings and conferences for specific topics related to ESFRI/RI requirements and priorities:

<table><thead><tr><td>Date</td><td>Title</td><td>Type</td><td>Location</td></tr></thead><tbody><tr><td>26 February 2019</td><td>EST and the VO</td><td>Meeting (ASTERICS-ESCAPE cross-over)</td><td>Strasbourg</td></tr><tr><td>28 February 2019</td><td>Radio Astronomy and the VO</td><td>Meeting (ASTERICS-ESCAPE cross-over)</td><td>Strasbourg</td></tr><tr><td>11-12 March 2019</td><td>ROB/CDS meeting (EST)</td><td>Visit</td><td>Brussels</td></tr><tr><td>05 July 2019</td><td>KM3Net and the VO</td><td>Meeting</td><td>Strasbourg</td></tr><tr><td>25-26 July 2019</td><td>ECAP/UHEI meeting (KM3Net)</td><td>Visit</td><td>Erlangen</td></tr><tr><td>06 November 2019</td><td>CTA and KM3NeT Provenance meeting</td><td>Meeting</td><td>Strasbourg</td></tr><tr><td>12 November 2019</td><td>KIS/UHEI meeting (EST)</td><td>Visit</td><td>Freiburg</td></tr><tr><td>06-10 January 2020</td><td>ASTRON/UHEI Meeting (JIVE/radioastronomy)</td><td>Visit</td><td>Dwingeloo</td></tr><tr><td>28 January – 14 February 2020</td><td>EGO-Virgo partner visit to CDS</td><td>Visit</td><td>Strasbourg</td></tr><tr><td>02 July 2020</td><td>Radioastronomy Interest Group first virtual meeting</td><td>Virtual meeting</td><td>Web</td></tr><tr><td>26 August 2020</td><td>Radio Astronomy visibility data discovery strategies I (CDS/JIVE/ASTRON)</td><td>WP4 Event</td><td>Web</td></tr><tr><td>3 September 2020</td><td>Radio Astronomy visibility data discovery strategies I (CDS/NENUFAR-Nançay)</td><td>WP4 Event</td><td>Web</td></tr><tr><td>30 September 2020</td><td>Radio Astronomy visibility data discovery strategies II (CDS/JIVE/ASTRON/Nançay)</td><td>WP4 Event</td><td>Web</td></tr></tbody></table>
<table><tr><td>14 January 2021</td><td>CEVO Radio Astronomy Pulsar data in the VO</td><td>WP4 Event</td><td>Web</td></tr><tr><td>25 June 2021</td><td>IVOA metadata and High Energy Astrophysics</td><td>Preliminary working meeting</td><td>Web</td></tr><tr><td>9 July 2021</td><td>SKAO CEVO provenance meeting</td><td>Working meeting</td><td>Web</td></tr><tr><td>1 September 2021</td><td>CEVO (CTAO + High energy) datamodels and standards for interoperability</td><td>Kickoff meeting</td><td>Web</td></tr><tr><td>13 September 2021</td><td>HiPS in the Data Lake</td><td>ESCAPE event</td><td>Web</td></tr><tr><td>08 December 2021 Links 1, 2, 3,</td><td>CTA/KM3Net CEVO "datamodels for interoperability" workshop</td><td>Workshop</td><td>Web</td></tr></table>

<figure><img src="image_48.png" /></figure>

<figure><img src="image_49.png" /></figure>
## Appendix 2 – IVOA Interoperability Meeting – Milestones.

### IVOA Meeting Milestones:

<table><thead><tr><td>Title</td><td>Milestone Date</td><td>Link / ESCAPE contributions</td></tr></thead><tbody><tr><td>M4.1 / MS20 Progress and Priorities at IVOA</td><td>June 2019</td><td>Link / 30 contributions</td></tr><tr><td>M4.2 / MS21 Progress and Priorities at IVOA</td><td>November 2019</td><td>Link / 22 contributions</td></tr><tr><td>M4.3 / MS22 Progress and Priorities at IVOA</td><td>June 2020</td><td>Link / 22 contributions</td></tr><tr><td>M4.4 / MS23 Progress and Priorities at IVOA</td><td>December 2020</td><td>Link / 15 contributions</td></tr><tr><td>M4.6 / MS25 Progress and Priorities at IVOA</td><td>June 2021</td><td>Link / 29 contributions</td></tr><tr><td>M4.7 / MS26 Progress and Priorities at IVOA</td><td>November 2021</td><td>14 contributions</td></tr></tbody></table>

<figure><img src="image_50.png" /></figure>

<figure><img src="image_51.png" /></figure>