# Pipelines for interoperability of distributed resources in ecology: the AnaEE-France context

Christian Pichot, Philippe Clastre, Benjamin Jaillet, Damien Maurice, Rachid Yahiaoui

► To cite this version:

Christian Pichot, Philippe Clastre, Benjamin Jaillet, Damien Maurice, Rachid Yahiaoui. Pipelines for interoperability of distributed resources in ecology: the AnaEE-France context. 5ème édition des Journées Nationales du Développement Logiciel (JDEV2020), Jul 2020, Rennes, France. hal-03028515

HAL Id: hal-03028515

https://hal.science/hal-03028515

Submitted on 15 Dec 2020

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
<figure><img src="image_2.png" /></figure>

AnaEE

Analysis and Experimentation on Ecosystems

France

<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /></figure>

# Pipelines for interoperability of distributed resources in ecology: the AnaEE-France context

PICHOT C. (1), CLASTRE P. (1), JAILLET B. (1), MAURICE D. (2), YAHIAOUI R. (3)

1. INRAE URFM 228 route de l'Aérodrome 84914 Avignon

2. INRAE UMR SILVA route d'Amance 54280 Champenoux.

3. INRAE US INFOSOL 2163 avenue de la Pomme de Pin 45075 Orléans

<figure><img src="image_5.png" /></figure>

<figure><img src="image_6.png" /></figure>

D2KAB

<figure><img src="image_7.png" /></figure>

LICENCE OUVERTE
OPEN LICENCE

## Context and objective

The AnaEE-France (Analysis and Experimentation on Ecosystems) Research Infrastructure offers experimental facilities for studying ecosystems and biodiversity. The data generated by the AnaEE platforms are most often managed in relational database.

A distributed Information System (IS) is developed, based on semantic interoperability of its components and using common vocabularies (AnaeeThes thesaurus and OBOE-based ontology). Discovery and access portals are fed by information (rdf triples) produced by the semantic annotation of the resources that also generate metadata and datasets.

Two pipelines are developed for facilitating the semantic annotation and exploitation processes which may represent a huge conceptual and practical work.

## Experimental, analytical and modelling platforms

<figure><img src="image_8.png" /></figure>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /></figure>

<figure><img src="image_12.png" /></figure>

<figure><img src="image_13.png" /></figure>

<figure><img src="image_14.png" /></figure>

<figure><img src="image_15.png" /></figure>

Relational databases

<figure><img src="image_16.png" /></figure>

Semantic

<figure><img src="image_17.png" /></figure>

Graph database

<figure><img src="image_18.png" /></figure>

Semantic

<figure><img src="image_19.png" /></figure>

## Semantic data generation from a pipeline for database annotation

<figure><img src="image_20.png" /></figure>

Variable semantic description

<table><thead><tr><th>AnaEE standard</th><th>Category</th><th>Context</th><th>Entity</th><th>Characteristic</th><th>Unit</th></tr></thead><tbody><tr><td>Phytoplankton</td><td>Biodiversity</td><td>Water</td><td>Phytoplankton</td><td>Volume Per Volume</td><td>MicroMeterCubed Per Millimeter</td></tr><tr><td>WaterPH</td><td>Physical Chemistry</td><td>Water Column</td><td>Water</td><td>pH</td><td>pHUnit</td></tr></tbody></table>

<figure><img src="image_21.png" /></figure>

← A **csv input file** describes the main characteristics of the variables submitted for semantic annotation, allowing to parametrize generic semantic graphs (Yed* format).

The **annotation pipeline** can be used in different contexts of ontologies and databases.

It includes shell scripts and specific java developments and requires the Yed, Ontop, Corese and Blazagraph softwares. It is deployed as a classical scripted application or as a dockerised one.

The annotation pipeline consists in 4 steps:

1. generation of the mapping files to be used by Ontop** (database connexion parameters, sets of sql queries and of the corresponding triples)

2. production of the initial triples (ttl files) by a specific program and Ontop tool.

3. production of inferred triples by Corese*** from the ontology rules

4. uploading of the ttl files within the Blazagraph**** software and initialisation of the end point

Current developments will allow data annotation from several relational database software (PostgreSQL, MySQL, ...) and csv flat files.

<figure><img src="image_22.png" /></figure>

<figure><img src="image_23.png" /></figure>

<figure><img src="image_24.png" /></figure>

End Point

<figure><img src="image_25.png" /></figure>

Step4_pipeline
processing

<figure><img src="image_26.png" /></figure>

raw data with
inferred triples

<figure><img src="image_27.png" /></figure>

Step3_pipeline
processing

<figure><img src="image_28.png" /></figure>

raw data

<figure><img src="image_29.png" /></figure>

Specific
component

<figure><img src="image_30.png" /></figure>

## Semantic data exploitation by a pipeline for metadata and datasets generation

Outputs of the annotation pipeline are used by an **exploitation pipeline** for:

1. generation (through a SPARQL query on raw data) of synthetic data that feeds the discovery portal,

2. generation of standardised GeoDCAT and ISO19115/19139 metadata records,

3. generation of data file (NetCDF as first format) from selected perimeters (e.g years, experimental sites, variable categories..). In that case, the annotation pipeline is launched using a dedicated webservice.

Metadata and data products are transferred to a Dataverse repository

<figure><img src="image_31.png" /></figure>

End Point

<figure><img src="image_32.png" /></figure>

perimeter
delimitation

<figure><img src="image_33.png" /></figure>

OBOE
metadata

<figure><img src="image_34.png" /></figure>

conversion

<figure><img src="image_35.png" /></figure>

GeoDCAT
metadata

<figure><img src="image_36.png" /></figure>

API
(XSLT)

<figure><img src="image_37.png" /></figure>

19139

<figure><img src="image_38.png" /></figure>

<figure><img src="image_39.png" /></figure>

<figure><img src="image_40.png" /></figure>

<footnote>*https://www.yworks.com/products/yed</footnote>

<footnote>**Diego Calvanese, Benjamin Cogrel, Sarah Komla-Ebri, Roman Kontchakov, Davide Lanti, Martin Rezk, Mariano Rodriguez-Muro, and Guohui Xiao. Ontop: Answering SPARQL Queries over Relational Databases. In: Semantic Web Journal 8.3 (2017), pp. 471-487.</footnote>

<footnote>***http://wimmics.inria.fr/corese</footnote>

<footnote>****https://wiki.blazagraph.com/wiki/index.php/Main_Page</footnote>