# SO FAIR - A generic tool for publishing observation data within an ecosystem of interoperable services

Hervé Squividant, Mario Adam, Mikael Faucheux, Ophélie Fovet, Christophe

Geneste, Tom Loree, Thierry Morvan, Laurent Ruiz, Zahra Thomas

► To cite this version:

Hervé Squividant, Mario Adam, Mikael Faucheux, Ophélie Fovet, Christophe Geneste, et al.. SO FAIR - A generic tool for publishing observation data within an ecosystem of interoperable services. TERENO-OZCAR conference, TERENO and OZCAR research networks, Sep 2023, Bonn, Germany. hal-04236491

HAL Id: hal-04236491

https://hal.inrae.fr/hal-04236491

Submitted on 10 Oct 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# SO FAIR - A generic tool for publishing observation data within an ecosystem of interoperable services

<figure><img src="image_2.png" /></figure>

2nd TERENO-OZCAR Conference 2023

28 September 2023

Herve Squividant, Mario Adam, Mikael Faucheux, Ophelie Fovet, Christophe Geneste, Tom Loree, Thierry Morvan, Laurent Ruiz & Zahra Thomas.
Context

Maturity of FAIrisation of geographic data

• 2000: OGC W*S standards

• 2007: INSPIRE directive

• 2009 : SDI

• 2010 → ... : Platforms + communities + ...

• ... birth of the spatial open data ecosystem

Late start for the "liberation" of temporal data

• 2006: Standard SOS "orphaned by its WMS

• 2015: New OGC SensorThings standard

• 2016: FROST Server

• 2021 : SensorThings as input of Theia/OZCAR IS

Complexity of configuring a temporal open data service

Compartmentalization between FAIR services for "geographic" AND "temporal" data
> Question

How to **publish** temporal datas **simply** and **integrate** them into an ecosystem of pre-existing services ?

<figure><img src="image_3.png" /></figure>
> Our answer

By initiating an internal working group with a mix of different
skill profiles : metrologist, scientist, FAIR data scientist, API
developer, service deployer / infrastructure manager, ...

<figure><img src="image_4.png" /></figure>

To work together to define a methodological, technical and
human chain for routing data from the sensor to the
ecosystem of FAIR data services based on subsidiarity as we
made before with INSPIRE SDI and geographical data sharing.

<figure><img src="image_5.png" /></figure>

The project SO FAIR -> Sensor Observations to FAIR data

S<sub>ensor</sub>

O<sub>bservations</sub>

<figure><img src="image_6.png" /></figure>

<figure><img src="image_7.png" /></figure>

<figure><img src="image_8.png" /></figure>

F<sub>indable</sub>

A<sub>ccessible</sub>

I<sub>nteroperable</sub>

R<sub>eusable</sub>

<figure><img src="image_9.png" /></figure>

<figure><img src="image_10.png" /></figure>

<figure><img src="image_11.png" /></figure>

<figure><img src="image_12.png" /></figure>

<figure><img src="image_13.png" /></figure>

SAS

INRAO

l'institut Agro
agriculture • alimentation • environnement

AGRO
CAMPUS
Nantes
<figure><img src="image_14.png" /></figure>

# SO FAIR middleware service

<figure><img src="image_15.png" /></figure>

FROST
Server

STEAN

SensorThings Enhanced API Node

<figure><img src="image_16.png" /></figure>

<figure><img src="image_17.png" /></figure>

<figure><img src="image_18.png" /></figure>

<figure><img src="image_19.png" /></figure>

<figure><img src="image_20.png" /></figure>

<figure><img src="image_21.png" /></figure>

<figure><img src="image_22.png" /></figure>

mviewer

<figure><img src="image_23.png" /></figure>

<figure><img src="image_24.png" /></figure>

INRAO

l'institut Agro

agriculture • alimentation • environnement

<figure><img src="image_25.png" /></figure>
<figure><img src="image_26.png" /></figure>

# The SO FAIR project

1

Creating a SensorThings service
→ api.geosas.fr/<MyInstance>/v1.0

<figure><img src="image_27.png" /></figure>

## Configuring a SensorThings service

- Things

2

- Sensors

- ObservedProperties

- Thesaurii

- ...

<figure><img src="image_28.png" /></figure>

## Feeding datas to service

3

- Automatic : Lora, ...

- Semi-automatic : Outputs from proprietary software

- Manual : upload CSV

- ...

4

## Connection to the service ecosystem

- Automated generation of INSPIRE metadata and OGC layers in SDI

- Dataverse Metadata

<figure><img src="image_29.png" /></figure>

Automated generation of web-based tools for viewing and downloading time series, with or without spatial inputs.

5

<figure><img src="image_30.png" /></figure>

<figure><img src="image_31.png" /></figure>
1

Creating a SensorThings service

-> api.geosas.fr/<MyInstance>/v1.0

<figure><img src="image_32.png" /></figure>

Configuring a SensorThings service

- Things

2

Sensors

ObservedProperties

Thesaurii

- ...

<figure><img src="image_33.png" /></figure>

Feeding datas to service

3

- Automatic : Lora, ...

- Semi-automatic : Outputs from proprietary software

- Manual : upload CSV

- ...

4

Connection to the service ecosystem

- Automated generation of INSPIRE metadata and OGC layers in SDI

- Dataverse Metadata

<figure><img src="image_34.png" /></figure>

Automated generation of web-based tools for viewing and downloading time series, with or without spatial inputs.

5

<figure><img src="image_35.png" /></figure>

<figure><img src="image_36.png" /></figure>

INRAO

l'institut Agro

agriculture • alimentation • environnement

AGRO
CAMPUS
NANT
## SO FAIR API Service

### New standard

<figure><img src="image_37.png" /></figure>

### A python implementation
of the OGC API

<figure><img src="image_38.png" /></figure>

pygeoapi

### deployed in a beta version

https://geosas.fr/sofair-dev/api/

<figure><img src="image_39.png" /></figure>

1

#### Processes in this service

<table><thead><tr><td>Name</td><td>Description</td></tr></thead><tbody><tr><td>SensorThings API creation</td><td>This function allows you to create a new instance of a SensorThings service that is immediately accessible online.</td></tr><tr><td>Setting up the sensorthings service</td><td>This function uploads an XLSX file to create the various objects in the SensorThings service: Things, Sensors, Observed Properties, Locations, Features Of Interest and Datastreams.</td></tr><tr><td>Upload observations</td><td>This function uploads observations from CSV or XLSX file to the SensorThings service.</td></tr><tr><td>Publishes a SensorThings service in an IDG</td><td>This process publishes INPIRE-compliant layer and metadata from a SensorThings service to a Spatial Data Infrastructure.</td></tr><tr><td>Create SensorThings geographic portal</td><td>This function automatically generates a URL pointing to a map portal displaying chronicles of observations from the SensorThings service provided.</td></tr></tbody></table>

<figure><img src="image_40.png" /></figure>

INRAO

l'institut Agro

agriculture • alimentation • environnement

AGRO
CAMPUS
NANT
> SO FAIR API Service

<figure><img src="image_41.png" /></figure>

<figure><img src="image_42.png" /></figure>
# SO FAIR API Service

## Un nouveau standard

<figure><img src="image_43.png" /></figure>

## Une implémentation de l'API OGC en python

<figure><img src="image_44.png" /></figure>

pygeoapi

## Déployée en mode béta

https://geosas.fr/sofair-dev/api/

<figure><img src="image_45.png" /><figcaption>### Processes in this service</figcaption></figure>

<table><thead><tr><td>Name</td><td>Description</td></tr></thead><tbody><tr><td>SensorThings API creation</td><td>This function allows you to create a new instance of a SensorThings service that is immediately accessible online.</td></tr><tr><td>Setting up the sensorthings service</td><td>This function uploads an XLSX file to create the various objects in the SensorThings service: Things, Sensors, Observed Properties, Locations, Features Of Interest and Datastreams.</td></tr><tr><td>Upload observations</td><td>This function uploads observations from CSV or XLSX file to the SensorThings service.</td></tr><tr><td>Publishes a SensorThings service in an IDG</td><td>This process publishes INPIRE-compliant layer and metadata from a SensorThings service to a Spatial Data Infrastructure.</td></tr><tr><td>Create SensorThings geographic portal</td><td>This function automatically generates a URL pointing to a map portal displaying chronicles of observations from the SensorThings service provided.</td></tr></tbody></table>

2

<figure><img src="image_46.png" /></figure>

<figure><img src="image_47.png" /></figure>

INRAO

l'institut Agro

agriculture • alimentation • environnement

AGRO
CAMPUS
CAMPUS

2nd TERENO-OZCAR Conference

Bonn - 28 september 2023
# From environmental observatory to SensorThings Model

<figure><img src="image_48.png" /></figure>

Reality in the field

How ?

Information system

SensorThings
model

<figure><img src="image_49.png" /></figure>

<figure><img src="image_50.png" /></figure>

<figure><img src="image_51.png" /></figure>
<figure><img src="image_52.png" /></figure>

# Iterative teamwork

<figure><img src="image_53.png" /></figure>

## Scientists

* Need to understand the model,

* Involvement in the choice of concepts and thesaurus definitions.

## Metrologist, sensor installer

* Need to understand the model,

* Knowledge of observatory site, sensors, observed properties and data path.

<figure><img src="image_54.png" /></figure>

<figure><img src="image_55.png" /></figure>

## Sensorthings model

<figure><img src="image_56.png" /></figure>

## FAIR data scientist

* Good knowledge of the model, services & tools,

* Facilitating role to extract knowledge from the scientist and the metrologist.

<figure><img src="image_57.png" /></figure>

<figure><img src="image_58.png" /></figure>

SAS

INRAO

l'institut Agro
agriculture • alimentation • environnement

AGRO
CAMPUS
NANT
# SO FAIR API Service

## New standard

OGC API-Processes

(Open Geospatial Consortium)

<figure><img src="image_59.png" /></figure>

<figure><img src="image_60.png" /></figure>

## A python implementation of the OGC API

<figure><img src="image_61.png" /></figure>

pygeoapi

## deployed in a beta version

https://geosas.fr/sofair-dev/api/

<figure><img src="image_62.png" /></figure>

### Processes in this service

<table><thead><tr><td>Name</td><td>Description</td></tr></thead><tbody><tr><td>SensorThings API creation</td><td>This function allows you to create a new instance of a SensorThings service that is immediately accessible online.</td></tr><tr><td>Setting up the sensorthings service</td><td>This function uploads an XLSX file to create the various objects in the SensorThings service: Things, Sensors, Observed Properties, Locations, Features Of Interest and Datastreams.</td></tr><tr><td>Upload observations</td><td>This function uploads observations from CSV or XLSX file to the SensorThings service.</td></tr><tr><td>Publishes a SensorThings service in an IDG</td><td>This process publishes INPIRE-compliant layer and metadata from a SensorThings service to a Spatial Data Infrastructure.</td></tr><tr><td>Create SensorThings geographic portal</td><td>This function automatically generates a URL pointing to a map portal displaying chronicles of observations from the SensorThings service provided.</td></tr></tbody></table>

3

<figure><img src="image_63.png" /></figure>

<figure><img src="image_64.png" /></figure>

INRAO

l'institut Agro
agriculture • alimentation • environnement

<figure><img src="image_65.png" /></figure>

2nd TERENO-OZCAR Conference

Bonn - 28 september 2023
<figure><img src="image_66.png" /><figcaption># Upload observations in SensorThings API</figcaption></figure>

<figure><img src="image_67.png" /></figure>
## SO FAIR API service

### New standard

<figure><img src="image_68.png" /></figure>

### A python implementation
of the OGC API

<figure><img src="image_69.png" /></figure>

pygeoapi

### deployed in a beta version

https://geosas.fr/sofair-dev/api/

<figure><img src="image_70.png" /></figure>

<figure><img src="image_71.png" /></figure>

SAS

INRAO

l'institut Agro

agriculture • alimentation • environnement

AGRO
CAMPUS
NANT
<figure><img src="image_72.png" /></figure>

# Creating Métadata and layers in SDI

<figure><img src="image_73.png" /></figure>

* View/download/request by OGC W*S

* Solves the problem of large number of Things or Features

* Increase SensorThings API visibility in the INSPIRE platform ecosystem via multiple harvesting mechanisms

* Openness to other standard, other catalog?
## SO FAIR API service

### New standard

OGC API-Processes

(Open Geospatial Consortium)

<figure><img src="image_74.png" /></figure>

<figure><img src="image_75.png" /></figure>

### A python implementation
of the OGC API

<figure><img src="image_76.png" /></figure>

pygeoapi

### deployed in a beta version

https://geosas.fr/sofair-dev/api/

5

<figure><img src="image_77.png" /></figure>

<figure><img src="image_78.png" /></figure>

INRAO

l'institut Agro

agriculture • alimentation • environnement

AGRO
CAMPUS
NORTH
# Dynamic creation of a cartographic web portal

* Automatic URL generation

e.g. : https://geosas.fr/mviewer-dev/?config=/apps/efele/config2.xml

* co-visualization of spatial data / temporal data

<figure><img src="image_79.png" /></figure>
<figure><img src="image_80.png" /></figure>
> Prospects

SO FAIR, missing link Spatial and temporal data Infrastructures - S&TDI ?

→ able to publish spatio-temporal data in a variety of complementary representations :

○ using time series of georeferenced raster or vector layers

<figure><img src="image_81.png" /></figure>

<figure><img src="image_82.png" /></figure>

<figure><img src="image_83.png" /></figure>

○ Using time series associated with each geographical object

<figure><img src="image_84.png" /></figure>

Parcelle 1

<figure><img src="image_85.png" /></figure>

Parcelle 2

...

<figure><img src="image_86.png" /></figure>

Parcelle n
> Prospects : BOSCO project example

► Time series of raster images representing soil surface moisture at 10 m resolution

<figure><img src="image_87.png" /></figure>

<figure><img src="image_88.png" /></figure>
<figure><img src="image_89.png" /></figure>

# Prospects : BOSCO project example

<figure><img src="image_90.png" /></figure>

Time series of raster images representing soil surface moisture at 10 m resolution

<figure><img src="image_91.png" /></figure>

<figure><img src="image_92.png" /></figure>
<figure><img src="image_93.png" /></figure>

# Prospects : BOSCO project example

<figure><img src="image_94.png" /></figure>

## Time series representing soil surface moisture on each crop plot

<figure><img src="image_95.png" /></figure>

<figure><img src="image_96.png" /></figure>

<figure><img src="image_97.png" /></figure>
Thank you / Questions ?

Git SO FAIR project : https://github.com/geosas/sofair

Git STEAN : https://github.com/Mario-35/STEAN

Our research unit SDI GéoSAS : https://geosas.fr