# "Sensor augmented weather prediction at high resolution" [anr-21-ce04-0014-03]" project DMP

Christophe Paoli, Jean-François Muzy, Frédéric Bosseur, Jean Baptiste Filippi, Marie Laure Nivet, Xavier Silvani, Rachel Baïle, Gilles Notton, Ghjuvan Antone Faggianelli, Cyril Voyant, et al.

► To cite this version:

Christophe Paoli, Jean-François Muzy, Frédéric Bosseur, Jean Baptiste Filippi, Marie Laure Nivet, et al. "Sensor augmented weather prediction at high resolution" [anr-21-ce04-0014-03]" project DMP. Université de Corse-Pascal-Paoli. 2022. hal-03737267v2

HAL Id: hal-03737267

https://hal.science/hal-03737267v2

Submitted on 2 Feb 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# "SENSOR AUGMENTED WEATHER PREDICTION AT HIGH-RESOLUTION" [ANR-21-CE04-0014] PROJECT DMP V2

A Data Management Plan created using DMP OPIDoR, based on the "ANR - DMP template (english)" template provided by Agence nationale de la recherche (ANR).

## PLAN DETAILS

<table><tr><td>Plan title</td><td>"Sensor Augmented weather Prediction at High-Resolution"<br/>Project DMP v2</td></tr><tr><td>Deliverable</td><td>1.1.3</td></tr><tr><td>Version</td><td>Mid-term version</td></tr><tr><td>Plan<br/>purpose/scope</td><td>This second version of the SAPHIR Data Management Plan (DMP) is deliverable 1.1.3 of the work package 1.1 "Data Base design and Data Management Plan". This document is a mid-term update of the first DMP (Deliverable 1.1.1). A definitive version will be proposed at the end of the project (T48), considering especially the sharing and preservation of SAPHIR research data after the end of the project. We pay close attention to the integration of FAIR (Findable, Accessible, Interoperable and Reusable) principles to the data management process.<br/>The structure of the document follows the ANR DMP template.</td></tr><tr><td>Fields of science<br/>and technology<br/>(from OECD<br/>classification)</td><td>Earth and related environmental sciences, Physical sciences,<br/>Computer and information sciences</td></tr><tr><td>Language</td><td>Eng</td></tr><tr><td>Creation date</td><td>2022-02-04</td></tr><tr><td>Last modification<br/>date</td><td>2024-09-24</td></tr><tr><td>Identifier</td><td></td></tr><tr><td>Identifier type</td><td></td></tr><tr><td>License</td><td>Name Creative Commons Attribution Non-Commercial Share<br/>Alike 4.0 International<br/>URL http://spdx.org/licenses/CC-BY-NC-SA-4.0.json</td></tr><tr><td>Associated<br/>documents</td><td>SAPHIR Meta Data repository: SAPHIR Dataverse<br/>Project website: SAPHIR website<br/>SAPHIR Code repository: Koda SAPHIR (access on demand)</td></tr></table>

## PROJECT DETAILS

<table><tr><td>Project<br/>title</td><td>Sensor Augmented weather Prediction at High-Resolution [ANR-21-CE04-0014]</td></tr><tr><td>Acronym</td><td>SAPHIR</td></tr></table>
**Abstract** The project "Sensor Augmented weather Prediction at High Resolution" (SAPHIR) takes place within the fast growing and very promising context of data-driven weather forecasting methods. SAPHIR project consists in exploiting the outputs of a high-resolution NWP system and a variety of (real-time or post-processed) data gathered from meteorological safety sensor networks, satellite observations and also from ad hoc local sensor networks through a "Deep Learning" architecture. We will focus more precisely on two different outcomes at some specific locations in Corsica, namely the prediction of episodes of intense thunderstorms with significant rainfalls and lightning activity and wind/solar power forecasting. Indeed, severe weather conditions and notably thunderstorms and flooding are among one of the most dangerous weather risks in the Mediterranean region. Even if warning systems based on operational NWP forecasts provide reliable recommendations, accurate predictions at high resolution of the occurrence, location, timing and intensity of extreme events remain very challenging especially in a complex mountainous and maritime region such as the island of Corsica. This issue is especially critical since the global warming notably increases the occurrence of extreme weather events. In the field of renewable energy (RE), the power delivery of wind farms and solar power plants is strongly dependent on the volatile energy resource and its intermittency. As a consequence, it is characterized by a large amount of uncertainty that may be costly and difficult to manage for energy suppliers. However, the share of electricity produced by photovoltaics and wind energy systems increases and according to France Multi Annual Energy Plan, the installed capacity for renewable electricity in 2028 will be doubled as compared to 2017. Then, accurate wind speed and solar irradiance predictions and thereby forecast of power delivery are more and more important for the energy managers in order to make these intermittent energy sources amenable to integration into a power grid. Our ambition in that respect will be (i) to study the possibility of improving current predictions at horizons from few tens of minutes to 1 day, (ii) to consider microgrid flow control issues associated with such predictions and (iii) to study the possibility to predict extreme wind events in order to mitigate their impact on energy production.

**Funding** French National Research Agency: ANR-21-CE04-0014

**Start date** 2022-01-06

**End date** 2026-02-05

**Partners**
- University of Corsica Pasquale Paoli, UMR CNRS 6134 "Sciences pour l'Environnement" (SPE) (200012203U)
- INRIA Paris Research Center (196718247G)
- Laboratory of Aerology, LAERO UMR 5560 (199512000V)
Contributors

<table><thead><tr><td>Name</td><td>Affiliation</td><td>Roles</td></tr></thead><tbody><tr><td>Muzy Jean-François</td><td>SPE UMR 6134 CNRS, University Of Corsica Pasquale Paoli</td><td>Project coordinator<br>Forecasting models</td></tr><tr><td>Baïle Rachel</td><td>SPE UMR 6134 CNRS, University Of Corsica Pasquale Paoli</td><td>Forecasting models (Wind)</td></tr><tr><td>Roberta Baggio</td><td>SPE UMR 6134 CNRS, University Of Corsica Pasquale Paoli</td><td>Forecasting models</td></tr><tr><td>Bosseur Frédéric</td><td>SPE UMR 6134 CNRS, University Of Corsica Pasquale Paoli</td><td>HPC facilities</td></tr><tr><td>Coquillat Sylvain</td><td>LAERO, University Toulouse III - Paul Sabatier</td><td>Atmospheric Science</td></tr><tr><td>De Guibert Pierre</td><td>LAERO, University Toulouse III - Paul Sabatier</td><td>Data manager (Atmospheric instrumentation)</td></tr><tr><td>Di Martino Bernard</td><td>SPE UMR 6134 CNRS, University Of Corsica Pasquale Paoli</td><td>Flooding simulations</td></tr><tr><td>Duchaud Jean-Laurent</td><td>SPE UMR 6134 CNRS, University Of Corsica Pasquale Paoli</td><td>Data contributor (SPE ENR instrumentation)</td></tr><tr><td>Escobar Juan</td><td>LAERO, CNRS</td><td>NWP, Meso-NH support</td></tr><tr><td>Faggianelli Ghjuvan Antone</td><td>SPE UMR 6134 CNRS, University Of Corsica Pasquale Paoli</td><td>Forecasting models (Energy production)</td></tr><tr><td>Filippi Jean-Baptiste</td><td>SPE UMR 6134 CNRS, University Of Corsica Pasquale Paoli</td><td>NWP, and forecasting models</td></tr><tr><td>Grante Diego</td><td>SPE UMR 6134 CNRS, University Of Corsica Pasquale Paoli</td><td>Forecasting models</td></tr><tr><td>Killian-Pujol Nicolas</td><td>LAERO, University Toulouse III - Paul Sabatier</td><td>NWP, and forecasting models</td></tr><tr><td>Lambert Dominique</td><td>LAERO, University Toulouse III - Paul Sabatier</td><td>Coordinator LAERO</td></tr><tr><td>Nivet Marie-Laure</td><td>SPE UMR 6134 CNRS, University Of Corsica Pasquale Paoli</td><td>DMP manager, Data Stewart</td></tr><tr><td>Notton Gilles</td><td>SPE UMR 6134 CNRS, University Of Corsica Pasquale Paoli</td><td>Forecasting models (Resource)</td></tr><tr><td>Pantillon Florian</td><td>LAERO, CNRS</td><td>NWP - Meso-NH simulations</td></tr><tr><td>Paoli Christophe</td><td>SPE UMR 6134 CNRS, University Of Corsica Pasquale Paoli</td><td>DMP manager, Data Stewart</td></tr><tr><td>Roblou Laurent</td><td>CNRS</td><td>Ocean science</td></tr><tr><td>Sainte-Marie Jacques</td><td>INRIA</td><td>Coordinator INRIA</td></tr><tr><td>Silvani Xavier</td><td>SPE UMR 6134 CNRS, University Of Corsica Pasquale Paoli</td><td>Data manager - Sensor designing</td></tr><tr><td>Voyant Cyril</td><td>SPE UMR 6134 CNRS, University Of Corsica Pasquale Paoli</td><td>Forecasting models (Solar)</td></tr><tr><td>Wautelet Philippe</td><td>CNRS</td><td>Computer science - Meso-NH support</td></tr></tbody></table>
"SENSOR AUGMENTED WEATHER PREDICTION AT HIGH-RESOLUTION" [ANR-21-CE04-0014-03]" PROJECT DMP V2

1. DATA DESCRIPTION AND COLLECTION OR RE-USE OF EXISTING DATA

1a. How will new data be collected or produced and/or how will existing data be re-used?

A part of the data will be produced ex nihilo and/or collected in various ways:

* Ground measurement campaigns based on sensors (cup or sonic anemometers, thermocouples, solar measuring PV cells, micro rain radar, rain gauges, disdrometers, etc.) installed over a set of selected sites on land and at sea around the town of Ajaccio (Lat: 41° 55' 0.02", Long: 8° 43' 59.99"), which has been chosen as an experimental zone.

* Computed outputs of the high-resolution NWP meteorological research code Meso-NH, including the ground fields of temperature (°C), pressure (Pa), water vapor content (kg/kg), cumulative rain precipitation (mm), turbulent kinetic energy (m2s-2) and horizontal wind components (km/h).

* Computed outputs of the SAPHIR forecasting models under development.

* Source codes produced for the SAPHIR project shared via the CNRS Koda software forge.

Another part is existing data with different uses: calibration or preparation of the models under development; experimental testing of new forecasting methods

* Ground meteorological observation data made available under strict conditions by the French National Meteorological Service or other national meteorological services.

* Ground meteorological observation data recorded in Ajaccio, France, on the SPE UMR CNRS 6134 lab main building.

* European Centre for Medium-Range Weather Forecasts (ECMWF) operational forecasts.

* Time and space locations of VHF sources that are radiated by lightning flashes in the 60-66 MHz VHF band.

* Data provided by Vigicrues is available on many Corsican rivers and historical records of recent rain events.

* Global Horizontal Irradiance data from the Agroclimatic Information System for Irrigation (SiAR) made available under specific conditions by the Spanish Ministry of Agriculture, Fisheries, Food and the Environment.

* Hourly mean wind speeds recorded by the Royal Netherlands Meteorological Institute (KNMI) at various stations spread over the Netherlands.

This list of data could be further expanded with other data types and sources used during the SAPHIR experiments.
The metadata of new or existing SAPHIR data sets are described and referenced in the repository Harvard Dataverse SAPHIR (shown in the screen-shot below), making our data more discoverable to the research community.

<figure><img src="image_2.png" /><figcaption>Figure 1: Screenshot of the [DataVerse associated with the SAPHIR project](#) (September 2024)</figcaption></figure>

## 1b. What data (for example the kind, formats, and volumes), will be collected or produced?

Most of the data manipulated in the SAPHIR project are in digital format. They are stored in agreed weather common formats and standards: csv, NetCDF4 (Network Common Data Form) which contains a straightforward description of the contained data (matrices tensors dimensions, variables names and stored measure units), and GRIB (GRidded Binar).

We estimate the need for data delivered by various data sources throughout the project duration to be at least 20 Tb of data storage (sensor data, specific data collected from existing sources and data generated within SAPHIR WPs) and 100 Tb of archiving capacity (NWP model outputs).
## 2. DOCUMENTATION AND DATA QUALITY

**2a. What metadata and documentation (for example the methodology of data collection and way of organising data) will accompany the data?**

Our final objective concerning SAPHIR data is to be hosted on the [AERIS web platform](.). AERIS is an atmosphere Data and Services Centre that federates national data management activities and atmospheric science expertise. Subsequently, we choose the AERIS metadata standard to describe our data sets.

Pending the acceptance of the SAPHIR project by AERIS, we first create a [SAPHIR Dataverse collection](https://dataverse.cns.suveraen.eu) to allow partners to rigorously describe their research data sets. Data set creation and deposit in this research data repository enhance the web visibility and academic credit of the data, both inside and outside the SAPHIR project.

**2b. What data quality control measures will be used?**

For data issued from Ground measurement campaigns, data consistency and quality are defined and controlled depending on the type of data and the physical signal measured. Indeed, the concept of data quality depends on many factors, relating to the research objectives and methodologies used. Ad hoc quality control procedures are used and described throughout the various [scientific publications resulting from SPAHiR project](.).

In all the cases, the raw data are stored and qualified with metrics collectively determined, according to community standards (amount of missing data, duration of the lack of data...). Raw data without processing are also saved.

## 3. STORAGE AND BACKUP DURING THE RESEARCH PROCESS

**3a. How will data and metadata be stored and backed up during the research?**

Pending the acceptance of the SAPHIR project by AERIS we use a Dataverse instance to publish [SAPHIR metadata](.).

Concerning the data themselves, they are stored on the SPE laboratory server with an incremental automatic daily backup provided by the University of Corsica IT support service.

**3b. How will data security and protection of sensitive data be taken care during the research?**

The University of Corsica IT support service provides us a data storage and recover service throughout SAPHIR project duration.

Different cases must be considered concerning the access to the data:
* Ground measurement campaigns and SAPHIR forecasting will be upload to a website dedicated to SAPHIR data: data collected or produced over the course of the research will be freely accessible to everyone on condition that due acknowledgement is made.

* Reused data set will be accessible after authentication (login/psw) to partners only.

**4. LEGAL AND ETHICAL REQUIREMENTS, CODE OF CONDUCT**

**4a. If personal data are processed, how will compliance with legislation on personal data and on security be ensured?**

No personal data is processed.

**4b. How will other legal issues, such as intellectual property rights and ownership, be managed? What legislation is applicable?**

Data produced during the SAPHIR project will be openly accessible according to the principles "as open as possible and as closed as necessary" as promoted by ANR in its Open Research Data orientation initiative.

**4c. What ethical issues and codes of conduct are there, and how will they be taken into account?**

Scientists involved in SAPHIR project pay particular attention to research integrity promoted by The European Code of Conduct for Research Integrity and the french national charter for research integrity promoted by CNRS.

**5. DATA SHARING AND LONG-TERM PRESERVATION**

**5a. How and when will data be shared? Are there possible restrictions to data sharing or embargo reasons?**

First, we created a SAPHIR Dataverse collection to allow partners to rigorously describe their research data sets (Cf. Figure 1, p.5). All the data sets will be accessible and visible both inside and outside the SAPHIR project.

At the end of the SAPHIR project, AERIS website will host our data sets, ensuring and maximising FAIR principles respect.

Currently, no embargo period is planned for data produced throughout the project duration.

**5b. How will data for preservation be selected, and where data will be preserved long-term (for example a data repository or archive)?**
No decision on data preservation selection has been made for the moment. This point will
be discussed among partners during the next months.

Despite this, we have already identified [AERIS](https://www.atmosphere.eu/en/data-and-service-for-the-atmosphere-center-as-the-best-solution-for-long-term-data-preservation), the Data and Service for the Atmosphere Center as the best solution for long-term data preservation. We will submit a data hosting application to AERIS scientific comity.

**5c. What methods or software tools are needed to access and use data?**

[Dataverse](https://dataverse.org/) repository is used for metadata exposition and storage.

Ground measurement data are stored in an open-source Time Series Database [Influx DB](https://influxdb.com/), and PostgreSQL database systems.

Numerical Weather Prediction input and output are directly stored on the high-
performance computer data directories and are accessible by classic FTP.

[Grafana](https://grafana.com/) an open-source analytics and monitoring solution is used to query and visualize
times series.

Data are stored according to standard format (csv, NetCDF,...). The re-use of data is
possible using the standard software or library (Python xarray, Pandas, MATLAB,
spreadsheet software, ParaView, etc.).

Ad hoc Python code is provided to facilitate the re-use of complex re-aggregated data (input and output of Original Forecasting SAPHIR models).

Models and source codes are share by the project partners via the CNRS public forge
named Koda.

**5d. How will the application of a unique and persistent identifier (such as a Digital Object Identifier (DOI)) to each data set be ensured?**

Every data sets are declared and published on the [SAPHIR Dataverse](https://dataverse.org/) and - at the end of the project - on [AERIS](https://www.atmosphere.eu/en/data-and-service-for-the-atmosphere-center-as-the-best-solution-for-long-term-data-preservation). Both these repositories automatically assign a unique Digital Object Identifier (DOI) which is a persistent identifier that can be used to facilitate the identification of data sets.

**6. DATA MANAGEMENT RESPONSIBILITIES AND RESOURCES**

**6a. Who (for example role, position, and institution) will be responsible for data management (i.e. the data steward)?**

The ground measurement campaigns are under the operational responsibility of Mr. Xavier Silvani research engineer at SPE Laboratory.
Each partner has a designated data manager in charge of data inventory and responsible for guaranteeing the availability of his data.

Mr. Pierre De Guibert (pierre.deguibert@aero.obs-mip.fr) is the data manager for athmospheric instrumentation at LAERO, University Toulouse III - Paul Sabatier. He is responsible for the Level 1 (L1) SAETTA data set. It consists in time and space locations of VHF sources that are radiated by lightning flashes in the 60-66 MHz VHF band.

Mr. Jean-Baptiste Filippi (filippi_j@univ-corse.fr) for SPE and Mr. Florian Pantillon (florian.pantillon@aero.obs-mip.fr) for LAERO are both data producer, in charge of the Corsica NWP data set produced by the meteorological research code Meso-NH.

The quality procedures are discussed between all the data expert partners and are specific to the different data.

Mr. Frédéric Bosseur BOSSEUR (bosseur_f@univ-corse.fr) research engineer in charge at SPE Laboratory of high performance computing facilities, is in charge of overseeing the Data Research Infrastructure support.

The two heads of the DMP are Mr. Christophe PAOLI (paoli_c@univ-corse.fr) and Ms. Marie-Laure NIVET (nivet_m@univ-corse.fr) both senior lecturers at SPE Laboratory. They are jointly assuming the role of data stewards and ensure the regular updating of the DMP.

Archiving will be managed jointly by the IT facilities of the University of Corsica and after the end of the SAPHIR project, by AERIS.

**6b. What resources (for example financial and time) will be dedicated to data management and ensuring that data will be FAIR (Findable, Accessible, Interoperable, Re-usable)?**

In the SAPHIR project, a working package is dedicated to the database design and Data Management Plan. The main goal of this package is to gather and organize data. This plan is elaborated by three Data Managers (involvement of 3 p.m each) together with all partners. The Data Managers are in charge of the data management and also ensure that data will be FAIR: Findable, Accessible, Interoperable, Reusable.