# Training neural mapping schemes for satellite altimetry with simulation data

Quentin Febvre, Julien Le Sommer, Clément Ubelmann, Ronan Fablet

► To cite this version:

Quentin Febvre, Julien Le Sommer, Clément Ubelmann, Ronan Fablet. Training neural mapping schemes for satellite altimetry with simulation data. 2023. hal-04312218

HAL Id: hal-04312218

https://hal.science/hal-04312218v1

Preprint submitted on 28 Nov 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# Training neural mapping schemes for satellite altimetry with simulation data

Q. Febvre¹,⁴, J. Le Sommer², C. Ubelmann³, R. Fablet¹,⁴

¹IMT Atlantique, Lab-STICC, Brest, France

²Université Grenoble Alpes, CNRS, IRD, Grenoble, France

³Datlas, Grenoble, France

⁴INRIA Team ODYSSEY

**Key Points:**

* We propose to train neural mapping schemes for real altimeter data from ocean simulation data.

* The trained neural schemes significantly outperform the operational mapping of real altimetry data for a Gulf Stream case-study.

* More realistic simulation datasets improve the performance of the trained neural mapping with a 20% improvement in the spatial scales.
**Abstract**

Satellite altimetry combined with data assimilation and optimal interpolation schemes have deeply renewed our ability to monitor sea surface dynamics. Recently, deep learning (DL) schemes have emerged as appealing solutions to address space-time interpolation problems. The scarcity of real altimetry dataset, in terms of space-time coverage of the sea surface, however impedes the training of state-of-the-art neural schemes on real-world case-studies. Here, we leverage both simulations of ocean dynamics and satellite altimeters to train simulation-based neural mapping schemes for the sea surface height and demonstrate their performance for real altimetry datasets. We analyze further how the ocean simulation dataset used during the training phase impacts this performance. This experimental analysis covers both the resolution from eddy-present configurations to eddy-rich ones, forced simulations vs. reanalyses using data assimilation and tide-free vs. tide-resolving simulations. Our benchmarking framework focuses on a Gulf Stream region for a realistic 5-altimeter constellation using NEMO ocean simulations and 4DVar-Net mapping schemes. All simulation-based 4DVarNets outperform the operational observation-driven and reanalysis products, namely DUACS and GLORYS. The more realistic the ocean simulation dataset used during the training phase, the better the mapping. The best 4DVarNet mapping was trained from an eddy-rich and tide-free simulation datasets. It improves the resolved longitudinal scale from 151 kilometers for DUACS and 241 kilometers for GLORYS to 98 kilometers and reduces the root mean squared error (RMSE) by 23% and 61%. These results open research avenues for new synergies between ocean modelling and ocean observation using learning-based approaches.

**Plain Language Summary**

For an artificial intelligence (AI) to learn, one need to describe a task using data and an evaluation procedure. Here we aim at constructing images related to the ocean surface currents. The satellite data we use provide images of the ocean surface with a lot of missing data (around 95% of missing pixels for a given day), and we aim at finding the values of the missing pixels. Because we don't know the full image, it is challenging to train an AI on this task using only the satellite data. However, today's physical knowledge makes it possible to numerically simulate oceans on big computers. For these simulated oceans, we have access to the gap-free image, so we can train AI models by first hiding some pixels and checking if the model fill the gaps with the correct values. Here, we explore under which conditions AIs trained on simulated oceans are useful for the real ocean. We show that today's simulated oceans work well for training an AI on this task and that training on more realistic simulated oceans improve the performance of the AI!

**1 Introduction**

Satellite altimeters have brought a great leap forward in the observation of sea surface height on a global scale since the 80's.

Altimetry data have greatly contributed to the monitoring and understanding of key processes such as the sea-level rise and the role of mesoscale dynamics. The scarce and irregular sampling of the measurements presents a challenge for training deep neural networks. The retrieval of mesoscale-to-submesoscale sea surface dynamics for horizontal scales smaller than 150 km however remains a challenge for operational systems based on optimal interpolation (Taburet et al., 2019) and data assimilation (Lellouche et al., 2021) schemes. This has motivated a wealth of research to develop novel mapping schemes (Ballarotta et al., 2020; Ubelmann et al., 2021; Guillou et al., 2021).

In this context, data-driven and learning-based approaches (Alvera Azcarate et al., 2005; Barth et al., 2022; Lguensat et al., 2017; Fablet, Amar, et al., 2021; Martin et al.,
2023) appear as appealing alternatives to make the most of the available observation and simulation datasets. Especially, Observing System Simulation Experiments (OSSE) have stressed the potential of neural schemes trained through supervised learning for the mapping of satellite-derived altimetry data (Fablet, Amar, et al., 2021; Beauchamp et al., 2023). Their applicability to real datasets has yet to be assessed and recent studies have rather explored learning strategies from real gappy multi-year altimetry datasets (Martin et al., 2023). Despite promising results, schemes trained with unsupervised strategies do not reach the relative improvement of the operational processing suggested by OSSE-based studies.

Here, we go beyond using OSSEs as benchmarking-only testbeds. We explore their use for the training of neural mapping schemes and address the space-time interpolation of real satellite altimetry observations. Through numerical experiments on a Gulf Stream case-study with a 5-nadir altimeter constellation, our main contributions are three-fold. We demonstrate the relevance of the simulation-based learning of neural mapping schemes and their generalization performance for real nadir altimetry data. We benchmark the proposed approach with state-of-the-art operational products as well as neural schemes trained from real altimetry datasets. We also assess how the characteristics of the training datasets, especially in terms of resolved ocean processes, drives the mapping performance. To ensure the reproducibility of our results, our code is made available through an open source license along with the considered datasets and the trained models (Febvre, 2023).

The content of this paper is organized as follows. Section 2 offers background information on related work, Section 3 presents our method, Section 4 reports our numerical experiments, and Section 5 elaborates on our main contributions.

# 2 Background

## 2.1 Gridded satellite altimetry products

The ability to produce gridded maps from scattered along-track nadir altimeter measurements of sea surface height is key to the exploitation of altimeter data in operational services and science studies (Abdalla et al., 2021). As detailed below, we can distinguish three categories of approaches to produce such maps: reanalysis products (Lellouche et al., 2021) using data assimilation schemes, observation-based products (Taburet et al., 2019) and learning-based approaches (Fablet, Amar, et al., 2021).

Reanalysis products such as the GLORYS12 reanalysis (Lellouche et al., 2021) leverage the full expressiveness of state-of-the-art ocean models. They aim at retrieving ocean state trajectories close to observed quantities through data assimilation methods including among others Kalman filters and variational schemes (Carrassi et al., 2018). Such reanalyses usually exploit satellite-derived and in situ data sources. For instance, GLORYS12 reanalysis assimilates satellite altimetry data, but also satellite-derived observations of the sea surface temperature, sea-ice concentration as well as in situ ARGO data (Wong et al., 2020).

The second category involves observation-based products. In contrast to reanalyses, they only rely on altimetry data and address a space-time interpolation problem. They usually rely on simplifying assumptions on sea surface dynamics. In this category, optimal-interpolation-based product DUACS (Data Unification and Altimeter Combination System) (Taburet et al., 2019) exploits a covariance-based prior, while recent studies involve quasi-geostrophic dynamics to guide the interpolation scheme (Guillou et al., 2021; Ballarotta et al., 2020).

Data-driven and learning-based approaches form a third category of SSH mapping schemes. Similarly to observation-based methods, they are framed as interpolation schemes.
Especially deep learning schemes have gained some attention. Recent studies have explored different neural architectures both for real and OSSE altimetry datasets (Archambault et al., 2023; Beauchamp et al., 2021; Martin et al., 2023). These studies investigate both different training strategies as well as different neural architectures from off-the-shelf computer vision ones such as convolutional LSTMs and UNets (Ronneberger et al., 2015) to data-assimilation-inspired ones (Beauchamp et al., 2021; Fablet, Chapron, et al., 2021).

## 2.2 Ocean Modeling and OSSE

Advances in modeling and simulating ocean physics have largely contributed to a better understanding of the processes involved in the earth system and to the development of operational oceanography (Barnier et al., 2006; Ajayi et al., 2020). High-resolution simulations used in Observing System Simulation Experiments (OSSE) also provide a great test-bed for the design and evaluation of new of ocean observation systems (Benkiran et al., 2021). The availability of numerical model outputs enables the computation of interpretable metrics directly on the quantities of interest. This avoids challenges met when working solely with observation data that may be incomplete, noisy or indirectly related to the desired quantity. For example, in the case of the recently launched SWOT mission, OSSEs combined ocean and instrument simulations to address calibration issues and interpolation performance for SWOT altimetry data (Dibarboure et al., 2022). Such OSSEs have also promoted novel developments for the interpolation of satellite altimetry such as the BFN-QG and 4DVarNet schemes (Guillou et al., 2021; Beauchamp et al., 2023).

In OSSE settings, we can train learning-based mapping schemes in a supervised manner using model outputs as the "ground truth" during the training phase. Nonetheless, these training methods cannot be straightforwardly applied to Observing System Experiments (OSEs) due to a lack of comprehensive groundtruthed observation datasets. Applied machine learning practitioners often grapple with insufficient amount of labelled data during the training of supervised learning schemes, as the collection of large annotated datasets for a specific task can be costly or unattainable. Proposed solutions includes the exploitation of large existing datasets (such as ImageNet Deng et al. (2009)) to train general purpose models (like He et al. (2016)). Another approach involves the generation of synthetic datasets to facilitate the creation of groundtruthed samples (Gomez Gonzalez et al., 2017; Dosovitskiy et al., 2015). OSSEs, which combine ocean model outputs and observing system simulators (Boukabara et al., 2018), can deliver such large synthetic groundtruthed datasets. We propose to investigate how OSSE-based training strategies apply to the analysis of real satellite altimetry datasets. Recent results of SSH super-resolution model trained on simulation datasets and evaluated on real ones (Buongiorno Nardelli et al., 2022) support the relevance of such strategies.

## 2.3 Physics-aware deep-learning

In the last decades, DL advances combined with the rise in computational resources and amount of data have shown the power of extracting knowledge from data in domains ranging from computer vision to language processing (LeCun et al., 2015). Yet, despite to the universality of DL architectures (Hornik et al., 1989), a central challenge persists in learning from data: the generalization performance beyond the distribution of the training data. To tackle this problem, the literature includes a variety of strategies such as data augmentation (Shorten & Khoshgoftaar, 2019) and regularization techniques, including dropout layers (Srivastava et al., 2014) and weight decay schemes (Krizhevsky et al., 2012). This is of critical importance for physical systems, where models trained on past data will be challenged when the system evolves and reaches dynamics absent from the training data. We can see evidence of this shortcoming in the instability challenges faced by neural closures for climate models (Brenowitz et al., 2020).
There have been a variety of approaches to harness physical priors within learning schemes to address this issue. Some injects trainable components in classical integration schemes of physical models such as Yin et al. (2021). Others leverage physical priors within their learning setups which can been used in the training objective (Raissi et al., 2019; Greydanus et al., 2019), as well as in the architecture (Li et al., 2020; Wang et al., 2020). However most of these works have focused on relatively simple physical models and it remains challenging to combine current state-of-the-art ocean models with such methods. Obstacles include the complexity and cost of running the physical models, the differences in programming tools and the computing infrastructures used in each domain, as well as the availability of automatic differentiation tools for state-of-the-art ocean models.

The proposed simulation-based training strategy offers another way to benefit from
the advances in high-resolution ocean modeling in the design of deep neural models for
ocean reanalysis problems.

**3 Method**

**3.1 Overview**

We designate our approach as "simulation-based", it consists in leveraging ocean
models and simulations of observing systems to design supervised training environments.
In this section, we describe the proposed method for assessing the potential of simulation-
based neural schemes for the mapping real altimetry tracks. We describe the architec-
ture considered in our study, as well as the different datasets used for training purposes.
We also detail our simulation-based training setup and the proposed evaluation frame-
work on real altimetry.

<figure><img src="image_2.png" /><figcaption>Figure 1: **Overview of the experimental setup.** On the left side we display the simulation-based training strategy based on an ocean simulation which will be used for 1) generating synthetic observation and 2) computing the training objective of the neural mapping scheme. On the right side we show the evaluation principle of splitting the available satellite observations to evaluate the method on data that were not used for the inference.</figcaption></figure>
## 3.2 Neural mapping scheme

The neural mapping scheme considered for this study is the 4DVarNet framework (Fablet, Amar, et al., 2021). We choose this scheme due to the performance shown in the OSSE setup. As reported in Beauchamp et al. (2023), it significantly outperforms the DUACS product (Taburet et al., 2019) in the targeted Gulf stream region. 4DVarNet relies on a variational data assimilation formulation. The reconstruction results from the minimization of a variational cost. This cost encapsulates a data fidelity term and a regularization term. It exploits a prior on the space-time dynamics through a convolutional neural network inspired from (Fablet et al., 2018), and an iterative gradient-based minimization based on a recurrent neural network as introduced by Andrychowicz et al. (2016). The overall architecture and components are similar to those presented in Beauchamp et al. (2023). We adapt some implementation details based on cross-validation experiments to improve the performance and reduce the training time. We refer the reader to the code for more details (Febvre, 2023).

## 3.3 SSH Data

<table><caption>Table 1: Summary table of the different synthetic SSH fields used for training. The last column indicate whether the Dynamic Atmospheric Correction was applied on the synthetic SSH. It justifies the presence of both eNATL60-0 and NATL60 to isolate the impacts of resolution and tide.</caption><thead><tr><th></th><th></th><th>Resolution</th><th>Reanalysis</th><th>Tide</th><th>DAC</th></tr></thead><tbody><tr><td>NATL60</td><td>(Ajayi et al., 2020)</td><td>1/60°</td><td>No</td><td>No</td><td>No</td></tr><tr><td>eNATL60-t</td><td>(Brodeau et al., 2020)</td><td>1/60°</td><td>No</td><td>Yes</td><td>Yes</td></tr><tr><td>eNATL60-0</td><td>(Brodeau et al., 2020)</td><td>1/60°</td><td>No</td><td>No</td><td>Yes</td></tr><tr><td>GLORYS12-r</td><td>(Lellouche et al., 2021)</td><td>1/12°</td><td>Yes</td><td>No</td><td>No</td></tr><tr><td>GLORYS12-f</td><td>(Lellouche et al., 2021)</td><td>1/12°</td><td>No</td><td>No</td><td>No</td></tr><tr><td>ORCA025</td><td>(Barnier et al., 2006)</td><td>1/4°</td><td>No</td><td>No</td><td>No</td></tr></tbody></table>

We use numerical simulations of ocean general circulation models (OGCM) to build our reference SSH datasets. Such simulations involve a multitude of decisions that affect the resulting simulated SSH. Here we consider NEMO (Nucleus for European Modelling of the Ocean) (Gurvan et al., 2022) which is among the state-of-the art OGCM in operational oceanography (Ajayi et al., 2020) as well as in climate studies (Voldoire et al., 2013). The selected SSH datasets reported in Table 1 focus on three main aspects: the added-value of high-resolution eddy-rich simulations, the impact of reanalysis datasets and the relevance of tide-resolving simulations.

In order to evaluate the impact of eddy-rich simulations, we consider NATL60, GLORYS12-f and ORCA025 free runs, respectively with a horizontal grid resolution of 1/60°, 1/12°, and 1/4°. Finer grids allow for more processes to be simulated. We therefore expect higher-resolution simulations to exhibit structures closer to the real ocean and the associated trained deep learning model to perform better. Regarding the impact of reanalysis data, we compare numerical experiments with the GLORYS12-r reanalysis and the associated free run GLORYS12-f. This reanalysis dataset relies on the assimilation of temperature, sea level and sea ice concentration observations. Besides, the recent eNATL60 twin simulations eNATL60-t and eNATL60-0 allow us to evaluate the impact of tide-resolving simulations. We summarize in Table 1 the characteristics of the different datasets.
## 3.4 OSSE-based training setup

We sketch the proposed OSSE-based training setup on the left side of the Figure 1. In order to fairly evaluate the datasets' quality as a training resource, we standardize the training procedure. We regrid all simulations to the same resolution (1/20°) and we use daily-averaged SSH fields as training targets. We generate noise-free pseudo-observations by sampling values of the daily-averaged fields corresponding to realistic orbits of a 5 altimeter-constellation. We train all models from a one-year dataset in a Gulfstream domain from (66°W, 32°N) to (54°W, 44°N) in which we keep the same two months for validation. The hyper-parameters of the model and training procedure such as the number of epoch, learning rate scheduler are the same for all the experiments. The detailed configuration can be found by the reader in the available implementation. As training objective, we combine the mean square errors for the SSH fields and the amplitude of the gradients as well as a regularization loss for the prior model.

## 3.5 OSE-based evaluation setup

As sketched on the right side of the Figure 1, the evaluation setup relies on real altimetry data from the constellation of 6 satellites from 2017 (SARAL/Altika, Jason 2, Jason 3, Sentinel 3A, Haiyang-2A and Cryosat-2). We apply the standardized setup presented in a data-challenge https://github.com/ocean-data-challenges/2021a_SSH_mapping-OSE. We use the data from the first five satellites as inputs for the mapping and the last one (Cryosat-2) for computing the performance metrics. We compute these metrics in the along-track geometry. The evaluation domain spans from (65°W, 33°N) to (55°W, 43°N) and the evaluation period from January 1st to December 31st 2017. Given $\eta_{c2}$ and $\hat{\eta}$ the measured SSH and the reconstructed SSH respectively, we compute the following two metrics:

* $\mu_{ssh}$ is a score based on the normalized root mean squared (nRMSE) error computed as $1 - \frac{RMS(\hat{\eta} - \eta_{c2})}{RMS(\eta_{c2})}$

* $\lambda_x$ is the wavelength at which the power spectrum density (PSD) score $1 - \frac{PSD(\hat{\eta} - \eta_{c2})}{PSD(\eta_{c2})}$ crosses the 0.5 threshold, which characterize the scales resolved by the reconstruction (the error below that wavelength makes up for more than half of the total signal)

In Table 3, we also consider the root mean square error (RMSE) as well as the nRMSE score of the sea level anomaly $\mu_{sla}$ obtained by subtracting the mean dynamic topography to the SSH. Lastly, we assess the performance degradation resulting from the transition from simulated to real data by quantifying the improvement relative to DUACS in the resolved scale $\lambda_x$ on our OSE setup as well as on the OSSE benchmarking setup proposed in Guillou et al. (2021). This benchmarking setup relies on NATL60-CJM165 OSSE dataset. We refer the reader to https://github.com/ocean-data-challenges/2020a_SSH_mapping\_NATL60 for a detailed description of this experimental setup.
<figure><img src="image_3.png" /><figcaption>Figure 2: Samples Kinetic energy and relative vorticity of the training and reconstruction data of January 6^th^ . The reconstructed year is 2017 while the training year vary depending on the simulation. The first two columns (a) and (b) show the training data while columns (c) and (d) show the associated 4DVarNet reconstruction. The kinetic energy is displayed in columns ((a) and (c)) and the relative vorticity normalized by the local Coriolis parameter in columns ((b) and (d)). Each row shows the experiment using respectively: ORCA025 (I), GLORYS12-f (II), GLORYS12-r (III), NATL60 (IV), eNATL60-t (V) and eNATL60-0 (VI)</figcaption></figure>
# 4 Results

This section details our numerical experiments for the considered real altimetry case-study for a Gulf Stream region as described in Section 3.5. We first report the benchmarking experiments to assess the performance of the proposed learning-based strategy with respect to (w.r.t.) state-of-the-art mapping schemes. We then analyse how the characteristics of the training datasets drive the mapping performance.

## 4.1 Benchmarking against the state of the art

We report in Table 2 the performance metrics of state-of-the-art approaches including both operational observation products (Taburet et al., 2019; Ubelmann et al., 2021), deep-learning-based schemes trained on observation data (Archambault et al., 2023; Martin et al., 2023) as well as methods using explicitly a model-based prior on sea surface dynamics (Guillou et al., 2021; Ballarotta et al., 2020; Lellouche et al., 2021). We compare those methods with a 4DVarNet trained on eNATL60-0 OSSE dataset. The latter outperforms all other methods on the two metrics considered (22% improvement in RMSE w.r.t. the DUACS product and 33% improvement in the resolved scale). We report a significantly worse performance for GLORYS12 reanalysis. This illustrates the challenge of combining large ocean general circulation models and observation data for the mapping of the SSH.

The last column indicates that the 4DVarNet scheme leads to the best mapping scores for both the OSE and OSSE setups. For the latter, the reported improvement of 47% is twice greater than the second best at 22%. The performance of the 4DVarNet drops by 11% when considering the former. By contrast, other methods do not show such differences between the OSE and OSSE case-studies. This suggests that the finer-scale structures that are well reconstructed in the OSSE setup are not as beneficial in the OSE setup. While one could question the representativeness of the OSSE datasets for the fine-scale patterns in the true ocean, real nadir altimetry data may also involve multiple processes which could impede the reconstruction and evaluation of horizontal scales below 100km.

<figure><img src="image_4.png" /><figcaption>Figure 3: Space-time spectral densities of the training datasets (first row) and of their associated reconstruction (second row). Darker blue in the lower left corner indicates higher energy at larger wavelength and periods. The different SSH fields exhibit different energy cascades when moving to finer temporal (upward) or spatial (rightward) scales.</figcaption></figure>
<table><caption>Table 2: SSH reconstruction performance of the benchmarked methods (a)
4DVarNet from this study trained on eNATL60-0 (b) Archambault et al. (2023), (c and d) ConvLstm-SST and ConvLstm from Martin et al. (2023), (e) DYMOST from Bal-larotta et al. (2020), (f) MIOST from Ubelmann et al. (2021), (g) BFN-QG from Guillou et al. (2021), (h) DUACS from Taburet et al. (2019), (i) GLORYS12 from Lellouche et al. (2021). The columns indicate from left to right: whether the mapping schemes rely only on SSH data or also exploit additional data such as gap free SST products; if the method uses deep learning architectures; the data used to calibrate (or train) the mapping scheme; the numerical model of the ocean used for the mapping if any (QG stands for quasi-geostrophic); μ and λx are the metrics as described in Section 3.5</caption><thead><tr><th></th><th>SSH<br>Only</th><th>Deep<br>Learning</th><th>Calibrated on<br>data from</th><th>Physical<br>Model</th><th>rmse<br>(cm)</th><th>μ<sub>ssh</sub><br>()</th><th>λ<sub>x</sub><br>(km)</th><th>1 - λ<sub>x</sub>/λ<sub>ref</sub><br>(% ose, osse)</th></tr></thead><tbody><tr><td>(a) 4DVarNet</td><td>Yes</td><td>Yes</td><td>Simulation</td><td>–</td><td><b>5.9</b></td><td><b>0.91</b></td><td><b>100</b></td><td><b>33, 47</b></td></tr><tr><td>(b) MUSTI</td><td>No</td><td>Yes</td><td>Satellite</td><td>–</td><td>6.3</td><td>0.90</td><td>112</td><td>26, 22</td></tr><tr><td>(c) ConvLstm-SST</td><td>No</td><td>Yes</td><td>Satellite</td><td>–</td><td>6.7</td><td>0.90</td><td>108</td><td>28, –</td></tr><tr><td>(d) ConvLstm</td><td>Yes</td><td>Yes</td><td>Satellite</td><td>–</td><td>7.2</td><td>0.89</td><td>113</td><td>25, –</td></tr><tr><td>(e) DYMOST</td><td>Yes</td><td>No</td><td>Satellite</td><td>QG</td><td>6.7</td><td>0.90</td><td>131</td><td>13, 11</td></tr><tr><td>(f) MIOST</td><td>Yes</td><td>No</td><td>Satellite</td><td>–</td><td>6.8</td><td>0.90</td><td>135</td><td>11, 10</td></tr><tr><td>(g) BFN-QG</td><td>Yes</td><td>No</td><td>Satellite</td><td>QG</td><td>7.6</td><td>0.89</td><td>122</td><td>19, 21</td></tr><tr><td>(h) DUACS</td><td>Yes</td><td>No</td><td>Satellite</td><td>–</td><td>7.7</td><td>0.88</td><td>151</td><td>0, 0</td></tr><tr><td>(i) GLORYS12</td><td>No</td><td>No</td><td>Satellite</td><td>NEMO</td><td>15.1</td><td>0.77</td><td>241</td><td>-60, –</td></tr></tbody></table>

<table><caption>Table 3: Performance of 4DVarNet mapping schemes trained on different simulated datasets. The first column shows the source of the training dataset as described in Table 1; the subsequent columns indicate the reconstruction metrics described in Section 3.5. Note that the NATL60 could not be evaluated on the OSSE setup since the evaluation data were used for validation during the training stage.</caption><thead><tr><th>Training Data</th><th>RMSE<br>(cm)</th><th>μ<sub>ssh</sub></th><th>μ<sub>sla</sub></th><th>λ<sub>x</sub><br>(km)</th><th>1 - λ<sub>x</sub>/λ<sub>ref</sub><br>(% ose, osse)</th></tr></thead><tbody><tr><td>NATL60</td><td><b>5.9</b></td><td><b>0.91</b></td><td><b>0.80</b></td><td><b>98</b></td><td>(35, -)</td></tr><tr><td>eNATL60-t</td><td><b>5.9</b></td><td><b>0.91</b></td><td><b>0.80</b></td><td>100</td><td>(33, 48)</td></tr><tr><td>eNATL60-0</td><td><b>5.9</b></td><td><b>0.91</b></td><td><b>0.80</b></td><td>100</td><td>(33, 47)</td></tr><tr><td>GLORYS12-r</td><td>6.3</td><td>0.90</td><td>0.78</td><td>106</td><td>(30, 28)</td></tr><tr><td>GLORYS12-f</td><td>6.7</td><td>0.90</td><td>0.77</td><td>119</td><td>(21, 23)</td></tr><tr><td>ORCA025</td><td>7.1</td><td>0.89</td><td>0.76</td><td>126</td><td>(17, 17)</td></tr></tbody></table>
## 4.2 Eddy-present datasets versus eddy-rich ones

<figure><img src="image_5.png" /><figcaption>Figure 4: Spectral analysis of the training and reconstructed SSH datasets. We display the PSD of the training dataset (left plot), reconstructed SSH field (center plot) as well as the associated PSD score (right plot)</figcaption></figure>

We analyse here in more detail the impact of the spatial resolution of the training dataset onto the reconstruction performance. In Table 3, as expected, the higher resolution grid in the ocean run simulation leads to a better mapping with a 22% improvement in $\lambda_x$ and a 17% improvement in the RMSE score between the experiments with the coarsest (ORCA025) and finest (NATL60) resolutions. We also observe qualitative differences in the relative vorticity fields in Figure 2. Residual artifacts due to the altimetry tracks appear (60°W, 39°N) for the two lower-resolution training datasets. They are greatly diminished when considering the NATL60 dataset. Despite these differences, the reconstructed vorticity and kinetic energy fields in Figure 2 look very similar for the different 4DVarNet schemes, whatever the training datasets. By contrast, the vorticity and kinetic energy fields in the training datasets clearly depict fewer fine-scale structures and weaker gradients for the lower-resolution simulation datasets, namely ORCA025 and GLORYS12-f. These results support the generalization skills of 4DVarNet schemes to map real altimetry tracks despite being trained on SSH sensibly different from the reconstruction.

We draw similar conclusions from the analysis of the spectral densities shown in Figure 4. The differences in the energy distribution of the training data significantly reduce in the reconstructions. 4DVarNet schemes trained from higher-resolution datasets however result in more faithful reconstruction at all scales. The patterns observed for the temporal PSD are slightly different in Figure 3. We do not observe the same homogenization as for the spatial PSD. Lower-resolution training datasets involve a significant drop of an order of magnitude for periods greater than 10 days and wavelength greater than 200km.

## 4.3 Forced simulation datasets versus reanalysis ones

Looking in more specifically at the effect of ocean reanalysis between the two experiments GLORYS12-f and GLORYS12-r. We can first note the impact of observation data assimilation in Figure 3 where we see how the power spectrum of the reanalysis is significantly raised compared to the free run. The spectrum is closer to ones of the higher resolution simulations. Visually we also clearly see stronger gradients in the kinetic energy in Figure 2.
<figure><img src="image_6.png" /><figcaption>Figure 5: **Spectral impact of model reanalysis.** We display the PSD of the training dataset (left plot), reconstructed SSH field (center plot) as well as the associated PSD score (right plot)</figcaption></figure>

We can observe a similar behavior as in Section 4.2 in Figure 5 with the gap of in spectral density being diminished between the training and reconstruction data, and the PSD score indicating a lower energy of the error at all scales for the reanalysis-based experiment.

Quantitatively in Table 1 we see an improvement of 11% in both the RMSE and the scale resolved, besides training on a reanalysis increase the relative gain w.r.t. DU-ACS significantly more on real data (+9%) than on simulated data (+5%) as we can see in the right most column. This suggests that the reanalysis dataset conveys information on real world observations which improves the generalization performance.

## 4.4 Tide-free datasets versus tide-resolving ones

We assess here the impact of tide-resolving simulation used as training data. We use the twin eNATL60 runs eNATL60-t and eNATL60-0. Contrary to other runs, those simulations contain barometric and wind forcing, we therefore remove the Dynamic Atmospheric Correction (Carrere et al., 2016) from the SSH fields. Additionally since the barotropic tide signals are removed from real altimetry tracks prior to interpolation, we also remove the signal from the training data by subtracting the spatial mean over the training domain for each hourly snapshot before calculating the daily averages.

Given those processing steps, the two training datasets exhibit very similar wavenumber spectra as shown in Figures 3. We also find that training on those two datasets produce little differences in the reconstructions both quantitatively (see Table 3) and qualitatively (Fig. 2). The resulting performance is comparable to that of the NATL60 experiment.

We identify two hypotheses for explaining why tide-resolving simulation do not lead to better mapping schemes:

* The preprocessing applied on the training field remove the main tide signals. We therefore effectively measure the impact of tide modeling on other ocean processes that may be less significant;

* The evaluation procedure applied on altimetry tracks on which the barotropic tide has been filtered may not be interpretable enough to measure the reconstruction of residual tide signals. New instruments like the KaRIN deployed in the SWOT mission may provide new ways to better quantify those effects.
These findings provide motivation for carefully considering the purpose of the learning-
based model when making decisions about the training data. In our case, explicitly mod-
eling tide processes that are removed from the observations in the evaluation setup added
overheads in the computational cost of running the simulation as well as in the prepro-
cessing of the training data. Additionally given the considered evaluation data and met-
rics, we were not able to quantify any significant differences between the two trained map-
ping schemes.

5 Discussion

This study has been greatly facilitated by the standardized tasks and evaluation
setups proposed in data-challenges https://ocean-data-challenges.github.io/. Data-
challenges are used to specify a targeted problem of interest to domain experts through
datasets and relevant evaluation metrics. This preliminary work have been instrumen-
tal in constituting the comprehensive benchmark and combining methods from differ-
ent teams and institution around the world. Additionally, it also constitutes a strong ba-
sis for a trans-disciplinary collaboration between the ocean and machine learning research
communities.

Moreover, the results presented in this study introduce a use of ocean simulations for developing altimetry products. This opens new ways for ocean physicist, modelers and operational oceanographers to collaborate. In order to assess the range of these new synergies, it would be interesting to explore if the approach proposed here of training neural schemes using simulation data would generalize to other tasks such as forecast or sensor calibration and to other quantities like surface temperature, currents, salinity or biochemical tracers.

If the simulation-based training approach introduced here is successfully extended
to other ocean problems, one could envision training large foundation deep learning mod-
els (Brown et al., n.d.) capturing the inner structure of high resolution ocean simulations
which could then be used in many downstream applications. This could be the way to
capitalize on all the advancement in ocean modeling without having to run OGCM nu-
merical simulation for each downstream products.

Furthermore, we would like to highlight the cost consideration when running nu-
merical simulation intended for training learning based schemes. Indeed given that the
eNATL60 run took 2700x CPU hours and 350x memory compared to the ORCA025 run
for a smaller domain, a trade-off arises between generating multiple "cheap" trajecto-
ries versus generating a single more realistic trajectory.

To conclude, we have shown in this study that training machine learning models
on simulations datasets leads good performance on real altimetry data mapping and out-
performs current state of the art approaches. The model trained on NATL60 reduces the
RMSE by 18% compared neural schemes trained on observation data and improves the
scales resolved by 33% compared to the DUACS operational product. Even the coars-
est simulation considered ORCA025 provides competitive results with current operational
methods. We have shown that using a more realistic SSH fields using reanalysis or higher
resolution simulations increases the performances of the trained model. This is an ex-
citing result that shows the potential for training operational products from ocean sim-
ulations and how advances in ocean modeling in operational oceanography can be ben-
eficial. The results shown here are limited to the interpolation problem on a regional do-
main but the robustness of the performance shown are encouraging for further develop-
ing these results using a larger domain.
**Open Research Section**

The authors provide the training data, source code, reconstructed maps and trained model for each experiments of the manuscript at https://doi.org/10.5281/zenodo.8064114.

**Acknowledgments**

This work was supported by ANR Projects Melody and OceaniX and CNES. It benefited from HPC and GPU resources from GENCI-IDRIS (Grant 2020-101030) and Ifremer.

**References**

Abdalla, S., Abdeh Kolahchi, A., Ablain, M., Adusumilli, S., Aich Bhowmick, S., Alou-Font, E., ... Zlotnicki, V. (2021, July). Altimetry for the future: Building on 25 years of progress. *Advances in Space Research, 68*(2), 319-363. doi: 10.1016/j.asr.2021.01.022

Ajayi, A., Le Sommer, J., Chassignet, E., Molines, J.-M., Xu, X., Albert, A., & Cosme, E. (2020). Spatial and Temporal Variability of the North Atlantic Eddy Field From Two Kilometric-Resolution Ocean Models. *Journal of Geophysical Research: Oceans, 125*(5), e2019JC015827. doi: 10.1029/2019JC015827

Alvera Azcarate, A., Barth, A., Rixen, M., & Beckers, J.-M. (2005). Reconstruction of incomplete oceanographic data sets using empirical orthogonal functions: Application to the Adriatic Sea surface temperature. *Ocean Modelling, 9*(4). doi: 10.1016/j.ocemod.2004.08.001

Andrychowicz, M., Denil, M., Gómez, S., Hoffman, M. W., Pfau, D., & Schaul, T. (2016, November). Learning to learn by gradient descent by gradient descent.

Archambault, T., Filoche, A., Charantonnis, A., & Béréziat, D. (2023, February). Multimodal Unsupervised Spatio-Temporal Interpolation of satellite ocean altimetry maps. In *VISAPP*.

Ballarotta, M., Ubelmann, C., Rogé, M., Fournier, F., Faugère, Y., Dibarboure, G., ... Picot, N. (2020, September). Dynamic Mapping of Along-Track Ocean Altimetry: Performance from Real Observations. *Journal of Atmospheric and Oceanic Technology, 37*(9), 1593-1601. doi: 10.1175/JTECH-D-20-0030.1

Barnier, B., Gurvan, M., Thierry, P., Jean-Marc, M., Anne-Marie, T., Julien, L. S., ... Beverly, D. C. (2006, December). Impact of partial steps and momentum advection schemes in a global ocean circulation model at eddy-permitting resolution. *Ocean Dynamics, 56*(5), 543-567. doi: 10.1007/s10236-006-0082-1

Barth, A., Alvera-Azcárate, A., Troupin, C., & Beckers, J.-M. (2022, March). DINCAE 2.0: Multivariate convolutional neural network with error estimates to reconstruct sea surface temperature satellite and altimetry observations. *Geoscientific Model Development, 15*(5), 2183-2196. doi: 10.5194/gmd-15-2183-2022

Beauchamp, M., fablet, r., Ubelmann, C., Ballarotta, M., & Chapron, B. (2021, January). Data-driven and learning-based interpolations of along-track Nadir and wide-swath SWOT altimetry observations. In *Proceedings of the 10th International Conference on Climate Informatics* (pp. 22-29). New York, NY, USA: Association for Computing Machinery. doi: 10.1145/3429309.3429313

Beauchamp, M., Febvre, Q., Georgenthum, H., & Fablet, R. (2023, April). 4DVarNet-SSH: End-to-end learning of variational interpolation schemes for nadir and wide-swath satellite altimetry. *Geoscientific Model Development, 16*(8), 2119-2147. doi: 10.5194/gmd-16-2119-2023

Benkiran, M., Ruggiero, G., Greiner, E., Le Traon, P.-Y., Rémy, E., Lellouche, J. M., ... Tchonang, B. (2021). Assessing the Impact of the Assimilation of
SWOT Observations in a Global High-Resolution Analysis and Forecasting System Part 1: Methods. *Frontiers in Marine Science*, 8.

Boukabara, S.-A., Ide, K., Zhou, Y., Shahroudi, N., Hoffman, R. N., Garrett, K., ... Atlas, R. (2018, October). Community Global Observing System Simulation Experiment (OSSE) Package (CGOP): Assessment and Validation of the OSSE System Using an OSSE-OSE Intercomparison of Summary Assessment Metrics. *Journal of Atmospheric and Oceanic Technology*, 35(10), 2061–2078. doi: 10.1175/JTECH-D-18-0061.1

Brenowitz, N. D., Beucler, T., Pritchard, M., & Bretherton, C. S. (2020, December). Interpreting and Stabilizing Machine-Learning Parametrizations of Convection. *Journal of the Atmospheric Sciences*, 77(12), 4357–4375. doi: 10.1175/JAS-D-20-0082.1

Brodeau, L., Sommer, J. L., & Albert, A. (2020, September). *Ocean-next/eNATL60: Material describing the set-up and the assessment of NEMO-eNATL60 simulations*. Zenodo. doi: 10.5281/zenodo.4032732

Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., ... Henighan, T. (n.d.). Language Models are Few-Shot Learners.

Buongiorno Nardelli, B., Cavaliere, D., Charles, E., & Ciani, D. (2022, January). Super-Resolving Ocean Dynamics from Space with Computer Vision Algorithms. *Remote Sensing*, 14(5), 1159. doi: 10.3390/rs14051159

Carrassi, A., Bocquet, M., Bertino, L., & Evensen, G. (2018). Data assimilation in the geosciences: An overview of methods, issues, and perspectives. *WIREs Climate Change*, 9(5), e535. doi: 10.1002/wcc.535

Carrere, L., Faugère, Y., & Ablain, M. (2016, June). Major improvement of altimetry sea level estimations using pressure-derived corrections based on ERA-Interim atmospheric reanalysis. *Ocean Science*, 12(3), 825–842. doi: 10.5194/os-12-825-2016

Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., & Fei-Fei, L. (2009, June). ImageNet: A large-scale hierarchical image database. In *2009 IEEE Conference on Computer Vision and Pattern Recognition* (pp. 248–255). doi: 10.1109/CVPR.2009.5206848

Dibarboure, G., Ubelmann, C., Flamant, B., Briol, F., Peral, E., Bracher, G., ... Picot, N. (2022, January). Data-Driven Calibration Algorithm and Pre-Launch Performance Simulations for the SWOT Mission. *Remote Sensing*, 14(23), 6070. doi: 10.3390/rs14236070

Dosovitskiy, A., Fischer, P., Ilg, E., Häusser, P., Hazirbas, C., Golkov, V., ... Brox, T. (2015, December). FlowNet: Learning Optical Flow with Convolutional Networks. In *2015 IEEE International Conference on Computer Vision (ICCV)* (pp. 2758–2766). doi: 10.1109/ICCV.2015.316

Fablet, R., Amar, M. M., Febvre, Q., Beauchamp, M., & Chapron, B. (2021, June). END-TO-END PHYSICS-INFORMED REPRESENTATION LEARNING FOR SATELLITE OCEAN REMOTE SENSING DATA: APPLICATIONS TO SATELLITE ALTIMETRY AND SEA SURFACE CURRENTS. *ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences*, V-3-2021, 295–302. doi: 10.5194/isprs-annals-V-3-2021-295-2021

Fablet, R., Chapron, B., Drumetz, L., Mémin, E., Pannekoucke, O., & Rousseau, F. (2021). Learning Variational Data Assimilation Models and Solvers. *Journal of Advances in Modeling Earth Systems*, 13(10), e2021MS002572. doi: 10.1029/2021MS002572

Fablet, R., Ouala, S., & Herzet, C. (2018, September). Bilinear Residual Neural Network for the Identification and Forecasting of Geophysical Dynamics. In *2018 26th European Signal Processing Conference (EUSIPCO)* (pp. 1477–1481). doi: 10.23919/EUSIPCO.2018.8553492

Febvre. (2023, June). Code and Data release Simulation-based 4dVarNets for satellite altimetry.
doi: 10.5281/zenodo.8119300

Gomez Gonzalez, C. A., Wertz, O., Absil, O., Christiaens, V., Defrère, D., Mawet, D., ... Surdej, J. (2017, July). VIP: Vortex Image Processing Package for High-contrast Direct Imaging. *The Astronomical Journal*, **154**, 7. doi: 10.3847/1538-3881/aa73d7

Greydanus, S., Dzamba, M., & Yosinski, J. (2019). Hamiltonian Neural Networks. In *Advances in Neural Information Processing Systems* (Vol. 32). Curran Associates, Inc.

Guillou, F. L., Metref, S., Cosme, E., Ubelmann, C., Ballarotta, M., Sommer, J. L., & Verron, J. (2021, April). Mapping Altimetry in the Forthcoming SWOT Era by Back-and-Forth Nudging a One-Layer Quasigeostrophic Model. *Journal of Atmospheric and Oceanic Technology*, **38**(4), 697-710. doi: 10.1175/JTECH-D-20-0104.1

Gurvan, M., Bourdallé-Badie, R., Chanut, J., Clementi, E., Coward, A., Ethé, C., ... Moulin, A. (2022, March). NEMO ocean engine. doi: 10.5281/zenodo.6334656

He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep Residual Learning for Image Recognition. In *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition* (pp. 770-778).

Hornik, K., Stinchcombe, M., & White, H. (1989, January). Multilayer feedforward networks are universal approximators. *Neural Networks*, **2**(5), 359-366. doi: 10.1016/0893-6080(89)90020-8

Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). ImageNet Classification with Deep Convolutional Neural Networks. In *Advances in Neural Information Processing Systems* (Vol. 25). Curran Associates, Inc.

LeCun, Y., Bengio, Y., & Hinton, G. (2015, May). Deep learning. *Nature*, **521**(7553), 436-444. doi: 10.1038/nature14539

Lellouche, J., Greiner, E., Bourdallé-Badie, R., Garric, G., Melet, A., Drévillon, M., ... Le Traon, P. (2021). The Copernicus Global 1/12° Oceanic and Sea Ice GLORYS12 Reanalysis. *Frontiers in Earth Science*, **9**.

Lguensat, R., Tandeo, P., Ailliot, P., Pulido, M., & Fablet, R. (2017, October). The Analog Data Assimilation. *Monthly Weather Review*, **145**(10), 4093-4107. doi: 10.1175/MWR-D-16-0441.1

Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., & Anandkumar, A. (2020). *Fourier neural operator for parametric partial differential equations*.

Martin, S. A., Manucharyan, G. E., & Klein, P. (2023). Synthesizing Sea Surface Temperature and Satellite Altimetry Observations Using Deep Learning Improves the Accuracy and Resolution of Gridded Sea Surface Height Anomalies. *Journal of Advances in Modeling Earth Systems*, **15**(5), e2022MS003589. doi: 10.1029/2022MS003589

Raissi, M., Perdikaris, P., & Karniadakis, G. E. (2019, February). Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. *Journal of Computational Physics*, **378**, 686-707. doi: 10.1016/j.jcp.2018.10.045

Ronneberger, O., Fischer, P., & Brox, T. (2015). U-Net: Convolutional Networks for Biomedical Image Segmentation. In N. Navab, J. Hornegger, W. M. Wells, & A. F. Frangi (Eds.), *Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015* (pp. 234-241). Cham: Springer International Publishing. doi: 10.1007/978-3-319-24574-4\_28

Shorten, C., & Khoshgoftaar, T. M. (2019, July). A survey on Image Data Augmentation for Deep Learning. *Journal of Big Data*, **6**(1), 60. doi: 10.1186/s40537-019-0197-0

Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: A Simple Way to Prevent Neural Networks from Overfitting.
ting. *Journal of Machine Learning Research*, 15(56), 1929–1958.

Taburet, G., Sanchez-Roman, A., Ballarotta, M., Pujol, M.-I., Legeais, J.-F., Fournier, F., ... Dibarboure, G. (2019, September). DUACS DT2018: 25 years of reprocessed sea level altimetry products. *Ocean Science*, 15(5), 1207–1224. doi: 10.5194/os-15-1207-2019

Ubelmann, C., Dibarboure, G., Gaultier, L., Ponte, A., Ardhuin, F., Ballarotta, M., & Faugère, Y. (2021). Reconstructing Ocean Surface Current Combining Altimetry and Future Spaceborne Doppler Data. *Journal of Geophysical Research: Oceans*, 126(3), e2020JC016560. doi: 10.1029/2020JC016560

Voldoire, A., Sanchez-Gomez, E., Salas y Mélia, D., Decharme, B., Cassou, C., Sénési, S., ... Chauvin, F. (2013, May). The CNRM-CM5.1 global climate model: Description and basic evaluation. *Climate Dynamics*, 40(9), 2091–2121. doi: 10.1007/s00382-011-1259-y

Wang, R., Kashinath, K., Mustafa, M., Albert, A., & Yu, R. (2020). Towards physics-informed deep learning for turbulent flow prediction. *Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining*.

Wong, A. P. S., Wijffels, S. E., Riser, S. C., Pouliquen, S., Hosoda, S., Roemmich, D., ... Park, H.-M. (2020). Argo Data 1999–2019: Two Million Temperature-Salinity Profiles and Subsurface Velocity Observations From a Global Array of Profiling Floats. *Frontiers in Marine Science*, 7.

Yin, Y., Guen, V. L., Dona, J., de Bézenac, E., Ayed, I., Thome, N., & Gallinari, P. (2021, December). Augmenting physical models with deep networks for complex dynamics forecasting*. *Journal of Statistical Mechanics: Theory and Experiment*, 2021(12), 124012. doi: 10.1088/1742-5468/ac3ae5