# A Combined Color and Wave-based Approach to Satellite Derived Bathymetry using Deep Learning

Mahmoud Al Najar, Youssra El Bennioui, Grégoire Thoumyre, Rafael Almar, Erwin W J Bergsma, Rachid Benshila, Jean-Marc Delvit, Dennis G Wilson

► To cite this version:

Mahmoud Al Najar, Youssra El Bennioui, Grégoire Thoumyre, Rafael Almar, Erwin W J Bergsma, et al.. A Combined Color and Wave-based Approach to Satellite Derived Bathymetry using Deep Learning. XXIV ISPRS Congress (2022), International Society for Photogrammetry and Remote Sensing, Jun 2022, Nice, France. pp.9-16, 10.5194/isprs-archives-XLIII-B3-2022-9-2022. [https://hal.science/hal-03736442v1](https://hal.science/hal-03736442v1)

HAL Id: hal-03736442

https://hal.science/hal-03736442v1

Submitted on 24 May 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# A COMBINED COLOR AND WAVE-BASED APPROACH TO SATELLITE DERIVED BATHYMETRY USING DEEP LEARNING

Mahmoud Al Najar<sup>a,c,d,*</sup>, Youssra El Benniouji<sup>b</sup>, Grégoire Thoumyre<sup>b</sup>, Rafael Almar<sup>b</sup>, Erwin W. J. Bergsma<sup>c</sup>, Rachid Benshila<sup>a</sup>,
Jean-Marc Delvit<sup>c</sup>, Dennis G. Wilson<sup>d</sup>

## Commission III, WG III/1

**KEY WORDS:** Bathymetry Estimation, Wave Kinematics, Convolutional Neural Networks, Sentinel-2, Remote Sensing.

## ABSTRACT:

Knowledge of the evolution of the littoral zone over time is paramount for coastal science and coastal zone management. However, traditional bathymetric surveys using echo-sounding techniques are unsuitable for large-scale applications due to a variety of constraints. On the other hand, remote sensing data such as satellite imagery allow for the development and application of bathymetry inversion models on a large scale. Deep learning is a growing field of artificial intelligence that allows for the automatic construction of models from data and has been successfully used for various Earth Observation and model inversion applications. In this work, we develop and apply a deep learning-based depth inversion model combining wave kinematics and water color information from Sentinel-2 satellite imagery. We present two different satellite image processing methods to augment wave kinematics and color information as inputs to the proposed deep learning-based models. We show competitive results with a state-of-the-art physical inversion method for satellite derived bathymetry, Satellite to Shores (S2Shores), demonstrating a promising direction for the use of deep learning models in Satellite Derived Bathymetry (SDB) and Earth observation in general.

## 1. INTRODUCTION

Coastal areas around the world are continuously evolving due to a variety of natural and anthropogenic pressures. Water depth (i.e. bathymetry) plays a major role in a variety of natural processes occuring in the coastal zone. Our knowledge of the evolution of coastal bathymetry is paramount for many applications such as coastal development, risk monitoring and management. Traditionally, bathymetric surveys are done using echo-sounding or Light Detection and Ranging techniques. However, these techniques are expensive, time-consuming and are constrained by different environmental and logistical factors (Jagalingam et al., 2015, Gao, 2009, Salameh et al., 2019). In recent years, remote sensing tools such as shore-based video cameras and space-born satellite constellations have emerged as important tools for the collection of large amounts of earth observation data. These data products have been used to study a wide array of natural processes in the coastal zone (Brando and Dekker, 2003, Yuan et al., 2020, Wei et al., 2021, Schratz et al., 2021, da Silveira et al., 2021). In the domain of bathymetry inversion from satellite imagery, two main methodologies have been studied and can be identified in the literature (Al Najar et al., 2022). These can be categorized based on the natural phenomena or process they exploit to invert water depth. Namely, the effect of bathymetry on the propagation and dispersion of surface waves (wave kinematics), as well as the relation between water depth and light penetration and reflectance in water (water color). Color-based methods can be used to estimate depth in optically shallow waters and are able to detect smaller-scale bathymetric features, with an absolute error order of 10–20% of the target value, and an average RMSE of 1.5 m (Pacheco et al., 2015, Chénier et al., 2018, Traganos

et al., 2018, Lyzenga, 1978, Giardino et al., 2012, Legleiter et al., 2009, Caballero and Stumpf, 2019, Evagorou et al., 2019, Sagawa et al., 2019). However, these are sensitive to the optical properties of seawater and are generally limited to clear and non-turbid waters (Cesbron et al., 2021, Almar et al., 2021). On the other hand, wave kinematics-based methods study the interaction between water depth and the propagation and dispersion of surface waves. Compared to color-based methods, wave kinematics-based methods are able to estimate bathymetry in significantly deeper areas, but with a larger error margin when applied globally (RMSE between 6–9 m, (Almar et al., 2021)). The development of a depth estimation method applicable to satellite data is non-trivial and remains a topic of ongoing research due to the great potential it offers to in-expensively monitor coastal morphodynamics at a large scale. Machine learning (ML) algorithms are a family of data-driven optimization techniques that have shown promising performance in a wide array of application domains. ML algorithms which have been used in the past to estimate depth from satellite imagery include multi-layered perceptrons, support vector machines and random forests (Sandidge and Holyer, 1998, Vojinovic et al., 2013, Sagawa et al., 2019). Recently, modern Deep Learning (DL) algorithms are being used more frequently in Earth observation and remote sensing applications (Iglovikov et al., 2017, Zhu et al., 2017, Hoeser and Kuenzer, 2020, Ma et al., 2019), including bathymetry estimation (Ghorbanidehno et al., 2021, Collins et al., 2021, Mandlburger et al., 2021). In the context of satellite derived bathymetry, DL-based works in the literature mostly explore the color-based approach, example studies include (Dickens and Armstrong, 2019, Wilson et al., 2020, Lumban-Gaol et al., 2021). On the other hand, great expectations come from the combination of different methods, in particular, based on wave information (Danilo and Melgani, 2016, Benshila et al., 2020,

<footnote><sup>a</sup>CNRS, LEGOS, UMR-5566, 14 Avenue Edouard Belin, 31400 Toulouse, France; mahmoud.al.najar@legos.obs-mip.fr</footnote>

<footnote><sup>b</sup>IRD, LEGOS, UMR-5566, 14 Avenue Edouard Belin, 31400 Toulouse, France</footnote>

<footnote><sup>c</sup>CNES, Earth Observation Laboratory, 18 Avenue Edouard Belin, 31400 Toulouse, France</footnote>

<footnote><sup>d</sup>ISAE-SUPAERO, 10 Avenue Edouard Belin, 31055 Toulouse, France</footnote>

<footnote>* Corresponding author</footnote>
Al Najar et al., 2021, Al Najar et al., 2022). In this work, we develop on our Deep Single-Point Estimation of Bathymetry method (DSPEB) as presented in our previous works (Al Najar et al., 2021, Al Najar et al., 2022). Specifically, we train and apply the DSPEB method using a number of Sentinel-2 L2A images coupled with bathymetry surveys obtained from the French Naval Hydrographic and Oceanographic Service. We present two different satellite image pre-processing techniques to augment wave kinematics and color information as inputs to two DSPEB models. In addition, as a continuation of our preliminary work on hybrid models in (Al Najar et al., 2022), we investigate a multi-headed convolutional neural network architecture (CNN) named Hybrid-DSPEB (H-DSPEB) with the aim of incorporating both color and wave kinematics information into a multi-input single-output model. We present the performances of the three DSPEB models and we compare them to a state-of-the-art bathymetry inversion models based on wave kinematics, Satellite to Shores (S2Shores) (Bergsma et al., 2021). The layout of this article is as follows. In Section 2 we present our satellite image preprocessing chain to augment wave kinematics and water color information, our dataset creation methodology, in addition to presenting the three methods studied (DSPEB, H-DSPEB, S2Shores). Then, Section 3 documents the different experiments done during the development of the H-DSPEB model and presents a performance comparison of the different models studied. Finally, we present a brief discussion of the results and we conclude this work in Section 4.

## 2. METHODS

### 2.1 Sentinel-2 Data Pre-Processing

As mentioned in Section 1, we make a distinction between two types of information in satellite imagery that can be used for bathymetry estimation. Namely, information relating to the light radiance and reflection in the water column, as well as the propagation of surface waves in wave kinematics-based SDB. In this work, we explore a hybrid approach to SDB, named H-DSPEB, that combines both types of information. After the image date selection based on the performance of S2Shores (we refer to (Al Najar et al., 2022) for further details on date selection), our preprocessing workflow is focused on detecting and separating the signals representing actual ocean waves from the remaining information found in our Sentinel-2 subtiltes. For each subtile, a pass-band filter in the spectral domain is applied to retain signals corresponding the range of ocean-specific wavelengths (periods $T_{min} = 5$ s to $T_{max} = 25$ s).

We apply a pass-band filter to our input subtiltes in the range of ocean-specific wavelengths (periods $T_{min} = 5$ s to $T_{max} = 25$ s). First, we create a frequency filter based on $T_{min}$ and $T_{max}$. Then, a Discrete Fourier Transform (DFT) is applied in two dimensions to each band (layer) in the input subtile. For wave kinematics-based DSPEB (W-DSPEB), we further process our filtered subtiltes by calculating the two-dimensional normalized cross-correlation (NORMXCORR) of each band, in order to extract the most consistent and recurring wave signals corresponding to the crests of actual ocean waves. For C-DSPEB, we subtract the detected ocean-wave signals from the raw input image in order to retain the remainder of the information, most importantly the background color. Figure 1 demonstrates our pre-processing workflow on an example 400 × 400 m Sentinel-2 subtile. For C-DSPEB, we scale all input images such that the minimum and maximum pixel values over the full dataset are equal to -0.9 and 0.9, respectively.

<figure><img src="image_2.png" /><figcaption>Figure 1. Pre-processing of a single 400 × 400 m Sentinel-2 subtile band (10 m resolution). (a) Raw subtile. (b) Passband-filtered subtile. (c) Final NORMXCORR result used as input to W-DSPEB. (d) Color-augmented subtile.</figcaption></figure>

### 2.2 Dataset Creation

For the present study, the methods are tested and compared on the coast of French Guiana. Bathymetry data from the French Naval Hydrographic and Oceanographic Service have been coupled with a set of Sentinel-2 L2A images in order to create a supervised dataset that can be used to train the deep learning models. Figure 2 shows the location and distribution of depth points used in this study.

<figure><img src="image_3.png" /><figcaption>Figure 2. Water depth distribution of the bathymetry survey obtained for French Guiana from the French Naval Hydrographic and Oceanographic Service. (a) Spatial distribution of the survey used. (b) Distribution of depth points retained in the dataset.</figcaption></figure>
To create the supervised dataset, we first discard all points deeper than 40 meters from the original bathymetry survey. Then, for each depth point, we extract the surrounding 400 × 400 meter subtile from each of the selected Sentinel-2 images, such as the depth point is at the center of each of the extracted subtiles. These subtiles are first preprocessed according to the method described in Section 2.1, then recorded on disk for model training. The recorded data points are randomly shuffled and split into training and validation sets. The exact same process, but using a different set of Sentinel-2 images, is then applied to create a test set for model performance evaluation and comparison.

## 2.3 Deep Single-Point Estimation of Bathymetry

Deep learning is a subfield of machine learning dealing with (deep) artificial neural networks that has shown impressive performance in a wide array of applications. Training such networks is done using Stochastic Gradient Descent (SGD) and the backpropagation algorithm, where a network’s prediction error over a batch of samples is calculated according to an objective function and then propagated backward through the different layers of the network in order to update the network parameters. This process is repeated over multiple iterations of the available data and is stopped according to varying criteria. As part of the optimization process, a learning rate is employed to control the scale of parameter updates that are done at each step. The Deep Single-Point Estimation of Bathymetry method (DSPEB) (Al Najar et al., 2021) is a bathymetry inversion algorithm that makes use of ResNet (He et al., 2016) as its neural architecture; it operates on 40 × 40 × 4 px multi-spectral input subtiles, corresponding to the red, green, blue and near-infrared (RGB-NIR) bands of the Sentinel-2 satellite constellation at 10 meters resolution (Drusch et al., 2012). DSPEB approximates a single depth point per input subtile, corresponding to water depth at the center of the input. A simple moving window technique can then be used to map the bathymetry of a complete area. A comprehensive description of the DSPEB method and its development can be found in our previous works (Al Najar et al., 2022, Al Najar et al., 2021). In this work, DSPEB is used with two different input subtile pre-processing schemes augmenting wave kinematics and color information to form two DSPEB-based models for SDB. Namely, wave kinematics-based DSPEB (W-DSPEB) and color-based DSPEB (C-DSPEB). These models are then used to train a Hybrid model that makes use of both types of inputs as presented in Section 2.4.

## 2.4 Hybrid-DSPEB Model

Previous work in deep learning for computer vision has proposed multi-input convolutional neural networks to improve performance on tasks where different views of the same input are useful for approximating a single output. This can be done by grouping multiple neural networks, or duplicating a single network architecture, through an MLP-like architecture near the output of the merged network (Dua et al., 2021, Oktay et al., 2016, Cheng et al., 2016). In this work, we propose H-DSPEB as a variant of DSPEB which merges both of C-DSPEB and W-DSPEB into a single neural architecture. The architecture of H-DSPEB can be seen in Figure 3.

The MLP head which we append to the end of the two pre-trained models is composed of two new fully connected layers which connect to the last hidden layer of each sub-model, in addition to the output layer of each of the sub-models. The single

<figure><img src="image_4.png" /><figcaption>Figure 3. The neural architecture of Hybrid-DSPEB combining the pretrained C-DSPEB and W-DSPEB models. The blue arrows represent non-parametric connections (flattening layer). Dotted lines represent pre-trained layers that are frozen during H-DSPEB training.</figcaption></figure>

output of the MLP head corresponds to the final output of H-DSPEB. As presented in Section 3, we tested various network head architectures however little difference was noted in their final performances. During training, all pre-trained weights from C-DSPEB and W-DSPEB were frozen, as indicated by the dotted lines in Figure 3.

The aim of this architecture is to evaluate whether color and celerity information can be automatically combined for enhanced estimation, due to the different conditions in which these two model types function. By including higher-level features from the final layer, our goal is that the hybrid model learns to estimate depth using both color and celerity information. Because the two approaches are complementary for clear and turbid waters, contrary to previous deep learning bathymetry inversion applications, the combination of the two unlocks the potential of inversion of the bathymetry from a satellite at any coast worldwide.

## 2.5 Satellite to Shores

Satellite to Shores (S2Shores) (Bergsma et al., 2021) is a wave kinematics-based bathymetry inversion model that uses a combined radon transform (RT) and a discrete Fourier transform (DFT) to detect spectral wave characteristics such as the spectral wave phase shift and the wave number to invert water depth using the linear dispersion relation for free surface waves (Equation (2)). The RT is first used to produce a sinogram of integrated pixel-intensities per direction. A 1D DFT procedure per direction over the sinogram enables the transformation to a complex spectral representation of the observed wave signals in polar space. The wave phase and amplitude per wave number and per direction can then be determined. The difference in phase (ΔΦ) can be found between (several) pairs of detector bands. Presuming that the wavenumber (k) is constant or near-constant over the sub-window, ΔΦ can be seen as representative of ω(t), and given that the timing between the different detector bands (Δt) is constant, the wave celerity (c) can be determined according to Equation (1). For each wavenumber or celerity pair, Equation (2) can then be solved for depth.

$$c = \frac{\Delta\Phi}{2\pi k \Delta t} = \frac{\Delta\Phi\lambda}{2\pi \Delta t} \quad (1)$$
$$c^2 = \frac{g}{h} \tanh(kh) \iff h = \frac{\tanh^{-1}\left(1\left(\frac{c^2k}{g}\right)\right)}{k} \quad (2)$$

# 3. EXPERIMENTS & RESULTS

## 3.1 Core DSPEB models

We train the wave kinematics-based and color-based DSPEB models using the Adam optimizer (Kingma and Ba, 2014), with mean squared error (MSE) loss and a batch size of 256. Adam's hyperparameters including $lr$, $\epsilon$, $\beta_1$, and $\beta_2$ were respectively set to $1 \times 10^{-4}$, $1 \times 10^{-8}$, 0.99, and 0.999 for W-DSPEB, and to $1 \times 10^{-3}$, $1 \times 10^{-6}$, 0.5, and 0.9 for C-DSPEB. To stop the training, an automatic early stopping mechanism (patience) is used that stops training if no improvement in performance on the validation set is achieved for 10 consecutive epochs. The learning curves of the trained models are presented in Figure 4, showing the models' errors on the training and validation sets at each training step.

<figure><img src="image_5.png" /><figcaption>Figure 4. Training of the W-DSPEB and C-DSPEB models, showing the MSE losses over the training and validation sets in French Guiana. Dashed lines represent performance on the validation set.</figcaption></figure>

Figure 4 shows the MSE training and validation losses of W-DSPEB and C-DSPEB on the French Guiana study site and demonstrates the difference in convergence speed between the two models due to the higher learning rate used for C-DSPEB. We note that W-DSPEB was unable to converge using larger learning rates. The training of the W-DSPEB and C-DSPEB models stopped at 53 and 30 epochs respectively, achieving test RMSE scores of 4.2 and 5.8 meters.

## 3.2 Hybrid-DSPEB Architectures

In order to create the H-DSPEB model, we study five architecture variants of the neural network's output head which makes use of the C-DSPEB and W-DSPEB models in its architecture in order to produce a single depth estimation. This is done by merging the last fully connected layers and output nodes of the two pretrained C-DSPEB and W-DSPEB models into a single output node. The architectures tested are shown in Figure 5. The learning curves of the combined architectures can be seen in Figure 6. Then, Table 1 presents a comparison of the different candidate architectures according to the root-mean-squared-error (RMSE) and the Pearson correlation coefficient.

Figure 5 presents the different neural architectures of the combined network's head. The main goal behind the different architectures in this experiment is to test different mappings and output paths, while enabling the final output node to make use

<figure><img src="image_6.png" /><figcaption>Figure 5. Combined neural network architectures using an MLP-like head. (V0) Initial W-DSPEB and C-DSPEB output layers. (V1–V5) Candidate combination schemes forming a multi-input, single-output neural architecture (H-DSPEB). Dotted connections represent connections with pre-trained weights from C-DSPEB or W-DSPEB. The numbers represent the size of the latent feature space at each layer.</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>Figure 6. The learning curves of the different H-DSPEB architectures used in the study.</figcaption></figure>

of the last fully connected layers and the output nodes of each of C-DSPEB and W-DSPEB.

All H-DSPEB architectures shown in Figure 5 were trained using the same training configuration used for W-DSPEB. Specifically, the models are trained with mean squared error (MSE) loss and a batch size of 256, and the learning hyperparameters $lr$, $\epsilon$, $\beta_1$, and $\beta_2$ were respectively set to $1 \times 10^{-4}$, $1 \times 10^{-8}$, 0.99, and 0.999. Interestingly, using fixed hyperparameters, we note no significant difference in performance between architecture variants 2-5 (Figure 6). On the other hand, variant 1 goes through a significantly larger number of training steps before early stopping due to patience, but achieves similar performance after training.
<table><caption>Table 1. Performance comparison of the different candidate H-DSPEB architectures.</caption><thead><tr><td>Variant</td><td>RMSE [m]</td><td>r</td></tr></thead><tbody><tr><td>V1</td><td>3.57</td><td>0.9</td></tr><tr><td>V2</td><td>3.47</td><td>0.91</td></tr><tr><td>V3</td><td>3.81</td><td>0.9</td></tr><tr><td>V4</td><td>3.86</td><td>0.89</td></tr><tr><td>V5</td><td>3.6</td><td>0.91</td></tr></tbody></table>

Table 1 compares the different architectures presented in Figure 5 according to each model’s RMSE and correlation (r) score on the test set. All models are able to achieve a correlation of ≈ 0.9 and an RMSE score in the range ≈ [3.5, 3.8] meters. For the remainder of this study, architecture variant (V2) has been used to evaluate and test the H-DSPEB model due to its superior correlation and lower RMSE score.

### 3.3 Hybrid-DSPEB Input Masking

In this experiment, parts of the inputs to the H-DSPEB model were masked in order to better understand the model’s reliance on its two different types of inputs (color and wave information). To mask part of the input, the wave kinematics-augmented inputs to H-DSPEB were replaced with zeros (de-activated), such that the model would have access to the color inputs only during the forward pass. Then, the reverse was applied, where the color inputs were masked and the model was applied using the wave inputs only. Table 2 presents the results of this experiment.

<table><caption>Table 2. Performance comparison of the H-DSPEB architecture after masking parts of the inputs.</caption><thead><tr><td>Mask</td><td>RMSE [m]</td><td>r</td></tr></thead><tbody><tr><td>Masked color</td><td>14.32</td><td>0.88</td></tr><tr><td>Masked waves</td><td>9.23</td><td>0.88</td></tr><tr><td>No mask</td><td>3.47</td><td>0.91</td></tr></tbody></table>

When masking parts of the input, the model’s performance deteriorates significantly, achieving an RMSE score of 14 and 9 meters when hiding the color inputs and the wave inputs respectively. We note that the majority of the difference is in terms of RMSE; the correlation score does deteriorate though not as drastically. This result shows that H-DSPEB relies on both types of inputs and submodels in order to construct its final estimations. In the following subsection, the performance of the full H-DSPEB model is compared to S2Shores as well as its original submodels C-DSPEB and W-DSPEB.

### 3.4 Performance Comparison

This section presents a full comparison of the models included in this study. Namely, S2Shores, W-DSPEB, C-DSPEB and H-DSPEB. Table 3 first compares the different models numerically by considering the Pearson correlation coefficient and the RMSE score over the test set. Then, a visual evaluation of the estimated profiles is presented in terms of correlation (Figure 7) and absolute error (Figure 8).

Table 3 compares H-DSPEB to S2Shores, W-DSPEB and C-DSPEB according to their correlation and RMSE scores. A comparison of H-DSPEB to C-DSPEB shows the ability of H-DSPEB to make use of its wave kinematics sub-model to correct the estimations of C-DSPEB.

Figures 7 and 8 present a visual comparison of H-DSPEB to S2Shores, W-DSPEB and C-DSPEB. Compared to the wave

<table><caption>Table 3. Comparison of the performance of H-DSPEB to W-DSPEB, C-DSPEB and S2Shores.</caption><thead><tr><td>Model</td><td>RMSE [m]</td><td>r</td></tr></thead><tbody><tr><td>S2Shores</td><td>3.63</td><td>0.93</td></tr><tr><td>W-DSPEB</td><td>3.26</td><td>0.91</td></tr><tr><td>C-DSPEB</td><td>3.96</td><td>0.89</td></tr><tr><td>H-DSPEB</td><td>3.47</td><td>0.91</td></tr></tbody></table>

<figure><img src="image_8.png" /><figcaption>Figure 7. Pearson correlation score comparison of S2Shores (a), W-DSPEB (b), C-DSPEB(c) and H-DSPEB (d).</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>Figure 8. RMSE score comparison of S2Shores (a), W-DSPEB (b), C-DSPEB(c) and H-DSPEB (d).</figcaption></figure>

kinematics-based methods, C-DSPEB and H-DSPEB show clear sensitivity to the effect of turbidity on background color and the error of the resulting model estimates. A comparison of the distributions of absolute errors between H-DSPEB and C-DSPEB suggests that, surprisingly, H-DSPEB is strongly biased towards the color-based section of the model. This result is counterintuitive due to the nature of the site chosen for this study, French Guiana, which normally is not suitable for color-based approaches due to high turbidity (Figure 9).
<figure><img src="image_10.png" /><figcaption>Figure 9. Example image of the test area in French Guiana showing the edge of high water turbidity.</figcaption></figure>

## 4. DISCUSSION & CONCLUSION

In this work, we have shown that deep learning can be used for SDB by showcasing the ability of the DSPEB method to estimate bathymetry using two different satellite image pre-processing schemes representing two intrinsically-different approaches to SDB. This work also presented the H-DSPEB model, a deep learning-based hybrid approach to SDB making use of both wave kinematics and color information. The DSPEB models are shown to be competitive with a state-of-the-art physics-based SDB method, S2Shores, achieving RMSE performance of 3–4 m over areas reaching 30 m depths.

While our results show that the different variants of DSPEB are competitive deep learning-based methods for SDB, there are limitations to the current workflow of DSPEB. Specifically, we highlight the current limitations of satellite image pre-selection based on dates and the requirement of local training in application sites.

The accuracies of satellite derived bathymetry methods that make use of optical satellite products are constrained by the practical limitations of these products including cloud cover for example. In this work, the models were found to be sensitive to the dates of the satellite images selected, especially in the case of W-DSPEB and S2Shores, due to their high reliance on the observability of waves in the input images. Currently, S2Shores is used to pre-select the dates where the wave conditions are suitable for the W-DSPEB submodel. A possible solution to this limitation could be the use of a separate CNN as a classifier to dictate whether an image contains useful wave kinematic information for estimating water depth. Such a model would greatly minimize the amount of computing power required for date selection, in addition to potentially providing insight into this issue. On the other hand, we believe that further development of the H-DSPEB model, although non-trivial, could help in resolving this issue due to it having access to both types of information used to estimate water depth. Moreover, the emergence of modern agile satellite constellations such as the Airbus CO3D constellation, currently being developed for the French Space Agency (CNES), are expected to mitigate some of these limitations (Turner et al., 2021).

Another limitation of the DSPEB workflow is that it currently requires local training at each application site prior to model deployment, limiting the usability of the model to sites with existing bathymetry survey data. A possible path for future development is the use of models pre-trained on different sites and fine-tuning them to previously unsees application sites, minimizing the computational resources required for model training and possibly minimizing model overfitting to any site-specific characteristics. Applying a pre-trained deep learning-based SDB

model to a site for which no survey data is available is another path for future work requiring further development in zero-shot learning (Xian et al., 2017).

Our experiments with H-DSPEB demonstrate that inputs which represent different information can be combined for bathymetry inversion, even if there is a strong tendency towards reliance on color information in the studied cases. While we separate the information into color and wave kinematics from the same satellite source, Sentinel-2, other works have shown that the fusion of different satellite products can be beneficial in estimating high resolution bathymetry maps (Le Quilleuc et al., 2021). As such, another direction for H-DSPEB would be to use different satellite products, for example Pleiades as used in (Almar et al., 2019), and combine them in similar architectures to those presented in 3.2.

This work presented first steps towards a generalized bathymetry estimation model that combines wave and color information to achieve local estimates with high accuracy. This is desirable as areas with different characteristics are more suited for bathymetry estimation through wave kinematics, such as turbid areas, while others are more suited for color-based methods, such as areas with low wave activity. The results from our experiments show that the current hybrid DSPEB model relies heavily on the color-based submodel C-DSPEB, but we strongly believe that a combined model incorporating both types of methods (color-based and wave kinematics-based methods) is the way forward to unlock the applicability of SDB to a global scale covering all types of coastal environments. While the H-DSPEB model is a deep learning architecture-based attempt at engineering such a model, other possibilities exist, such as traditional or deep learning-based data assimilation techniques (Arcucci et al., 2021). H-DSPEB represents a first step in this direction of trained bathymetry inversion models which incorporate both color information and wave kinematics.

## REFERENCES

Al Najar, M., El Bennioui, Y., Thoumyre, G., Almar, R., Bergsma, E. W. J., Benshila, R., Delvit, J.-M., Wilson, D. G., 2022. Coastal Bathymetry Estimation from Sentinel-2 Satellite Imagery: Comparing Deep Learning and Physics-Based Approaches. *Remote Sensing*, 14(5). https://www.mdpi.com/2072-4292/14/5/1196.

Al Najar, M., Thoumyre, G., Bergsma, E. W. J., Almar, R., Benshila, R., Wilson, D. G., 2021. Satellite derived bathymetry using deep learning. *Machine Learning*. https://doi.org/10.1007/s10994-021-05977-w.

Almar, R., Bergsma, E. W. J., Thoumyre, G., Baba, M. W., Cesbron, G., Daly, C., Garlan, T., Liffermann, A., 2021. Global Satellite-Based Coastal Bathymetry from Waves. *Remote Sensing*, 13(22). https://www.mdpi.com/2072-4292/13/22/4628.

Almar, R., Bergsma, E. W., Maisongrande, P., de Almeida, L. P. M., 2019. Wave-derived coastal bathymetry from satellite video imagery: A showcase with Pleiades persistent mode. *Remote Sensing of Environment*, 231, 111263.

Arcucci, R., Zhu, J., Hu, S., Guo, Y.-K., 2021. Deep data assimilation: integrating deep learning with data assimilation. *Applied Sciences*, 11(3), 1114.
Benshila, R., Thoumyre, G., Al Najar, M., Abessolo, G., Almar, R., Bergsma, E., Hugonnard, G., Labracherie, L., Lavie, B., Ragonneau, T. et al., 2020. A deep learning approach for estimation of the nearshore bathymetry. *Journal of Coastal Research*, 95(SI), 1011–1015.

Bergsma, E. W., Almar, R., Rolland, A., Binet, R., Brodie, K. L., Bak, A. S., 2021. Coastal morphology from space: A showcase of monitoring the topography-bathymetry continuum. *Remote Sensing of Environment*, 261, 112469. https://www.sciencedirect.com/science/article/pii/S0034425721001875.

Brando, V., Dekker, A., 2003. Satellite hyperspectral remote sensing for estimating estuarine and coastal water quality. *IEEE Transactions on Geoscience and Remote Sensing*, 41(6), 1378-1387.

Caballero, I., Stumpf, R., 2019. Retrieval of nearshore bathymetry from Sentinel-2A and 2B satellites in South Florida coastal waters. *Estuarine, Coastal and Shelf Science*, 226, 106277.

Cesbron, G., Melet, A., Almar, R., Lifermann, A., Tullot, D., Crosnier, L., 2021. Pan-European Satellite-Derived Coastal Bathymetry—Review, User Needs and Future Services. *Frontiers in Marine Science*, 8, 1591. https://www.frontiersin.org/article/10.3389/fmars.2021.740830.

Cheng, D., Gong, Y., Zhou, S., Wang, J., Zheng, N., 2016. Person re-identification by multi-channel parts-based cnn with improved triplet loss function. *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*.

Chénier, R., Faucher, M.-A., Ahola, R., 2018. Satellite-Derived Bathymetry for Improving Canadian Hydrographic Service Charts. *International Journal of Geo-Information*, 7(306), 15 pp.

Collins, A. M., Geheran, M. P., Hesser, T. J., Bak, A. S., Brodie, K. L., Farthing, M. W., 2021. Development of a Fully Convolutional Neural Network to Derive Surf-Zone Bathymetry from Close-Range Imagery of Waves in Duck, NC. *Remote Sensing*, 13(23), 4907.

da Silveira, C. B. L., Strenzel, G. M. R., Maida, M., Gaspar, A. L. B., Ferreira, B. P., 2021. Coral Reef Mapping with Remote Sensing and Machine Learning: A Nurture and Nature Analysis in Marine Protected Areas. *Remote Sensing*, 13(15). https://www.mdpi.com/2072-4292/13/15/2907.

Danilo, C., Melgani, F., 2016. Wave period and coastal bathymetry using wave propagation on optical images. *IEEE Transactions on Geoscience and Remote Sensing*, 54(11), 6307–6319.

Dickens, K., Armstrong, A., 2019. Application of Machine Learning in Satellite Derived Bathymetry and Coastline Detection. *SMU Data Science Review*; 2. https://scholar.smu.edu/datasciencereview/vol2/iss1/4.

Drusch, M., Bello, U. D., Carlier, S., Colin, O., Fernandez, V., Gascon, F., Hoersch, B., Isola, C., Laberinti, P., Martimort, P., Meygret, A., Spoto, F., Sy, O., Marchese, F., Bargellini, P., 2012. Sentinel-2: ESA's Optical High-Resolution Mission for GMES Operational Services. *Remote Sensing of Environment*, 120, 25 - 36. http://www.sciencedirect.com/science/article/pii/S0034425712000636.

Dua, N., Singh, S. N., Semwal, V. B., 2021. Multi-input CNN-GRU based human activity recognition using wearable sensors. *Computing*, 1–18.

Evagorou, E., Mettas, C., Agapiou, A., Themistocleous, K., Hadjimitsis, D., 2019. Bathymetric maps from multi-temporal analysis of Sentinel-2 data: the case study of Limassol, Cyprus. *Advances in Geosciences*, 45, 397–407. https://adgeo.copernicus.org/articles/45/397/2019/.

Gao, J., 2009. Bathymetric mapping by means of remote sensing: methods, accuracy and limitations. *Progress in Physical Geography*, 33, 103 - 116.

Ghorbanidehno, H., Lee, J., Farthing, M., Hesser, T., Darve, E. F., Kitanidis, P. K., 2021. Deep learning technique for fast inference of large-scale riverine bathymetry. *Advances in Water Resources*, 147, 103715.

Giardino, C., Candiani, G., Bresciani, M., Lee, Z., Gagliano, S., Pepe, M., 2012. BOMBER: A tool for estimating water quality and bottom properties from remote sensing images. *Computers & Geosciences*, 45, 313-318. https://www.sciencedirect.com/science/article/pii/S0098300411004158.

He, K., Zhang, X., Ren, S., Sun, J., 2016. Deep Residual Learning for Image Recognition. *2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*, 770-778.

Hoeser, T., Kuenzer, C., 2020. Object Detection and Image Segmentation with Deep Learning on Earth Observation Data: A Review-Part I: Evolution and Recent Trends. *Remote Sensing*, 12(10), 1667.

Iglovikov, V., Mushinskiy, S., Osin, V., 2017. Satellite imagery feature detection using deep convolutional neural network: A kaggle competition. *arXiv preprint arXiv:1706.06169*.

Jagalingam, P., Akshaya, B., Hegde, A. V., 2015. Bathymetry Mapping Using Landsat 8 Satellite Imagery. *Procedia Engineering*, 116, 560-566. https://www.sciencedirect.com/science/article/pii/S1877705815019815. 8th International Conference on Asian and Pacific Coasts (APAC 2015).

Kingma, D. P., Ba, J., 2014. Adam: A method for stochastic optimization.

Le Quilleuc, A., Collin, A., Jasinski, M. F., Devillers, R., 2021. Very high-resolution satellite-derived bathymetry and habitat mapping using pleiades-1 and ICESat-2. *Remote Sensing*, 14(1), 133.

Legleiter, C. J., Roberts, D. A., Lawrence, R. L., 2009. Spectrally based remote sensing of river bathymetry. *Earth Surface Processes and Landforms*, 34(8), 1039-1059. https://onlinelibrary.wiley.com/doi/abs/10.1002/esp.1787.

Lumban-Gaol, Y., Ohori, K., Peters, R., 2021. Satellite-derived bathymetry using convolutional neural networks and multispectral sentinel-2 images. *Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci*, 43, 201–207.

Lyzenga, D. R., 1978. Passive remote sensing techniques for mapping water depth and bottom features. *Appl. Opt.*, 17(3), 379–383. http://ao.osa.org/abstract.cfm?URI=ao-17-3-379.
Ma, L., Liu, Y., Zhang, X., Ye, Y., Yin, G., Johnson, B. A., 2019. Deep learning in remote sensing applications: A meta-analysis and review. *ISPRS journal of photogrammetry and remote sensing*, 152, 166–177.

Mandlburger, G., Kölle, M., Nübel, H., Soergel, U., 2021. BathyNet: A Deep Neural Network for Water Depth Mapping from Multispectral Aerial Images. *PFG–Journal of Photogrammetry, Remote Sensing and Geoinformation Science*, 1–19.

Oktay, O., Bai, W., Lee, M., Guerrero, R., Kamnitsas, K., Caballero, J., de Marvao, A., Cook, S., O’Regan, D., Rueckert, D., 2016. Multi-input cardiac image super-resolution using convolutional neural networks. S. Ourselin, L. Joskowicz, M. R. Sabuncu, G. Unal, W. Wells (eds), *Medical Image Computing and Computer-Assisted Intervention - MICCAI 2016*, Springer International Publishing, Cham, 246–254.

Pacheco, A., Horta, J., Loureiro, C., Ferreira, O., 2015. Retrieval of nearshore bathymetry from Landsat 8 images: A tool for coastal monitoring in shallow waters. *Remote Sensing of Environment*, 159, 102–116. https://www.sciencedirect.com/science/article/pii/S0034425714004878.

Sagawa, T., Yamashita, Y., Okumura, T., Yamanokuchi, T., 2019. Satellite Derived Bathymetry Using Machine Learning and Multi-Temporal Satellite Images. *Remote Sensing*, 11, 1155.

Salameh, E., Frappart, F., Almar, R., Baptista, P., Heygster, G., Lubac, B., Raucoules, D., Almeida, L. P., Bergsma, E. W. J., Capo, S., De Michele, M., Idier, D., Li, Z., Marieu, V., Poupardin, A., Silva, P. A., Turki, I., Laignel, B., 2019. Monitoring Beach Topography and Nearshore Bathymetry Using Spaceborne Remote Sensing: A Review. *Remote Sensing*, 11(19). https://www.mdpi.com/2072-4292/11/19/2212.

Sandidge, J. C., Holyer, R. J., 1998. Coastal Bathymetry from Hyperspectral Observations of Water Radiance. *Remote Sensing of Environment*, 65(3), 341–352. http://www.sciencedirect.com/science/article/pii/S0034425798000431.

Schratz, P., Muenchow, J., Iturritxa, E., Cortés, J., Bischl, B., Brenning, A., 2021. Monitoring Forest Health Using Hyperspectral Imagery: Does Feature Selection Improve the Performance of Machine-Learning Techniques? *Remote Sensing*, 13(23). https://www.mdpi.com/2072-4292/13/23/4832.

Traganos, D., Poursanidis, D., Aggarwal, B., Chrysoulakis, N., Reinartz, P., 2018. Estimating Satellite-Derived Bathymetry (SDB) with the Google Earth Engine and Sentinel-2. *Remote Sensing*, 10(6). https://www.mdpi.com/2072-4292/10/6/859.

Turner, I. L., Harley, M. D., Almar, R., Bergsma, E. W., 2021. Satellite optical imagery in Coastal Engineering. *Coastal Engineering*, 167, 103919. https://www.sciencedirect.com/science/article/pii/S037838392100079X.

Vojinovic, Z., Abebe, Y., Ranasinghe, R., Vacher, A., Martens, P., Mandl, D., Frye, S., Van Ettinger, E., De Zeeuw, R., 2013. A Machine Learning Approach for Estimation of Shallow Water Depths from Optical Satellite Images and Sonar Measurements. *Journal of Hydroinformatics*, 15(4), 1408–1424.

Wei, Z., Jia, K., Liu, P., Jia, X., Xie, Y., Jiang, Z., 2021. Large-Scale River Mapping Using Contrastive Learning and Multi-Source Satellite Imagery. *Remote Sensing*, 13(15). https://www.mdpi.com/2072-4292/13/15/2893.

Wilson, B., Kurian, N. C., Singh, A., Sethi, A., 2020. Satellite-derived bathymetry using deep convolutional neural network. *IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium*, IEEE, 2280–2283.

Xian, Y., Schiele, B., Akata, Z., 2017. Zero-shot learning-the good, the bad and the ugly. *Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition*, 4582–4591.

Yuan, Q., Shen, H., Li, T., Li, Z., Li, S., Jiang, Y., Xu, H., Tan, W., Yang, Q., Wang, J., Gao, J., Zhang, L., 2020. Deep learning in environmental remote sensing: Achievements and challenges. *Remote Sensing of Environment*, 241, 111716. https://www.sciencedirect.com/science/article/pii/S0034425720300857.

Zhu, X. X., Tuia, D., Mou, L., Xia, G.-S., Zhang, L., Xu, F., Fraundorfer, F., 2017. Deep learning in remote sensing: A comprehensive review and list of resources. *IEEE Geoscience and Remote Sensing Magazine*, 5(4), 8–36.