# Estimation of a motion field on satellite images from a simplified ocean circulation model

Isabelle Herlin, Etienne Huot, Jean-Paul Berroir, François-Xavier Le Dimet, Gennady K. Korotaev

► To cite this version:

Isabelle Herlin, Etienne Huot, Jean-Paul Berroir, François-Xavier Le Dimet, Gennady K. Korotaev. Estimation of a motion field on satellite images from a simplified ocean circulation model. ICIP 2006 International Conference on Image Processing, IEEE, Oct 2006, Atlanta, United States. pp.1077-1080, 10.1109/ICIP.2006.312742. inria-00604618

HAL Id: inria-00604618

https://inria.hal.science/inria-00604618

Submitted on 26 Feb 2021

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
ESTIMATION OF A MOTION FIELD ON SATELLITE IMAGES FROM A SIMPLIFIED
OCEAN CIRCULATION MODEL

I. Herlin, E. Huot, J.P. Berroir, F.X. Le Dimet

G. Korotaev

INRIA
Projects Clime and IDOPT
BP105-78153 Le Chesnay Cedex- FRANCE
Isabelle.Herlin@inria.fr

MHI
National Academy of Sciences of Ukraine
Sevastopol 99011
Ukraine

ABSTRACT

This paper aims at estimating the apparent velocity from sequences of satellite images. This study is an illustration of a more general methodology for generating, from satellite images, pseudo-observations of physical variables, that are assimilated within a geophysical forecast model in view of improving the quality of its results. In the case of the presented study, the used satellite images are sequences of Sea Surface Temperature (SST), from which pseudo-observations of sea surface velocities are generated, and assimilated within an ocean circulation model. The originality of the approach lies in the definition of an Image Model, that predicts the evolution of image information -here, SST- as a function of the pseudo-observations -here, surface velocity-. Satellite data are then assimilated within the Image Model, yielding an estimation of the pseudo-observations. In the case of this paper, this methods allows the estimation of sea surface velocities, even when large parts of the satellite images are corrupted by clouds. The Image Model plays the role of an intermediate model, between satellite data and the forecast model, and allows the assimilation of image information which is not directly linked to the state variables of the forecast model.

***Index Terms***— Image sequence analysis, Partial differential equations, Optimal control, Ocean

# 1. INTRODUCTION

The prediction of ocean physical variables, such as current, temperature, salinity, but also concentration of phytoplankton, is made possible by running ocean circulation models: a set of Partial Differential Equations governing the evolution and transport of the physical variables, and the associated boundary and forcing conditions. Ocean circulation models are highly demanding of input data, and yet make little use of the huge amount of available satellite measurements of the sea surface. The main reason for this is that very often, the information provided by satellite images cannot be easily related to the state variables of the models. Defining a methodology for

allowing the assimilation of satellite images within geophysical forecast models is therefore a challenging task required for reaching a significant improvement of forecasts' quality.

The presented paper illustrates a methodology for generating, from satellite images, pseudo-observations of the state variables of an ocean circulation model: Sea Surface Temperature (SST) image sequences are used to infer estimates of surface velocity, which are in turn assimilated within an ocean circulation model. Estimating velocities is usually approached by vision techniques, i.e. the estimation of apparent motion from sequences of SST images. However, results obtained by such methods are of poor quality, since due to cloud coverage and to other acquisition characteristics, a significant part of sea surface can be occluded in satellite acquisitions, making it impossible to compute temporal derivatives or to match pixels and structures. Figure 1 illustrates this issue, by displaying two successive SST images acquired by the NOAA-AVHRR sensor over the Black Sea: a significant cloud coverage (dark areas) of approximately 25% can be observed on these images.

We thus propose the definition of a new approach, and to address velocity estimation by devising an Image Model, that expresses the transport of temperature by surface velocity, using a simplified version of the 3D advection-diffusion equation that normally applies to temperature transport in oceans. We propose to assimilate satellite acquisitions of SST within this Image Model, in order to provide estimates of surface velocity even when a significant part of data is missing. Estimated velocities are in turn assimilated within the ocean circulation model to improve the quality of its results. The methodology, based on this intermediate Image Model, permits to define schemes for the assimilation of image information, not directly related to the state variables of the forecast model. In the presented application, images provide information on temperature, and the Image Model is used for establishing a link between temperature and velocity. The latter quantity can then be directly assimilated in the ocean circulation model. This process can be generalized to other types of image information, as for instance spatio-temporal structures
<figure><img src="image_2.png" /><figcaption>Fig. 1. Two successive frames of a SST sequence over the Black Sea, acquired by the NOAA-AVHRR sensor. Cloud covered areas appear black. White dots correspond to non acquired pixels.</figcaption></figure>

like fronts or trajectories.

The paper is organized as follows. In section 2, a survey of methods for sea surface velocity estimation and their assimilation within oceanic models is performed. The circulation model used for this study is also described. The Image Model, that expresses the transport of temperature in SST sequences, as well as the surface velocity estimation technique are presented in section 3. The assimilation of surface velocities within the circulation model is then presented in section 4, with results in section 5. Finally, conclusions and perspectives are given in section 6.

## 2. STATE OF THE ART

The forecast of ocean circulation relies on numerical simulation models, based on the so-called primitive equations [1], a set of Partial Differential Equations coupling the 3D velocity, temperature and salinity. 3D models are highly demanding in computer power and input data. Hence simplified 2D models, called shallow water models [2], are commonly used. In this study we make use of a shallow water model, developed and calibrated for the Black Sea circulation [3]. Its state variables are the 2D velocity $v = (u, v)$, and the height $h$ of the mixing layer, which constitutes the interface with the atmosphere. The evolution equations of the state variables are given in system (1), that introduces the Coriolis parameter $f$, the wind stress $\vec{\tau} = (\tau^{(x)}, \tau^{(y)})$, the horizontal diffusivity of the mixed layer $A_h$, and the reduced gravity $g' = g(\rho_0 - \rho_1)/\rho_0$, with $\rho_0$ corresponding to the reference density and $\rho_1$ to the average density of the mixed layer:

$$ 
\left\{ 
\begin{aligned}
\frac{du}{dt} - fv &= g' \frac{\partial h}{\partial x} + \frac{\tau^{(x)}}{\rho_0 h} + A_h \Delta u \\
\frac{dv}{dt} + fu &= g' \frac{\partial h}{\partial y} + \frac{\tau^{(y)}}{\rho_0 h} + A_h \Delta v \\
\frac{\partial h}{\partial t} + \frac{\partial(uh)}{\partial x} + \frac{\partial(vh)}{\partial y} &= 0.
\end{aligned}
\right. 
\quad (1) 
$$

This model requires input data for performing reliable forecasts: initial and boundary conditions of state variables, and forcing terms. Satellite data allow the observation of physical phenomena in geophysical sciences, and thus constitute an important potential input data source. They are however to a large extent under-utilized, because they often cannot be directly linked to the state variables of the forecast model. A prior image processing step can be applied to infer estimates of the state variables, which in turn can be directly assimilated. For instance, classical vision methods allow to infer the apparent motion field from image sequences. They rely on a conservation equation [4], that expresses the conservation over time of pixels' luminance, or, as suggested for fluid motion, of a volume or mass [5]. They have been applied to estimate sea surface velocity from SST images [6, 7, 8] but, as the conservation equation relies on the spatial and temporal derivatives of the image sequence, occlusions by clouds constitute a major limitation of their effective use. This is the reason why we introduce a new framework based on an Image Model, expressed in the image space, that provides a link between satellite measurements (SST) and the surface velocity: the Image Model is based on the physical models of the transport of temperature and thus is capable of providing information even under cloud-occluded areas.
### 3. DEFINITION OF THE IMAGE MODEL

The definition of the Image Model must allow the assimilation of satellite measurements to provide an estimate of sea surface velocity. Its state space therefore includes the two quantities: v, the 2D surface velocity, and T, the SST as measured by the NOAA-AVHRR sensor. The Image Model is built on two equations. The first one is a simplified version of the equation governing the transport of temperature by the 3D velocity field w, the advection-diffusion equation (2):

$$ \frac{\partial T}{\partial t} + \nabla T \cdot \mathbf{w} + T \cdot \mathrm{div} \mathbf{w} = K_T \Delta T + \mathrm{Src} - \mathrm{Snk} \quad (2) $$

where $K_T$ is the temperature diffusivity. Equation (2) is simplified: -assumed valid in the 2D image space (e.g. it applies to the 2D velocity with 2D derivatives of T); -water assumed incompressible (div w = 0); -source and sink terms, corresponding to the local heat flow, are neglected: a prior filtering of image data corrects most of their impact, at the expense of a lower confidence in data in the corresponding areas. The second equation should provide the evolution of the state variable v = (u, v). It has been assumed that, between two successive images, water particles evolve with a constant velocity, hence dv/dt = 0. The resulting Image Model is then defined as (eq. 3):

$$ \left\{ \begin{aligned} \frac{\partial T}{\partial t} + \nabla T \cdot \mathbf{v} &= K_T \Delta T \\ \frac{\partial u}{\partial t} + \nabla u \cdot \mathbf{v} &= 0 \\ \frac{\partial v}{\partial t} + \nabla v \cdot \mathbf{v} &= 0 \end{aligned} \right. \quad (3) $$

Estimation of surface velocity is performed using a variational 4DVAR data assimilation [9] scheme: NOAA-AVHRR measurements of SST are considered as observations of the state variable T, and v is considered as a control variable, adjusted to minimize the discrepancy between the Image Model's outputs for T and the observations of SST. This is formulated as:

$$ \left\{ \begin{aligned} \frac{\partial X}{\partial t} + F(X) &= \varepsilon_m \\ X(t_\bullet) &= X_\bullet(t_\bullet) + \varepsilon_\bullet \\ Y &= H(X) + \varepsilon_\bullet \end{aligned} \right. \quad (4) $$

where the first equation is the evolution model (3), with state vector $X = (T, u, v)$, the second equation is the initial condition, and the third equation links the observations Y to the state variables X through the observation operator H. Here, H reduces to the projection on the unit vector (1, 0, 0), since only the first state variable is observed. $\varepsilon_m, \varepsilon_\bullet, \varepsilon_\bullet$ stands for errors on model, initialisation and observations respectively, and are assumed to be uncorrelated in space and time. $\varepsilon_\bullet$ is given infinite values below clouds, large values in areas where

source and sinks have been filtered, and a constant value otherwise, determined by the sensor performance. The error covariance matrices B (model) and R (observation) are diagonal and built from $(\varepsilon_m, \varepsilon_\bullet)$ and $\varepsilon_\bullet$ respectively.

The data assimilation procedure consists in minimizing the energy functional J:

$$ J(X) = \frac{1}{2} \int_{\Omega,t} (X - X_\bullet)^T \mathbf{B}^{-1} (X - X_\bullet) \mathrm{d}\Omega \mathrm{d}t + \frac{1}{2} \int_{\Omega,t} (H(X) - Y)^T \mathbf{R}^{-1} (H(X) - Y) \mathrm{d}\Omega \mathrm{d}t \quad (5) $$

Minimization is performed using a quasi-Newtown method known as M1QN3 [10], involving the adjoint of system (3). The estimated velocities are then assimilated with the ocean circulation model, as described in section 4, and the results are presented in section 5.

### 4. ASSIMILATION OF FLOW FIELD WITHIN AN OCEANOGRAPHIC MODEL

The surface velocity field, estimated from NOAA-AVHRR images v = (u, v), is assimilated within the shallow water circulation model. A sequential Kalman-like data assimilation approach has been used. In this scheme, the analysis $X^a$ (i.e. the value of state variables after assimilation of measurements) is defined as:

$$ X^a = K[Y - HX] + X \quad (6) $$

where H stands for the observation operator and K is the Kalman gain matrix. To assess the potential of the methodology and prove its feasibility, we have implemented a simplified version known as nudging, where the gain matrix is replaced by a constant coefficient $\lambda$, whose value is fixed a priori:

$$ X^a = \lambda[Y - HX] + X. \quad (7) $$

### 5. RESULTS

The Black Sea shallow water model (1) has first been run without performing assimilation of velocities. An example of result is displayed in the top part of figure 2: the resulting motion field is very smooth, the global motion is correctly restituted. Pseudo-observations of sea surface velocities have been computed from SST sequences acquired by the NOAA-AVHRR sensor by assimilation within the Image Model (3). The pseudo-observations are then assimilated within the ocean model (1). The resulting velocity fields are displayed in the bottom part of figure 2. Due to the lack of ground truth, only a qualitative validation by experts is possible: much more local structures are forecasted, as for instance the small gyres close to the North-Eastern coast, and large structures are better described, for instance the size of the large gyre in the western part of the Black Sea is better forecasted after assimilation of sea surface velocities.
<figure><img src="image_3.png" /></figure>

<figure><img src="image_4.png" /><figcaption>**Fig. 2.** Results without (top) and with (bottom) assimilation of velocities.</figcaption></figure>

## 6. CONCLUSION AND PERSPECTIVES

The main contribution of this paper is the **definition** of a methodology for assessing state variables of a prediction model from satellite measurements, when there is no simple relation between them: an usual implementation of data assimilation would require building a complex observation operator, most likely non-linear, that expresses the information, extracted from satellite images, as a function of the model's state variables. Alternatively, we introduce an intermediate Image Model, expressed in the image space, that links satellite measurements to a subset of state variables, and that builds on a simplified physical representation of evolution equations.

This methodology is applied for satellite measurements of Sea Surface Temperature, and the Image Model couple these measurements to sea surface velocity. The latter, being a state variable, can be directly assimilated within ocean circulation models for improving the quality of forecast. It is also noticeable that this methodology permits the estimation of sea surface velocity even when large parts of satellite images are occluded by clouds.

Perspectives of this work concern the application of these approach to other types of information, extracted from satellite images. Existing data assimilation methods are actually restricted to the assimilation of images, considered as a collection of individual measurements, directly related to state

variables. The challenge is the design of methods for assimilating the structured information present in image data and sequences, such as fronts or trajectories. This study will be continued by devising specific Image Models for the evolution of structures like fronts, which assimilation would provide an estimate of state variables, directly assimilable within the forecast model.

## 7. REFERENCES

[1] R. Stewart, *Introduction to Physical Oceanography*, Department of Oceanography, Texas A&M University, 2002.

[2] B. de Saint-Venant, “Théorie du movement non-permanent des eaux,” *Compte-Rendu de l’Académie des Sciences*, Paris, 1871.

[3] G. Korotaev, T. Oguz, A. Nikiforov, and C. Koblinsky, “Seasonal, interanual, and mesoscale variability of the Black Sea upper layer circulation derived from altimeter data,” *Journal of Geophysical Research*, vol. 108, no. C4, 3122, 2003.

[4] B.K.P. Horn and B.G. Schunk, “Determining optical flow,” *Artificial Intelligence*, vol. Vol 17, pp. 185–203, 1981.

[5] D. Béréziat, I. Herlin, and L. Younes, “A generalized optical flow constraint and its physical interpretation,” in *Proceedings of CVPR’2000*, 2000.

[6] I. Herlin, F. X. Le Dimet, E. Huot, and J. P. Berroir, *Coupling models and data: which possibilities for remotely-sensed images?*, chapter e-Environement. progress and challenges, pp. 365–383, Instituto Politécnico Nacional, México, 2004.

[7] E. Huot, T. Isambert, I. Herlin, J.-P Berroir, and G. Korotaev, “Data Assimilation of Satellite Images within an Oceanographic Circulation Model,” in *Proceedings of ICASSP (International Conference on Acoustics, Speech and Signal Processing)*, Toulouse, France, May, 14-19 2006.

[8] X. Vigan, C. Provost, R. Bleck, and P. Courtier, “Sea surface velocities from Sea Surface Temperature image sequences,” *Journal of Geophysical Research*, August 2000.

[9] F.-X. Le Dimet and O. Talagrand, “Variational algorithms for analysis and assimilation of meteorological observations: Theoretical aspects.,” 2002, Tellus, 38A, 97-110.

[10] J.C. Gilbert and C. Lemaréchal, “The modules M1QN3 and N1QN3. Version 2.0c.,” Tech. Rep., INRIA, 1995.