Global climate change plays an essential role in our daily life. Mesoscale ocean eddies have a significant impact on global warming, since they affect the ocean dynamics, the energy as well as the mass transports of ocean circulation. From satellite altimetry we can derive high-resolution, global maps containing ocean signals with dominating coherent eddy structures. The aim of this study is the development and evaluation of a deep-learning based approach for the analysis of eddies. In detail, we develop an eddy identification and tracking framework with two different approaches that are mainly based on feature learning with convolutional neural networks. Furthermore, state-of-the-art image processing tools and object tracking methods are used to support the eddy tracking. In contrast to previous methods, our framework is able to learn a representation of the data in which eddies can be detected and tracked in more objective and robust way. We show the detection and tracking results on sea level anomalies (SLA) data from the area of Australia and the East Australia current, and compare our two eddy detection and tracking approaches to identify the most robust and objective method. The global ocean circulation is substantially affected by characteristic vortices, called mesoscale ocean eddies, moving circularly in parts of the Earth's great currents. Their main contribution to ocean circulation is due to the transport of a huge amount of kinetic energy. Hence, their detection and tracking is highly relevant in terms of oceanography and climate change studies, respectively. Pattern recognition in satellite imageries is a common task in remote sensing, where deep learning methods make the large amounts of data collected from space considerably more usable than traditional machine learning algorithms. These approaches allow the recognition of objects and coherent patterns from data, wherefore we develop a convolutional neural network (CNN)-based eddy detection and tracking framework. Mesoscale eddies are small circular currents that show characteristic cyclonal and anticyclonal patterns in global maps, e.g. sea level heights [1], and their automatic detection and identification is an active research topic. Several methods have been developed to approach the formulated detection task, some of which are geometrically inspired [2], or on the basis of signal decompositions [3]. In particular, the Okubo-Weiss method [4,5] is the most popular physical procedure to approach this problem, due to its physical interpretability, but open questions remain with respect to the threshold definiton [6]. On the other hand, deep learning algorithms are a breakthrough innovation to approach computer vision tasks such as object recognition, and have recently been used for oceanographic applications ( [7]). However, neural networks require a large amount of reference data, usually only available at a high cost, in order to learn an adequate model for recognition. Therefore, these supervised approaches are combined with data generation or data augmentation procedures to overcome this problem. [8,9], for example, propose automated techniques to generate training data. In contrast [10] suggest transfer learning approaches based on pretrained networks on large openly available datasets that are fine-tuned for the current classification task which significantly reduces the amount of training data needed. [11] develop a framework for the classification of multispectral images using deep learning techniques by segmentation with CNNs, and consider the scarcity of annotated satellite data by proposing the synthesization of reference data as an automated procedure. In this paper, we propose an eddy detection and tracking framework combining feature learning by CNNs with an established image processing tool, the Kanade-Lucas-Tomasi (KLT) [12] feature tracker. Furthermore, we compare this with a recurrent neural network (RNN), in particular a longshort term memory (LSTM), trained for eddy identification. We utilize the Okubo-Weiss method to tackle the lack of annotated data by detecting a few, yet precise eddies, which are used as training data. By combining these approaches, we are able to achieve a high recall as well as high precision, which tradeoff is generally a problem for ordinary eddy detection method such as the Okubo-Weiss procedure. Our study site covers seas around Australia and the South Equatorial current. The dataset contains high-resolution, global sea level anomaly maps (SLA), which are preprocessed products from the ESA Climate Change Initiative (ESACCI) [13]. The gridded SLA maps contain the mesoscale coherent eddy features and variabilities and enable the dis- The CNN-module learns a model from which we can detect eddy cores at a single epoch, see Sec. 3.2. The KLT-tracker tracks the detected eddies using a sparse optical flow. On the other hand, a RNN uses convolutional LSTM units to track the eddy cores, see 3.3. Since no reference data is given, we evaluated our results qualitatively with focus on plausability. In order to tune and train the neural network, we require annotated reference data, which generally does not exist in our application. Consequently, we propose an automated labeling process to generate precise pseudo-reference data based on the results of the Okubo-Weiss method [4,5,14]. The Okubo-Weiss method is an established algorithm for eddy detection. It aims at the computation of a physically motivated parameter that represents the balance between vorticity scales and shear strain rates. The value of Okubo-Weissparameter decides whether the vorticity dominates over strain, where an eddy core is defined as vorticity dominated area. Therefore, the choice of the threshold on the Okubo-Weissparameter defines the precision and recall of the eddy identification. Based on this, we define a threshold which provides a few, yet precise eddy cores, which are corrected manually and used as training and validation data for the CNN. We treat the task as a two-class problem such that each cell of the grid-based data is assigned to the class 'eddy' or 'non-eddy', nevertheless, the approach can be extended to distinguish between anticyclonic, cyclonic and non-eddies. Since each eddy core can only be defined with a certain precision, we define a circular domain around each detection and treat it as eddy core rather than defining only one grid cell. In order to track identified eddies over time, we developed two approaches.
First, we utilize the KLT-tracker with sparse optical flow estimations and, besides, we apply a convolutional LSTM network. Optical flow is defined as distribution of apparent velocities and motion patterns, and therefore leads to spatial information and the changes of spatial arrangements of objects. The KLT-tracker is a feature tracker that traces distinctive points over time based on the motion field of optical flow estimations [12]. Thus, the differences in motion between two consecutive time steps are defined as I(x, y, t) = I(x + x, y + y, t + t), where I is the gridded SLA data with position (x, y) at time t and the corresponding differentials x, y and t. In our study, the positions are estimated by the detection method. As second approach, we use RNNs which directly process time-varying image sequences of SLA data. In detail, we apply convolutional LSTMs, which are able to model spatiotemporal dependencies. We investigate the CNN-based detections regarding robustness and amount, and their suitability as input into a KLTtracker with sparse optical flow estimation. In contrast, we address the stand-alone tracking approach with convolutional LSTMs. For our study, the threshold for the Okubo-Weiss method is set to 0.1 W . Here,  W is the standard deviation of the Okubo-Weiss parameter. The radius of the eddy cores used as pseudo-reference data is defined as 5 grid cells in each direction which corresponds to 135 km. For training the CNN, we use a kernel size of 3 grid cells which is equal to the stride. Furthermore, we have trained the CNN on 100 epochs and used a batch size of 18 grid cells. However, the convolutional LSTM is trained on 300 epochs since the net needs to fit more hyperparameters. For those computations, we take advantage of a GPU's power, although, we need to resize the images to 49x49 grids. We extract training data from all time steps, except March 1993, which we use for testing. We use the data from March 1st 1993 for eddy detection, and the next four days for tracking. The implementations for our framework are realized in MATLAB and Python. The Python interfaces are based on Keras with TensorFlow backend. The results of the KLT-Tracker are presented in the SLA maps of day 2 up to day 5 in The convolutional LSTM reveals plausible and robust preliminary results. Hence, it is preferable to develop a convolutional LSTM that is able to distinct between single cyclonal and anticyclonal eddies. Furthermore, in order to make eddy detections more robust and precise, we propose to extend the feature space with dense optical flow information at each point. Further data like vorticity or sea surface temperatures are other options provided they are independent from SLA data. However, the spatial resolution is not adequate, such that the analysis of along-track data in place of gridded maps will be considered in the future .