## 3 DL Models for Ocean Studies
### Vit  
Originally designed for natural language processing, the Transformer architecture has been adapted to computer vision tasks in the form of the Vision Transformer (ViT). A typical ViT follows a structured process involving several key steps. First, the input image is divided into smaller patches, which are embedded into a sequence through a patch embedding layer. Next, the embedded patches are processed by a series of Transformer encoder layers, where self-attention mechanisms are applied to capture global dependencies across the entire image. Finally, the output from the encoder layers is passed through a classification head to generate the final prediction. In contrast to CNNs, which focus on local feature extraction, ViTs can directly model the global context. This global perspective proves to be advantageous in large-scale ocean remote sensing tasks, such as SST or MHV predictions, and sea ice classification [90,91]. The aforementioned models exhibit varying strengths across tasks like classification, segmentation, and time-series forecasting. MLPs are well suited for tasks involving structured tabular data, such as estimating ocean surface pH from variables like SST, sea surface salinity (SSS), and dissolved inorganic carbon (DIC). However, they lack the ability to model spatial or temporal dependencies, and thus, they are not suitable for processing high-dimensional gridded data (e.g., satellite imagery) or time series data. CNN-based architectures like Vanilla CNNs and U-Nets are widely used for spatial classification and segmentation tasks, such as sea ice classification, coral reef or kelp mapping, and coastal wetland monitoring, due to their ability to capture spatial features from gridded remote sensing images. However, these models are inherently limited by their local receptive fields, making it difficult to capture long-range spatial dependencies or global relationships. Besides, conventional CNNs cannot capture temporal information and often require integration with temporal modules such as LSTM. As a result, standalone CNN models are less suitable for applications involving temporal forecasting (e.g., sea level prediction) or for processing nongrid structured data (e.g., tabular in situ buoy observations). Tasks involving temporal forecasting, like SST and ENSO prediction, ship fuel consumption forecasting, and wind speed estimation, are beneficial from sequence models like GRU, LSTM, and TCN, which are effective in capturing sequential dependencies. Among them, LSTMs leverage gating mechanisms to retain long-term memory more effectively than traditional RNNs, enabling them to predict SST with lead times of up to 18 months. However, these models are inherently limited in capturing spatial structures and are often integrated with CNNs to model spatiotemporal patterns. Transformers and their variants, such as Vision Transformers, have demonstrated superior performance in complex spatiotemporal tasks due to their powerful self-attention mechanism, which allows the model to dynamically focus on relevant features across time and space. Despite these advantages, Transformer-based models often require high computational resources and large volumes of high-quality data, making them suitable for tasks involving abundant spatiotemporal datasets and less effective in applications with limited training samples. Several representative DL architectures applied to various oceanographic tasks are summarized in Tables 2-4, which report quantitative performance metrics such as accuracy, RMSE, and correlation coefficients. These analyses aim to provide some insights for researchers in selecting DL architectures based on the characteristics of their datasets and specific tasks. In addition to these commonly used models, other advanced architectures, such as Graph Neural Networks (GNNs), also play important roles in ocean-related studies [92].