SIFM: A Foundation Model for Multi-granularity Arctic Sea Ice Forecasting

## Abstract

Abstract Arctic sea ice performs a vital role in global climate and has paramount impacts on both polar ecosystems and coastal communities.
In the last few years, multiple deep learning based pan-Arctic sea ice concentration (SIC) forecasting methods have emerged and showcased superior performance over physics-based dynamical models.
However, previous methods forecast SIC at a fixed temporal granularity, e.g. sub-seasonal or seasonal, thus only leveraging inter-granularity information and overlooking the plentiful inter-granularity correlations.
SIC at various temporal granularities exhibits cumulative effects and are naturally consistent, with short-term fluctuations potentially impacting long-term trends and long-term trends provides effective hints for facilitating short-term forecasts in Arctic sea ice.
Therefore, in this study, we propose to cultivate temporal multi-granularity that naturally derived from Arctic sea ice reanalysis data and provide a unified perspective for modeling SIC via our S ea I ce F oundation M odel.
SIFM is delicately designed to leverage both intra-granularity and inter-granularity information for capturing granularity-consistent representations that promote forecasting skills.
Our extensive experiments show that SIFM outperforms off-the-shelf deep learning models for their specific temporal granularity.

## 1 introduction

Arctic sea ice has a profound influence on both local and global climate systems. The near-surface air temperature of Arctic regions has increased at a speed that is two to nearly four times faster than the global average from 1979 to 2021, a phenomenon known as “Arctic amplification” (Screen and Simmonds, ; Rantanen et al., ).
This accelerated temperature rise performs a key role in the unprecedented rapid diminishing of Arctic sea ice which has extensive consequences that could transcend the polar area.
For example, the accelerated reduction of Arctic sea ice could not only jeopardize the survival of species residing in polar regions but also pose adverse effects on local communities whose livelihoods and well-being depend on those animals; it could substantially affect mid-latitude summer weather by weakening the storm tracks (Vavrus, ); and it will bring new opportunities for marine transportation and new access to natural resources like fossil fuels (Vincent, ).

Due to its vital role in coastal communities, global climate, and potential impacts on the world’s economy, numerical and statistical models have been proposed to forecast pan-Arctic sea ice concentration (SIC) ranging from sub-seasonal to seasonal scale (Johnson et al., ; Wang et al., ).
However, numerical and statistical models usually rely on high-performance computing on CPU clusters and often lead to complex debugging processes and uncertain parameterization, which limits their performance in forecasting long-term SIC changes.
With the advent of deep learning models and their powerful capability in capturing complex patterns within high dimensional data, recent studies have developed end-to-end SIC forecasting models based on deep learning approaches and have presented a promising performance that exceeds previous numerical and statistical methods (Andersson et al., ; Ren et al., ).
Although the intrinsic annual cyclic trend and intra-seasonal predictability of Arctic sea ice (Wang et al., ) contains rich information both between and within temporal scales, existing deep learning-based methods mainly focus on predicting SIC at a specific temporal granularity, e.g., 7 days or 6 months’ averages, which leads to potential neglect of intrinsic correlation between different time scales and limits the performance of forecasting models.
Since the Arctic sea ice extent (SIE, where SIC value is larger than 15%) has been observed a continuous reclining trend during the last few decades (Figure 1(a)) and a clear recurrent variational pattern, i.e., the annual pan-Arctic sea ice edge usually starts to expand after the summer melting season in September (Figure 1(b)), utilizing inter-granularity and intra-granularity information could be mutually beneficial.
For instance, long-term trends in weekly granularity could help to calibrate short-term daily predictions and finer granularity features could provide more accurate initial conditions to facilitate seasonal forecasting.
Besides, the most commonly utilized U-Net architecture (Ronneberger et al., ) in previous work (Andersson et al., ) implicitly fulfills sequential modeling by channel-wise fusion operations.
The prediction of future SIC is essentially a spatio-temporal forecasting task involving the prediction of over a hundred thousand time series, each representing a non-overlapping grid location in the Arctic region.
We argue that considering forecasting future SIC is obviously a spatio-temporal task, and explicit modeling of SIC sequences could improve forecasting skills.

Figure: Figure 1: Visualization of Arctic sea ice trends. (a)The annual average SIC and SIE trend over the last 35 years (1987-2023); the monthly cyclic trend of SIC (b) and SIE (c). Note that the averaged SIC values are calculated over the entire pan-Arctic region which could only be used to observe the trend.
Refer to caption: x1.png

Based on the above-mentioned motivations, we propose the transformer-based Sea Ice Foundation Model (SIFM) that unifies the temporal granularity of interest to boost overall performance on forecasting SIC in pan-Arctic region.
Unlike previous approaches (as demonstrated in Figure 2), we propose to independently tokenize spatial features, explicitly extract sequential information and jointly model three granularities: daily, weekly average, and monthly average.
Specifically, SIFM first embeds SIC from each temporal granularity into independent spatial tokens and sequentially concatenated to represent temporal fluctuations within each granularity.
Then, we treat these independent sequences as correlated granularity variates and utilize the attention mechanism in conjunction with the feed-forward network (FFN) for extracting both intra-granularity and inter-granularity correlations.
By incorporating multi-granularity representation, SIFM could simultaneously generate future SIC in different temporal scales and boost overall performance.
Our contributions are three folds:

- •
We revisit the potentially overlooked inter-granularity information by previous methods for Arctic SIC forecasting and explore the effectiveness of independent spatial tokens representation of SIC for facilitating accurate predictions.
- •
We propose SIFM that leverages independent spatial tokenization of SIC and effectively unifies three temporal granularities that cover from sub-seasonal to seasonal scale for better overall representation and improved forecasting performance.
- •
The comprehensive experiments demonstrate that by adopting the approach of multi-granularity fusion, our SIFM achieves state-of-the-art on prediction in each granularity, which advances toward a more practical Arctic sea ice forecasting system.

Figure: Figure 2: The main differences between (a) existing mainstream SIC forecasting approaches and (b) our SIFM are follows: (1) Previous models take a channel-wise fusion to jointly extract spatial features, e.g., utilizing 2D convolution to expand and downsample SIC channels. In our case, we focus on capturing effective spatial tokens representation of SIC by the shared spatial encoder. (2) The correlation among input SIC sequence is implicitly modeled via the U-Net-based architecture in (a) while SIFM explicitly captures intra-granularity and inter-granularity correlation via sequential modeling. (3) We propose leveraging multi-granularity information that is naturally derived from the SIC and embedding it into granularity variates to improve overall forecasting skills.
Refer to caption: x2.png

## 2 Related works

### 2.1 Sea ice concentration forecasting

Researchers have proposed various approaches to forecasting SIC, encompassing numerical and statistical models (Wang et al., ; Yuan et al., ).
However, numerical and statistical models usually rely on the high-performance computing of the CPU cluster and tend to result in complex debugging processes and uncertain parameterization.
Recently, deep learning models have drawn the attention of sea ice research communities and have been widely investigated for Arctic sea ice forecasting (Petrou and Tian, ; Kim et al., ; Ali et al., ; Ali and Wang, ).
These methods utilize U-Net-based architectures to solve daily (SICNet (Ren et al., ), or monthly (IceNet (Andersson et al., ), MT-IceNet (Ali and Wang, )) SIC forecasting.
However, although these U-Net-based architectures are built on top of LSTM (Liu et al., ), CNN (Andersson et al., ) or vision transformer (Xu et al., ), the temporal information inherent in sea ice modeling can not be fully exploited.
Moreover, these methods and the latest Transformer-based model  (Zheng et al., ) concentrate on single-granularity SIC forecasting, where the inter-granularity information from multi-granularity sea ice modeling is overlooked.

### 2.2 Multi-scale representative learning

The multi-scale phenomenon is common in vision tasks, while it is always overlooked in sea ice modeling.
To exploit the information in multi-scale sources, multi-scale features are commonly exploited by using spatial pyramids (Lazebnik et al., ), dense sampling of windows (Yan et al., ), and the combination of them (Felzenszwalb et al., ) in the vision community.
The learning of CNN-based multi-scale representations is typically approached in two ways: utilizing external factors like multi-scale kernel architectures and multi-scale input architectures (Reininghaus et al., ), or designing internal network layers with skip and dense connections (Lin et al., ).
Recently, there has been a surge of interest in applying transformer-based architectures to computer vision tasks, with the Vision Transformer (ViT) being particularly successful in balancing global and local features compared to CNNs (Dosovitskiy, ).
When revisiting the task of forecasting sea ice concentration, its multiscale features stem from different temporal resolutions. Existing methods focus on a single scale, such as daily, weekly, or monthly. However, different temporal resolutions are inherently connected, and treating them as a single scale for modeling would increase the complexity of network learning.

## 3 SIFM for multi-granularity arctic sea ice forecasting

Given historical Arctic SIC records $Y=\{X_{T-L-1},...,X_{T-1},X_{T}\}\in[0\%,100\%]^{L\times H\times W}$, where $L$ is the input length of a specific granularity which includes the given observation time step $T$, $H$ and $W$ denotes the rectangle pan-Arctic region, the forecasting model predicts the subsequent SIC values $\hat{Y}=\{X_{T+1},...,X_{T+P-1},X_{T+P}\}\in[0\%,100\%]^{P\times H\times W}$ with forecasting lead times of $P$.
In this study, our SIFM jointly models three granularities, i.e., daily, weekly average, and monthly average SIC values that cover both sub-seasonal and seasonal variations, and simultaneously forecasting on all these temporal scales. For each temporal granularity, the input length $L$ equals the forecasting lead times $P$. The overview of the proposed SIFM architecture is presented in Figure 3. The shared spatial encoder and decoder perform SIC tokenization and restoration while multi-granularity fusion explicitly extracts sequential information.

Figure: Figure 3: Overview of proposed SIFM, which comprises three main components: (1) The shared spatial encoder first independently extracts spatial features of input SIC from each granularity (i.e. 7 days, 8 weeks’ averages and 6 months’ averages) to obtain spatial tokens, and then concatenates these spatial tokens accordingly. (2) The embedded spatial tokens are subsequently flattened with respect to their granularity and linearly projected into the same length. We propose to utilize an encoder-only transformer backbone to perform multi-granularity fusion which explicitly captures both inter-granularity and intra-granularity sequential features. (3) Lastly, the predicted multi-granularity features are restored to the shape of the input via linear transformation and the shared spatial decoder.
Refer to caption: x3.png

### 3.1 Sea ice concentration tokenization

Existing mainstream deep learning-based methods for SIC forecasting adopt U-Net architectures and leverage 2D convolution to perform channel-wise expansion and downsampling that extracts both spatial features and temporal dependencies.
However, since U-Net-based models are not specifically designed for sequence modeling (Azad et al., ), the joint spatial-channel fusion of SIC and implicit sequence modeling could be ill-posed properties for spatio-temporal forecasting tasks.
In this regard, we propose to independently tokenize spatial features at first, which could disentangle the above ill-posed problem and be beneficial for SIC forecasting.

Independent spatial embedding.
Since we aim to simultaneously model SIC derived from three temporal granularities, encoding their spatial features into shared embedding space not only yields consistent representation but also reduces the number of trainable parameters.
Inspired by prior works (Hu et al., ; Chen et al., ), we utilize Swin Transformer V2 (Liu et al., ) as the backbone for both shared spatial encoder and decoder.

Specifically, each SIC input is independently fed into the shared spatial encoder and partitioned by a non-overlapped window to generate patch representation (Dosovitskiy, ) with 32 spatial channels (the original SIC data has only one channel).
To preserve local SIC information, we choose the smallest 2 by 2 window size for the patch partition.
Then, the patch tokens are further transformed by the first two Swin Transformer blocks.
The multi-scale spatial features are extracted through the subsequent hierarchical encoder layers which comprise a patch merging operation and two Swin Transformer blocks.
The patch merging operation first concatenates the spatial feature of each group of 2 by 2 adjacent patch representations from the previous encoder layer.
The calculation of each pair of two consecutive Swin Transformer blocks in encoder layers can be described as follows:

$$ $\displaystyle z^{b}_{s}=LN(WMSA(z^{b-1}))+z^{b-1},$ $\displaystyle z^{b}=LN(MLP(z^{b}_{s}))+z^{b}_{s},$ $\displaystyle z^{b+1}_{s}=LN(SWMSA(z^{b}))+z^{b},$ $\displaystyle z^{b+1}=LN(MLP(z^{b+1}_{s}))+z^{b}_{s},$ (1) $$

where $z^{b}_{s}$ and $z^{b}$ represents the output spatial feature of the (Shifted) Window-Multi-head Self Attention module and the MLP module for block $b$, respectively; LN denotes the layer normalization operation (Lei Ba et al., ).
The attention mechanism with a shifted window could effectively extract neighboring SIC information and sufficiently represent the local correlation of sea ice.
After all input SIC are independently encoded into 2D spatial features, we apply linear projection to generate 1D token for each SIC to obtain compact spatial representation for sequential modeling.

The shared spatial decoder adopts an identical Swin Transformer backbone and the decoding procedure is symmetrical to the encoding process, except that the patch merging operation is replaced by the patch expanding operation (Cao et al., ).
While patch merging downsamples the input spatial feature dimension and increases the embedding channels, patch expanding symmetrically restores the resolution of the feature map and merges channels via linear transformation.

Spatial feature skip connection.
Since the SIC features encoded by Swin Transformer blocks will be tokenized into highly compact sequence representation, the spatial SIC information should be maximally preserved during the sequential modeling.
Besides, our proposed sequential modeling backbone comprises deep encoding layers which might lead to loss of embedded spatial features.
To preserve spatial SIC information and avoid insufficient restoration, we propose to add a skip connection between the output of the last pair of Swin Transformer blocks in the spatial encoder and the input of the first block in the shared decoder (see in Figure 3).

### 3.2 Multi-granularity fusion

We propose to jointly model three granularities that cover sub-seasonal to seasonal scale, i.e., 7 days, 8 weeks averages, and 6 months averages, and explicitly capture inter-granularity correlation and intra-granularity sequential information.

Figure: Figure 4: Comparison between different backbones for temporal sequence modeling: (a) Our proposed SIFM sequentially concatenates independent SIC tokens that are derived from each temporal scale as a granularity variate and applies an attention mechanism over the embedded variate tokens. The FFN transforms the variate representation for the input of the next layer; (b) For vanilla Transformer architecture (Vaswani, ), it applies an attention mechanism over temporal tokens and FFN is applied on multivariate representations; (c) The MLP-mixer (Tolstikhin et al., ) approach first performs token-wise mixing, then transpose the extracted features to apply channel-wise mixing. The vanilla Transformer and MLP-mixer both fall short of modeling the sequential information of sea ice.
Refer to caption: x4.png

Modeling granularity variates.
As mentioned in Section 3.1 the shared spatial encoder transforms each SIC into independent 1D tokens.
These individual spatial tokens are then concatenated sequentially based on their granularity respectively and utilized to form the multi-granularity representation.
As each granularity incorporates a different time span, the dimensions of concatenated granularity sequences are mismatched.
Considering that both the weekly average and monthly average are derived from daily SIC values, we choose to tokenize those sequences further and align their feature dimensions with the length of daily input using a linear transformation.
The generated multi-granularity variates are subsequently fed into the sequential modeling backbone.
Encouraged by prior work (Liu et al., ), we propose to adopt an encoder-only Transformer architecture as the sequential modeling backbone for multi-granularity fusion in Figure 3 that:
(1) applies multi-head self-attention on the embedded granularity variate tokens to explicitly capture inter-granularity correlations;
(2) each granularity variate is independently processed by FFN to extract intra-granulairty information (as depicted in Figure 4(a)).
As for the conventional usage of vanilla Transformer in sequence prediction, the attention mechanism is applied on embedded temporal tokens which comprise variate information collected from the same time step (as in Figure 4(b)).
The vanilla Transformer is challenged in forecasting series with larger lookback windows due to performance degradation and computation explosion.
Furthermore, the temporal token embeddings incorporate multiple variates that represent distinct physical measurements, which may struggle to capture variate-specific representations and potentially lead to the generation of incoherent attention maps.
However, in sea ice modeling, each dimension of the tokenized granularity variate incorporates SIC features that come from a different time span. This could lead to poor representation of sequential SIC features and restrict the effective modeling of inter-granularity correlations.
Experimentally, we will show in Section 4.4 that by adopting our sequential modeling, the overall performance is superior to alternative backbones.
After each SIC granularity variate token is properly fused and encoded, the final prediction of future granularity variate features is generated through a linear projection layer.

Sequential feature skip connection. Considering the concatenated sequence of SIC features are linearly transformed and aligned to form the multi-granularity variate representation, the significant original sequential feature needs appropriate preservation. Besides, the deep sequence encoding process could introduce unintended noise that deteriorates the modeling of intra-granularity correlation.
To compensate for the intra-granularity information and reduce the potential impact that impairs inter-granularity modeling, we propose to utilize the cross-attention mechanism as a sequential skip connection (as in Figure 3), where the latent query features are sourced from the concatenated sequence token before the linear projection and the predicted SIC sequence generates both key and value latent representations.
The details about this process can be found in Appendix.

## 4 Experiments

In this section, we evaluate the forecasting performance of our SIFM over 8 years of SIC data and compare it with other deep learning models. The implementation details of our SIFM is provided in Appendix.

### 4.1 Datasets

We evaluate our proposed SIFM framework on the G02202 Version 4 dataset from the National Snow and Ice Data Center (NSIDC). It records daily SIC data starting from October 25^th 1978 and provides the coverage of the pan-Arctic region (N:-39.36^∘, S:-89.84^∘, E:180^∘, W:-180^∘).
Each daily SIC data is formed of 448 x 304 pixels and each pixel corresponds to the area of a 25km x 25km grid. The SIC data has a range of 0% to 100% and areas where SIC value is greater than 15% indicate the SIE.
We choose data from October 25^th 1978 to the end of 2013 as the training dataset, the years 2014 and 2015 are selected as validation set, and data collected from 2016 to 2023 are used to test models.

**Table 1: Quantitative results of SIC forecasting. We compare the performance of SIFM in each temporal granularity with corresponding deep learning based methods. * marks that the performance figures are reported in their original papers for reference.**
| Temporal Scale | Lead Times | Methods | RMSE$\downarrow$ | MAE$\downarrow$ | $R^{2}$$\uparrow$ | NSE$\uparrow$ | IIEE$\downarrow$ | SIEdif$\downarrow$ |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Sub-seasonal | 7 Days (Daily) | SICNet | 0.0490 | 0.0100 | 0.982 | 0.979 | 976 | 0.0718 |
| SIFM | 0.0429 | 0.0096 | 0.987 | 0.985 | 926 | 0.0380 |  |  |
| 45 Days (Daily) | IceFormer* | 0.0660 | 0.0201 | 0.960 | - | - | - |  |
| 90 Days (Daily) | SICNet90* | - | 0.0512 | - | - | - | - |  |
| 8 Weeks Average (Weekly) | SIFM | 0.0625 | 0.0140 | 0.973 | 0.968 | 1600 | 0.1541 |  |
| Seasonal | 6 Months Average (Monthly) | IceNet* | 0.1820 | 0.0916 | 0.567 | - | - | - |
| MT-IceNet* | 0.0777 | 0.0197 | 0.915 | - | - | - |  |  |
| SIFM | 0.0692 | 0.0166 | 0.917 | 0.910 | 2156 | 0.2083 |  |  |

Figure: Figure 5: Qualitative analysis of SIE prediction. The derived SIE ground truth and prediction generated by SIFM and three single-granularity models (one for each temproal granularity) over: (a) The first week of September; (b) 4 weeks; (c) 1 month. Considering the abnormal increase of Arctic sea ice in 2022, our proposed method could still produce reasonable forecasts that keep the similar overall shape of Arctic SIE.
Refer to caption: x5.png

Figure: Figure 6: Spatial residual of predicted SIC. We examine the spatial patterns of forecasting results over the same period presented in Figure 5: SIFM could generate consistent daily forecasts (a). Considering the abnormal Arctic SIC change in 2022, the annual trend could be different than the SIC data on which the model was trained, SIFM could still predict weekly (b) and monthly (c) average SIC with a bounded residual region rather than scattered forecasting results. The spatial residual is calculated by using predicted SIC to subtract the ground truth value.
Refer to caption: x6.png

Figure: Figure 7: Averaged intra-granularity forecasting error. We evaluate models trained on multi-granularity and single-granularity SIC and plot RMSE and MAE of each lead time step in three temporal granularities over the test dataset.
Refer to caption: x7.png

### 4.2 Evaluation Metrics

To evaluate SIFM, we select widely used root mean square error (RMSE) and mean absolute error (MAE) for comparison of forecasting accuracy. We also leverage $R^{2}$ score to evaluate the performance:

$$ $\displaystyle R^{2}=1-\frac{RSS}{TSS}.$ (2) $$

where RSS represents the sum of squares of residuals and TSS denotes the total sum of squares. The Integrated Ice-Edge Error score (Goessling et al., ) is introduced to evaluate the prediction of SIE:

$$ $\displaystyle IIEE$ $\displaystyle=O+U,$ (3) $\displaystyle O$ $\displaystyle=\sum(Max(SIE_{p}-SIE_{t},0)),$ (4) $\displaystyle U$ $\displaystyle=\sum(Max(SIE_{t}-SIE_{p},0)),$ (5) $$

$$ $SIE_{p},SIE_{t}=\left\{\begin{array}[]{l}{1,SIC>15}\\ 0,SIC\leq 15\end{array}{}\right.$ (6) $$

where O and U represent the overestimated and underestimated SIE between the prediction ($SIE_{p}$) and the ground truth ($SIE_{t}$), respectively. The difference between the forecasted and ground truth sea ice area (in millions of $km^{2}$) is calculated as follows:

$$ $\displaystyle SIE_{dif}=\frac{\sum(|SIE_{p}-SIE_{t}|)\times 25\times 25}{10000 00}.$ (7) $$

We also adopt the Nash-Sutcliffe Efficiency (Nash and Sutcliffe, ) to further evaluate the predicted quality:

$$ $\displaystyle NSE=\frac{1-\sum((SIC_{t}-SIC_{p})^{2})}{\sum((SIC_{t}-Mean(SIC_ {t}))^{2})}$ (8) $$

### 4.3 Multi-granularity forecasting

#### Baselines.

Since our SIFM simultaneously generates predictions of three granularities, we select corresponding forecasting deep learning-based models for comparison.
Specifically, we re-implemented SICNet (Ren et al., ) and trained under an identical environment for direct comparison on 7 days SIC forecasting;
Due to dataset and code accessibility, we adopt performance of sub-seasonal forecasting methods as SICNet90 (Ren and Li, ), IceFormer (Zheng et al., ), and seasonal forecasting methods IceNet (Andersson et al., ), MT-IceNet (Ali and Wang, ) that reported in the original paper for reference.

#### Main results.

The overall performance of SIFM and baseline methods are listed in Table 1.
The lower RMSE/MAE indicates a more accurate forecast in SIC values. Methods with lower IIEE/SIEdif are more capable of identify the edge of sea ice while higher $R^{2}$/NSE suggests that the predicted spatial patterns are more close to the ground truth.
Our proposed method achieves the best performance in all metrics for forecasting 7 days SIC, establishes a new state-of-the-art method for sub-seasonal weekly average prediction, and presents superior seasonal SIC forecasting capability.
Considering the fact that baseline methods, except for SICNet, utilizes several additional atmospheric and oceanic variables to facilitate forecasting, and our SIFM only leverages SIC data with carefully extracted intrinsic inter-granularity correlation, it verifies the effectiveness of the proposed approach for multi-granularity forecasting.

#### Qualitative Analysis.

To visually verify the forecasting skills of SIFM, we select the end of the melting season in September 2022.
From Figure 1(a) we can observe that the annual Arctic sea ice in 2022 has increased by a considerable margin which is against the persisting long-term reclining trend.
This unusual rise makes SIC and SIE difficult for our model to predict since it only learns from the data collected before 2014.
Starting from September 1^st, we calculate averaged SIC of 7 days, 4 weeks and 1 month that correspond to three temporal granularities of SIFM.
The ground truth of calculated average SIC along with the ground truth and predicted SIE are visualized in Figure 5. The forecasting results in the lower row are produced by SIFM and the upper row represents predictions generated by three variants of SIFM that only leverage single-granularity SIC, we will discuss later in Section 4.4.

Despite the inconsistent annual trend of Arctic SIC in 2022, our method could still generate forecasts that are consistent with the average SIE in the first week of September (Figure 5(a)), and the general shape in both 4 weeks’ average (Figure 5(b)) and 1-month average (Figure 5(c)). Comparing to models with similar backbone of SIFM but only leverage single-granularity SIC, the prediction of SIE are noticeably different to the ground truth indicating that SIFM could effectively leverage multi-granularity SIC to improve forecasting skills.

We plot spatial residuals to further investigate the learned spatial patterns of our SIFM. In Figure 6(a), SIFM could accurately predict the first week of SIC, while in coarser weekly average granularity our SIFM tends to slightly underestimate in Arctic sea ice edge areas (Figure 6(b)). For the predicted monthly average of September 2022, the overall shape of SIE resembles the observation but overestimates SIC along the boundary.

### 4.4 Ablation Study

To further analyze the performance of our proposed method, we trained five additional variants of SIFM (as in Figure 7), i.e., three single-granularity models that respectively utilize temporal granularities in SIFM, and two multi-granularity forecasting models with different backbones to perform the multi-granularity fusion.

#### Effectiveness of Multi-granularity modeling.

We first verify our proposed multi-granularity modeling approach by comparing SIFM with models that comprise of similar model architecture but only adopt single granularity SIC data. Comprehensive experiments in Table 2 show that by leveraging the naturally derived multi-granularity SIC, the overall performance in all temporal scales can be promoted by a significant margin.
For each individual forecasting lead time, SIFM consistently outperforms models solely trained on single-granularity data (as shown in Figure 7).

**Table 2: Effectiveness of multi-granularity representation. Multi represents the proposed SIFM and Single stands for models with similar backbone but trained solely on single-granularity data.**
| Temporal Scale | Lead Time | Granularity | RMSE$\downarrow$ | MAE$\downarrow$ | $R^{2}$$\uparrow$ | NSE$\uparrow$ | IIEE$\downarrow$ | SIEdif$\downarrow$ |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Sub-seasonal | 7 Days | Single | 0.0704 | 0.0148 | 0.982 | 0.979 | 1018 | 0.0509 |
| Multi | 0.0429 | 0.0096 | 0.987 | 0.985 | 926 | 0.0380 |  |  |
| 8 Weeks Average | Single | 0.0771 | 0.0163 | 0.962 | 0.954 | 2208 | 0.3301 |  |
| Multi | 0.0625 | 0.0140 | 0.973 | 0.968 | 1600 | 0.1541 |  |  |
| Seasonal | 6 Months Average | Single | 0.0721 | 0.0191 | 0.882 | 0.873 | 2482 | 0.4298 |
| Multi | 0.0692 | 0.0166 | 0.917 | 0.910 | 2156 | 0.2083 |  |  |

#### Alternative backbone for multi-granularity fusion.

To validate the effectiveness of our proposed multi-granularities fusion and sequential modeling approach, we compare the performance of our SIFM with two other variants that are trained on the identical multi-granularity data with different sequential backbones, i.e., Transformer and MLP mixer (Tolstikhin et al., ; Ekambaram et al., ).
Considering intra-granularity performance in Figure 7, SIFM presents superior forecasting skill in each time step of daily, weekly average and monthly average when compared to multi-granularity variants, indicating the effectiveness of multi-granularity SIC variates for sequential modeling.
As shown in Table 3, our SIFM outperforms these variants by a great margin, demonstrating the intra-granularity and inter-granularity correlations inherent in the sea ice modeling benefits for the forecasting.

**Table 3: Effectiveness of proposed approach for multi-granularity fusion. We adopt conventional utilization of Transformer and recent trend in leveraging full MLP-based backbone (Tolstikhin et al., ) for temporal sequence modeling as counterparts of our proposed sequential backbone.**
| Temporal Scale | Lead Time | Method | RMSE$\downarrow$ | MAE$\downarrow$ | $R^{2}$$\uparrow$ | NSE$\uparrow$ | IIEE$\downarrow$ | SIEdif$\downarrow$ |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Sub-seasonal | 7 Days | MLP Mixing | 0.0506 | 0.0117 | 0.984 | 0.981 | 1153 | 0.1265 |
| Transformer | 0.0633 | 0.0159 | 0.970 | 0.965 | 1519 | 0.2338 |  |  |
| SIFM | 0.0429 | 0.0096 | 0.987 | 0.985 | 926 | 0.0380 |  |  |
| 8 Weeks Average | MLP Mixing | 0.0689 | 0.0169 | 0.969 | 0.963 | 2222 | 0.3839 |  |
| Transformer | 0.0771 | 0.0206 | 0.970 | 0.964 | 1718 | 0.2028 |  |  |
| SIFM | 0.0625 | 0.0140 | 0.973 | 0.968 | 1600 | 0.1541 |  |  |
| Seasonal | 6 Months Average | MLP Mixing | 0.0775 | 0.0206 | 0.857 | 0.845 | 2477 | 0.3837 |
| Transformer | 0.0913 | 0.262 | 0.833 | 0.821 | 3490 | 0.4902 |  |  |
| SIFM | 0.0692 | 0.0166 | 0.917 | 0.910 | 2156 | 0.2083 |  |  |

## 5 Conclusion and Future Work

In this paper, we propose SFIM, a transformer-based sea ice foundation model that unifies multi-granularity covering from sub-seasonal to seasonal scale to enhance the sea ice concentration forecasting.
Specifically, we propose to explore the independent spatial tokens representation of SIC to exploit the inter-granularity information.
These spatial tokens will be concatenated within their own granularity and go through multi-granularity fusion to explicitly model their inter-granularity correlations.
Experiments demonstrate that our SIFM fulfills skillful forecasting in each granularity compared with single-granularity methods.
Since our SIFM is a versatile framework, the multi-granularity climate information could also be incorporated easily in future work.

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: c.f. Weaknesses
soundness: 3
strengths: The paper is mostly well written and well constructed. The motivation behind the work is clear and the different concepts well explained.
The schematic figures are very well done, and help a lot in the reading of the paper.

Overall the model seems to perform very well, and outperforms the other proposed models.
The study also performed quite a bit of ablation studies, which allows to better understand the strength and weaknesses of the approach.
summary: The paper proposes to combine different temporal aggregations of image time series (daily, weekly and 6 months averages, called granularities) as inputs for a transformer architecture, in order to model Sea Ice Concentration (SIC). The paper proposes to forecast the SIE at all the granularities as well. The argument is that these different granularities will inform the model of the different short-, and medium-term temporal dynamics (sub-seasonal and seasonal variations), without explicitly modeling them.
weaknesses: ## "Foundation" model

The term foundation model feels very misplaced in this paper, and more like a term added as an afterthought because of the current trends in the deep learning community.
Generally, foundation models are models considered to be trained on a very large amount of data, and not for a particular task. The model is then being fine tuned on different down-stream tasks for particular applications. This is not the case here, as the model is not trained on multiple tasks, but rather trained to predict multiple granularities, which is one task. Additionally, the model is suited for this particular task, but could hardly be applied to something different, given the specificity of the application.

The paper reads as if, initially, the letter "F" in Sea Ice Foundation Model, was supposed to be "Fusion", but then changed last minute to follow a trend. The model is never argued as a foundation model, and save for the name, the term is never used. The concept of fusion on the other hand, is largely covered and mentioned multiple times throughout the paper.

I would strongly suggest changing Foundation to Fusion.

## Related Work

Line 137: "temporal information inherent in sea ice modeling can not be fully exploited". I would argue that LSTM combined with CNNs can actually extract the spatio-temporal information quite well. Maybe not particularly in Sea Ice modeling, but in other fields there is a slue of models making use of the spatio-temporal information, be it transformers or combined LSTM/CNN architectures (c.f. Yu et al. 2024, Giezendanner et al. 2023, Boulila et al. 2021, etc...).

## Temporality
### Terms
The temporalities are a bit all over the place. With a paper like this, where the temporal resolution is central to understanding the work, I feel like the wording is particularly important.
Line 186 explains the granularities relatively well: daily, weekly average, and monthly average. But after that, the terms get a bit muddy:
- Line 174: "7 days, 8 weeks’ averages and 6 months’ averages". My understanding is that these are the time steps. You take 7 days of daily data as input, 8 weeks of weekly average and 6 month of monthly average. Just reading the figure, this is not clear, and I had to piece this together from the text.
- Line 245: "We propose to jointly model three granularities that cover sub-seasonal to seasonal scale, i.e., 7 days, 8 weeks averages, and 6 months averages". Here the confusion is quite big. Area you taking 8 time steps of weekly average, or an average over 8 weeks? Same for month. I think I got the gist, but this is far from clear and I had to go over the paper multiple times to piece it together.
- Table 1 (line 358): Here it gets worse. Lead Times: 7 days (daily). Intuitively I read this as the 7 days lead time, temporal resolution of 1 day, i.e. the values of the 7th day. But I'm not sure this is right, since you mention you forecast at all time steps, so is the result the error of the 7th day, or the cumulative/average error over the 7 days? Same comment for 8 weeks and 6 month.

I think the paper would profit a lot from a figure describing the temporalities. Show exactly what the inputs and outputs are, and how you define the terms used in the rest of the paper.

### Number of Time steps per granularity

I might have missed it, but it is unclear to me if the number of time steps per granularity is justified anywhere.
Why take only the seven next days? Why not the 6 months of daily inputs as a whole? The multi-granularity approach probably makes the model lighter, but a model with 6 month of daily inputs could very well perform as well as the multi-granularity one.
Alternatively, one could also model the daily output, with a moving window, and average over the other other granularities.

## Dataset

It is not totally clear to me, from section 4.1, how big the dataset is. A rough estimation of the daily data from 1978 to 2013 for the training data yields ~13'000 training samples), but how were the different temporalities sampled? Is there a moving average? Does that mean the validation set has 365 points (assuming 6 months at the begining and the end have to be blocked for the monthly average)? And the test set 2,555?

## Results

Table 1: The results for the 7 days daily lead time are better, but almost seem marginal. The table is missing units, but assuming it's percentage, is a difference in RMSE of 0.0061 really significant?
The comparisons to 45 Days and 90 Days are hard without having the output of the model. Would it be possible to run the model with 45 and 90 days of daily inputs to compare?
Additionally, an ablation study on the number of time steps per granularity would be interesting.

The results on the 6 months seem more significant.

The ablation study "Effectiveness of multi-granularity representation" is interesting, but it would also be interesting to see the results for a single granularity aggregated on the other granularity. For example: daily forecast, aggregated over 6 months.

## Conclusions

The conclusion is very minimal, and would profit from a bit more of a discussion. What advancements do the results of this model bring?

## Minor comments

Line 44 and 131: "However, numerical and statistical models usually rely on high-performance computing on CPU clusters and often lead to complex debugging processes and uncertain parameterization" and "tend to result in complex debugging processes and uncertain parameterization". That might be true, nevertheless Deep learning methods can also have large debugging processes as well as uncertain parameterizations.

## Grammar comments

Line 144: move "while" to the beginning of the sentence.
Line 146: remove "in the vision community"

## Comments on figures

Line 063: "(a)" should be bold
Line 173: It would be helpful to have the numbers mentioned in the caption reflected in the figure (e.g. (1), shared spatial encoder, (2), ... (3), ...)

Figure 6: set the value of 0 to the middle, make sure it is on the white part, and set the limits of the color bar to be the same up and down.

Figure 7: use a shared x axis for each column, and a shared y axis for each line

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 2
questions: Please address my concerns in the weakness part.
soundness: 2
strengths: 1.The paper explicitly models sea ice concentration prediction at multiple temporal scales, which enhances the understanding of temporal dynamics in forecasting.

2.This paper consistently outperforms existing deep learning approaches across various metrics, demonstrating its capability to better capture sea ice dynamics.

3.Extensive ablation experiments validate the contributions of different model components, reinforcing the effectiveness of the multi-granularity approach.
summary: This paper presents the Sea Ice Foundation Model (SIFM), a new deep learning approach for forecasting Arctic sea ice concentration (SIC). It addresses the limitations of existing methods that operate at fixed temporal granularities, which overlook important inter-granularity correlations. SIFM simultaneously models SIC at daily, weekly, and monthly scales by independently tokenizing spatial features and utilizing an attention mechanism to capture relationships across these scales. The experimental results indicate that SIFM outperforms existing deep learning models, highlighting its effectiveness in integrating diverse temporal information for improved SIC forecasting. This research contributes a valuable perspective on enhancing predictive accuracy in Arctic sea ice studies.
weaknesses: 1.The idea of the multi-granularity approach is commendable. However, it is not a ‘Foundation Model’ for SIC forecasting. A foundation model generally refers to a large parameter scale model trained on large-scale data, which is an overclaim about its effectiveness. 
    
2.The model treats the SIC forecasting problem within a general spatio-temporal analysis framework without addressing the unique characteristics of sea ice prediction. There is limited exploration of the specific oceanographic and climate science features that influence these predictions. 
    
3.After the spatial feature encoding, the model applies linear mapping to weekly and monthly features for the alignment of multi-granularity tokens. This approach may disconnect the temporal context of these tokens, raising concerns about whether time series information is preserved and how different granularity tokens are interpreted within the attention mechanism.
    
4.Although the paper includes thorough ablation studies, the comparative experiments are insufficient. The authors use many performance metrics from the original papers without providing comparisons based on the new metrics introduced. Some methods are not evaluated within a unified framework, particularly regarding daily versus varying time frames (e.g., 7, 45, 90 days), and there is a lack of comparisons at the weekly granularity.
    
5.There exists some typos confusing the understanding. For example, the abstract contains a misstatement regarding "inter-granularity," which should be "intra-granularity."
    
6.The direct flattening of features obtained from the spatial encoder raises questions about potential loss of spatial correlation information in vertical dimension, as neighboring features may lose their adjacency relationships after flattening.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 2
questions: See W1. Can the authors apply their models to general spatiotemporal forecasting tasks?
soundness: 2
strengths: 1. The authors believe that enabling a model to perform multi-granularity forecasting is important and have focused on training such a model. In the field of spatiotemporal forecasting, having a foundation model capable of making hourly, daily, and weekly predictions would indeed be valuable.

2. Compared to models in the SIC forecasting domain, the experimental results appear to show some improvement.
summary: This paper presents the Sea Ice Foundation Model (SIFM), a transformer-based approach aimed at forecasting Arctic sea ice concentration (SIC) across multiple temporal granularities, from sub-seasonal to seasonal. The model's goal is to capture both intra-granularity and inter-granularity information, potentially improving the consistency of SIC forecasts by considering correlations across different time scales. The authors argue that existing models, which are fixed to specific granularities, miss out on such cross-scale information.

However, the paper lacks a compelling justification for why multi-granularity modeling would significantly enhance performance over current single-granularity approaches, especially given that SIC fluctuations across these timescales might not always yield useful cross-correlations. Furthermore, the methodology appears to rely heavily on token-based transformer operations, but without robust evidence of how this tokenization and fusion technique meaningfully captures unique SIC-specific dynamics. While the authors report some improvement over other models, it remains unclear if this approach genuinely advances forecasting accuracy in practical, real-world settings.
weaknesses: 1. The authors focus on the SIC forecasting problem, but it is unclear whether this model is specifically tailored to SIC forecasting. It seems that the model could be applied to any spatiotemporal data with multi-granularity forecasting needs. Therefore, using only a single specialized dataset is insufficient to validate the model’s effectiveness. The authors should consider comparisons with mainstream video prediction backbones and datasets, such as SimVP: "SimVP: Simpler yet Better Video Prediction." CVPR 2022.

2. The methodology appears to rely heavily on token-based transformer operations, yet lacks strong evidence that the tokenization and fusion techniques effectively capture SIC-specific dynamics. Additionally, the structure of this paper lacks a solid theoretical foundation.

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 2
questions: Please refer to the weaknesses.
soundness: 2
strengths: The investigated problem is novel and interesting.
summary: This paper introduces the Sea Ice Foundation Model (SIFM), a deep learning approach for forecasting Arctic sea ice concentration (SIC) by leveraging multi-granularity temporal data. Unlike existing methods that predict SIC at fixed temporal scales, SIFM integrates information across short-term, sub-seasonal, and seasonal granularities, capturing crucial cumulative and inter-granular effects. By using Arctic sea ice reanalysis data, SIFM learns consistent representations that enhance forecasting accuracy. Experimental results demonstrate SIFM’s superior performance over traditional models restricted to single temporal granularities, making it a valuable model for more precise SIC predictions that benefit climate studies and Arctic ecosystem monitoring.
weaknesses: The sea ice forecasting problem is indeed a spatial-temporal forecasting task within a widely-researched area, yet the current study does not include a comparison with established spatial-temporal forecasting methods. Notably, the incorporation of multi-scale temporal data (daily and weekly) has been widely studied and should be properly cited to situate this work within prior research.

The contributions outlined in the manuscript are not yet robust enough for a top-tier submission. The first and second contributions essentially address similar aspects and should be consolidated into a single point, leaving only two clear and distinct contributions overall.

The concept of "independent spatial tokenization" requires further clarification. Specifically, what distinguishes this from existing spatial processing techniques needs to be articulated. The authors claim in line 123 that previous models utilize 2D convolutions to extract spatial features, which could also be seen as a form of spatial tokenization since patches are independently processed using a shared kernel. This raises the question of what makes the current approach distinct.

The naming conventions and visual representations between Figure 2(a) and Figure 2(b) should be unified for consistency. In Figure 2(a), "channel-wise fusion" is implemented with a 2D CNN and functions as the spatial encoder, aligning it with the spatial encoder in Figure 2(b). Additionally, while U-Net is an approach for modeling both spatial and sequential correlations, the alternative is presented conceptually as the "sequential modeling" and "spatial decoder" in Figure 2(b). This lack of consistent terminology leads to confusion regarding the model architecture.

The chosen baselines are not representative of recent advancements in time series forecasting. Several are outdated (published in 2021) or lack competitiveness, as indicated by their publication in less recognized venues. Incorporating more recent and widely recognized methods would strengthen the comparative analysis.