Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
Authors: Gabriel Tseng
Anthony Fuller
Marlena Reil
Henry Herzog
Patrick Beukema
Favyen Bastani
James R Green
Evan Shelhamer
Hannah Kerner
David Rolnick
Venue: ICML 2025 poster (2025)
Keywords: remote sensing
ai for good
land cover mapping
satellite data
agriculture
Primary area: applications->everything_else
TL;DR: Flexible, multimodal pretrained remote sensing models which can model phenomena at different scales

Abstract: We introduce a highly multimodal transformer to represent many remote sensing modalities - multispectral optical, synthetic aperture radar, elevation, weather, pseudo-labels, and more - across space and time. These inputs are useful for diverse remote sensing tasks, such as crop mapping and flood detection. However, learning shared representations of remote sensing data is challenging, given the diversity of relevant data modalities, and because objects of interest vary massively in scale, from small boats (1-2 pixels and fast) to glaciers (thousands of pixels and slow). We present a novel self-supervised learning algorithm that extracts multi-scale features across a flexible set of input modalities through masked modeling. Our dual global and local contrastive losses differ in their targets (deep representations vs. shallow input projections) and masking strategies (structured vs. not). Our Galileo is a single generalist model that outperforms SoTA specialist models for satellite images and pixel time series across eleven benchmarks and multiple tasks.

Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
Gabriel Tseng" 123 Anthony Fuller "4 Marlena Reil 12
Henry Herzog?
Patrick Beukema Favyen Bastani
James R. Green 4 Evan Shelhamer 456
Hannah Kerner David Rolnick 122
Abstract
ness vast quantities of unlabeled data, as is available in the case of remote sensing. However, SSL methods for RS (Jean et al., 2019; Manas et al., 2021) have been specialized to certain input modalities or shapes. such as pixel time series vs. image time series, following pioneering methods for learning from images (Chen et al., 2020; He et al., 2022) and text (Devlin et al., 2018). In nutshell, these methods create two versions ("views") of an input and pretrain mod- els to predict one view given the other. After pretraining, the learned representations can then transfer to real tasks through finetuning or reuse as features, even with limited labels or computation. We unify SSL across multiple modal- ities and input shapes used for remote sensing in practice, yielding a flexible model of both image and pixel time series.
We introduce a highly multimodal transformer to represent many remote sensing modalities- multispectral optical, synthetie aperture radar, ele- vation, weather, pseudo-labels, and more-across space and time. These inputs are useful for di- verse remote sensing tasks, such as crop mapping and flood detection. However, learning shared rep- resentations of remote sensing data is challenging, given the diversity of relevant data modalities, and because objects of interest vary massively in scale, from small boats (1-2 pixels and fast) to glaciers (thousands of pixels and slow). We present novel self-supervised learning algorithm that extracts multi-scale features across a flexible set of input modalities through masked modeling. Our dual global and local contrastive losses differ in their targets (deep representations vs. shallow input projections) and masking strategies (struc- tured vs. not). Our Galileo is single generalist model that outperforms SoTA specialist models for satellite images and pixel time series across eleven benchmarks and multiple tasks.
For spatiotemporal scale, satellite imagery encompasses objects of a variety of spatial and temporal extents. Common resolutions are 10m per pixel and acquisitions per month. Thus - unlike most natural imagery (e.g., ImageNet (Deng at, al., 2009)) or video (e.g., Kinetics-400 (Kay et al., 2017))
object in RS (such a small fishing boat) may be represented by only single pixel in RS and can be present in just a single frame (Beukema et al., 2023). Conversely, an object may be a kilometer glacier that requires tracking over decades (Baraka et al., 2020).
1. Introduction
A second challenge is presented by the number and vari- ety of sensors used in RS, where most methods have only limited flexibility in the data types which they operate. Many methods model multispectral optical (MS) data (Cong et al., 2022; Noman et al., 2024; Nedungadi et al., 2024), synthetic aperture radar (SAR) data (Wang et al., 2024b;a), or joint MS and SAR data (Fuller et al., 2024; Xiong et al., 2024), but not other modalities and not across time. Other methods model MS data across time, but no other modali- ties (Bastani al., 2023; Szwareman et al., 2024). Limiting the number and diversity of views of the Earth for learning may limit the utility and generality of the resulting repre- sentations for predictions and analysis. This could limit transfer with or without finetuning, and especially without, which may be more computationally feasible for applied and interdisciplinary practitioners.
Learning representations of large-scale and multimodal re- mote sensing and geospatial data is a long-standing scientific and practical goal. This goal is motivated by the increasing impact of machine learning (ML) and remote sensing (RS) in societally important domains (e.g. food security (Kerner et al., 2020) and disaster response (Frame et al., 2024)) where labels are expensive or difficult to acquire (Kebede et al., 2024).
Self-supervised learning (SSL) can make it possible to har-
'Equal contribution 'Equal Supervision 'Mila - Quebec Al In- stitute -MeGill University Allen Institute for AI (Ai2) *Carleton University 'University of British Columbia VVector Institute 7 "Arizona State University. Correspondence to: Gabricl Tseng <gabriel.tseng @ ammiil.cgilll.a.
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
To address both of these challenges, we propose Galileo, new family of models for multiple modalities (optical,

---

Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
Figure 1. A single Galileo model can he applied wide range of remote sensing tasks. We achieve this by training Galileo on the diversity of remote sensing modal- ities used by practitioners for different ap- plications. In addition, we train Galileo process views of these modalities used by practitioners, ranging from pixel time series to multi timestep imagery to single timestep imagery.
Mullimodal Pixel Timesories
Agricultrall Land Mapping
Galileo
Malimodal Muhi Timestop Imagery
Flood Detection
Maltimodal Single Timestep Imagery
Masine Debeis Monitoring
radar, etc.), scales, and shapes (pixel time series, image time series, single images) of remote sensing data. Our models learn multimodal, multi-scale, and flexible representations for Earth observation with SoTA downstream task results. We achieve this with (i) a novel self-supervised learning (SSL) algorithm which extends the masked data modeling framework to learn useful representations of "local" and "global" features, and (ii) globally sampled, highly multi- modal pretraining dataset which includes inputs specifically selected because of their use across diverse remote sensing tasks.
tokenization of its inputs (Sec. 2.2.1). Our masking and prediction are performed over latent tokens and not on the pixels. Our global and local objectives differ in both targets and masking, as we now detail.
Deep targets - global features; shallow targets - local features. Our target prediction occurs in the latent space, so construct target tokens by passing our target input x2 to "frozen" encoder (Sec. 2.2.3). We construct global targets by processing our target input x; with our frozen encoder. We construct local targets by processing our target input x, with a minimal linear layer to match dimensionality. Intu- itively, deeper representations are more global due to more non-local processing by attention, while shallower representations are more local due to less processing of the inputs. Galileo learns representations of both global and local features by simultaneously pretraining on deep and shallow targets.
We demonstrate Galileo's accuracy on an extensive suite of benchmarks, covering many applications, domains, and RS data types. Specifically, our Galileo-Bas -Baee model ranks first above larger RS models specialized for images, such as SatMAE (Cong et al., 2022) and CROMA (Fuller al., 2024), and with the same set of weights Galileo-Bass ranks above RS models specialized for pixel time series such as Presto (Tseng et al., 2023).
Space-time masking + global features; unstructured masking -> local features. Masking determines which to- kens visible, i.e., which are used inputs and which used as target outputs (Sec. 2.2.2). The choice of masking matters for the learned representations by governing the type and difficulty of the predictions needed. Intuitively, larger masks require larger or more global predictions, while smaller masks require smaller or more local predictions.. We thus specialize global masking to divide visible and target tokens by larger spans, with correlated "space-time" masking, and specialize local masking to divide visible and target tokens uniformly at random. Galileo learns multi- scale features by simultancously applying global structured masking (longer spans) and unstructured local masking (shorter spans) during pretraining.
2. Global, Local, Multimodal Self-Supervision
We collect large, rich dataset of highly multimodal remote sensing data specifically sampled for geographic and seman- tic diversity (Sec. 2.3). To learn rich representations of the diverse modalities in this dataset across different spatiotem- poral scales, we design a novel and highly effective SSL algorithm to simultaneously learn local and global features.
2.1. Method Intuition
Galileo learns representations via two masked data modeling objectives: our global and local tasks (Figure 2). Masked modeling takes an input x, divides it into masked "targets" x, and "visible" view x,, then predicts the masked part from the visible part. We leverage transformer architecture for masked modeling of deep and shallow representations of remote sensing data, As a transformer, this model requires
2

---

Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
Data-types supported by Galileo
Space-Timi
Space
Time
Static
S2-Red-Edge
Population
S1-SAR
S2-NIR-1Om
Elevation
Weather
Location
S2-RGB
S2-NIR-20m
Dynamic World
TerraClimate
Dynamic World (arg.)
S2-SWIR
S2-NDVI
World Cereal
Nightlights
World Cereal (org)
Structured
Unstructured
Masking
Masking
Shallow
Deep
CEKA Linear Projector
Online
Target Encoder
exIt
Online Encoder 0
Encoder 0
OEMA
Dual-objective pretraining
stop grad
stop grad
Predictor
Predictor
Global
L Local
Figure 2. We train Galileo with our global (left) and local (right) pretraining losses. Black-outlined tokens model outputs, black- striped tokens are model inputs. Steps: sample from dataset and mask (structured left, unstructured right), 2 encode "visible" tokens, predict targets given target queries and visible encodings, encode targets (deep left, shallow right) with stop gradient, and @ calculate within-sampa sample token contrastive loss.
2.2. Method Details
Apart from our custom tokenization and use of resizable lin- ear projection weights (Beyer et al., 2023) our transformer architecture remains compatible with standard ViTs. This makes it simple to use and highly flexible w.r.t. input se- quence length, the various channel groups, and other differ- ences across our various sources of data.
2.2. INPUT TOKENIZATION
Our vision transformer (ViT)-based architecture requires tokenization to convert remote sensing inputs into a se- ries of tokens. Our encoder splits the input into spatial squares, timesteps, and channel groups (related subsets of channels from a remote sensing product (c.g. one chan- nel group for the 10m channels in Sentinel-2 data)). It then projects these inputs the encoder dimension D using the following, transformations: (i) Space-time data, R xXxxWxxxx + IR -TT-GX H is the height, W is the width, P is the patch size (in pixels per side), T is the timesteps, C' are the channels, G are the channel groups. (ii) Space data, IR HXCCC + # GxD (iii) Time data, TC + GxD, and (iv) Static data, RC RGXD
2.2.2. CONSTRUCTING INPUTS VIA MASKING
Given input x, we construct "visible" view x, R and a "target" view x; E I RL For both global and local tasks, the goal is to predict the target tokens given the visible tokens. However, our masking strategies (i.e., rules that govern view construction) differ between tasks.
Global features via space and time masking. "Space masking" randomly samples tokens across space while main- taining consistency across time; "time masking" randomly samples tokens across time while maintaining consistency across space. In both cases, we select modalities to be encoded or decoded. This strategy increases the distance between visible and target tokens.
Token Embeddings. After these linear projections, our encoder creates spatial and temporal sinusoidal position embeddings, learnable channel embeddings, and month em- beddings to enable seasonal reasoning; we denote these token position embeddings as E LLD where I. is the to- ken sequence length. Our encoder adds these embeddings to the computtd linear projections. It concatenates channel groups along the sequence dimension forming input sequence, IR LDD
Local features via unstructured masking. Unstructured masking randomly samples tokens with the same probability regardless of their space, time, or channel group position. This strategy minimizes the average distance between visi-
3

---

Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
Table 1. When compared existing pretrained remote sensing models, the Galileo models are hoth the best performing and most flexible models. Performance is measured via rankings (where lower numbers are better) on image tasks in Tables 15, 16 & 17 and pixel timeseress tasks in Table For clarity, we select the best architecture per method; full rankings are available in Table 18. Flexibility is measured by documenting which imputs are supported by the models: MultiSpectral (MS). Synthetic Aperture Radar (SAR), additional Remote Sensing modalities (+modalities), inputs with spatial dimensions and inputs with more than or 4 timesteps. Galileo-Base is the best performing model compared both amage-specialinn models (c.g. CROMA) and pixel-timeseries specialized models (e.g. Presto),
Rank +
Supported Inputs
Pixel- timescries
Method
Arch.
Images
MS
SAR
+modalitics
Spatial dims
timestep
timesteps
SatMAE
ViT-Large
10.4
N/A
SaMMA+++
ViT-Large
10.9
N/A
CROMA
-ase
4.3
N/A
SoftCon
BT-B
5.9
N/A
DOFA-vI
Vir-Large
9.4
N/A
Satlas
Swin- Wiin-Tiny
12.9
N/A
MMEarth
CNN-atto
12.3
N/A
DeCUR
ViT-Small
8.3
N/A
Prithvi 2.0
ViT-Large
11.7
N/A
<<<<<
AnySat
ViT-Base
11.1
4.5
Presto
ViT-Presto
N/A
3.0
Galileo
VIT-Nano
10.9
3,5
Galileo
TiyyTnny
6.4
Galileo
ITT-Base
3.0
1.8
ble and target tokens.
2.2.4. MAKING PREDICTIONS & COMPUTING LOSSES
A predictor transformer P receives the position, time, month, and channel group embeddings e, for the target tokens and predicts patch encodings pa by cross-attending to the visible encodings, i.e., P. = P(e,, z,). Finally, the predictions Pa and targets z, are compared to compute a loss L(Pt, 2,) that updates the online encoder.
2.2.3. ENCODING VISIBLE AND TARGET TOKENS
Inputs. Our "online" encoder computes encodings for the visible tokens, Z, = Ex..). This model's parameters are updated via gradient descent.
Targets. Our "target" encoder computes encodings for the target tokens, 2, ΣEMA (x). This model's parameters are updated via computing the exponential moving average of the online encoder; this use of EMA is common in SSL, (Chen et al., 2021; Assran et al., 2023). Unlike prior work, Galileo training chooses different depths (encoder layers) for the targets depending on the task (global vs. local).
We use the "Patch Discrimination" loss (PatchDisc (Wei et al., 2024)) for both tasks, which applies the InfoNCE loss between tokens within an input:
C(u, - ΣS log E jxp
exp(sim(u,, (.c,))
with the softmax temperature т, the input index i, the token index j. the number of tokens in the 8th input L,, and the lz normalized dot product sim(u, v) = u Tu. ///|||ll||l
Global features via deep targets. We target the token rep- resentations after the hh layer, where E varies by modality. We select& é based on the level of processing for each modal- ity: pseudo-labels use only linear projections (no encoder layers), Sentinel-1 and Sentinel-2 use all encoder layers, and other channels use half the encoder layers. We denote level-specific target encoder as E ELMA-
Amplify ing local features via input-contrastive learning. We design a challenging sulf-supe task by applying PatchDisc to shallow representations of inputs by linear pro- jections of the pixels. The predictor must output tokens that are similar the pixels at the same positions but dissimilar to pixels at other positions. This differs from reconstruction methods, like MAF (He et al., 2022), which predict pixels (via the mean-squared error), but do not repel other pixels in the sequence. This differs from joint embedding methods, like LatentMIM (Wei et al., 2024) or I-JEPA (Assran et al., 2023), which target deep representations only.
Local features via shallow targets. We target the lowest representation level: the input space. For compatible di- mensionality, we compute targets using the target encoder's linear projection, EgrA which maps all tokens to the embed- ding dimension D. This strategy skips all deeper processing.
Finally, we average the global and local losses:
4

---

Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
both, or neither. A single instance consists of 24 monthly timesteps and 96 96 pixels at 10m/pixel resolution.
Space-time varying data. These data consist of imagery acquired by Sentinel-1 & -2 satellites. For Sentinel-1, we take the VV and VH polarizations; and for Sentinel-2, we take all bands except the BI, B9 and B10 bands. All bands are resampled
10m/pixel resolution. We also include NDVI (Tucker, 1979) from Sentinel-2 as an input.
Space varying data. These data consist of elevation and slope captured by the Shuttle Radar Topography Mission (NASA JPL, 2000), which are constant in time; Dynamic World land cover map probabilities (Brown et al., 2022), averaged over time for temporal consistency; and World Cereal agricultural land cover maps (Van Tricht al., 2023).
50 100 110 700 250 302
350
Figure 3. The number of training points per H3 cell (Uber, 2018) at resolution = 2. We sample from the entire globe for ge- ographic and semantic diversity (as measured by WorldCover classes (Zanaga et al., 2022)).
Time varying data. These data consist of precipitation and temperature from the ERA5 dataset (Hersbach et al., 2020); climate water deficit, soil moisture, and actual evapotran- spiration from TerraClimate (Abatzoglou et al., 2018); and VIRS nighttime lights (Elvidge et al., 2017). Although these modalities vary in space as well, their spatial resolu- tion (ERA5 has spatial resolution of tens of kilometres per pixel) means we treat them as static in space from the perspective of single instance.
Table 2. Galileo's ' combined algorithm retains the within- -nnput di- versity of our local algorithm and achieves a between-inpu diver- sity between our global and local algorithms. Diversity is measured as -
cosine similarity over the EuroSat training data.
Pretraining Within-inpul Between-input
Objective
Diversity
Diversity
Global only
0.10
0,25
Local only
0.34
0.14
Combined 0.35
0.19
Static data. These data consist of population estimates from the LandScan dataset (Dobson et al., 2000), the spatial location of the instance, defined by its central latitude and longitude, Dynamic World classes spatially averaged over the instance, and World Cereal agricultural land cover maps spatially averaged over the instance. We include the aver- aged Dynamic World and World Cereal inputs in addition to the space-varying inputs.
LGoliteo (CCtobal + Loocat)
2.2.5. MEASURING LEARNED REPRESENTATIONS
We experiment to verify our intuition about global and local tasks: the global task should encourage between- input feature diversity and our local task should encourage within-ing input feature diversity. For all EuroSat training in- puts, we measure how features differ locally within an input by computing cosine similarities of token representations E RL..D. Similarly, measure how features differ globally between inputs by computing cosine similarities of global-averaged token representations z E R Our lo- cal task amplifies within-input features, while our global task amplifies between-inple features (Table 2). We confirm these intuitions on downstream tasks in Section 4.1.
3. Experimental Framework
Pretraining. We pretrain three model sizes for 500 epochs using the algorithm described in Section 2.2. Please see the Appendix B.2 for complete details.
Downstream Tasks. We evaluate our model on all Sentinel- 2 tasks in GeoBench (Lacoste et al., 2024). These cover image classification and segmentation in various applications and geographies. We also test on fine- grained segmentation via the MADOS marine debris dataset (Kikaki et al., 2024), Sentinel-1 image segmentation via Sen Floods1 (Bonafilia et al., 2020), image time series seg- mentation via PASTIS (Garnot & Landrieu, 2021). optical pixel time series classification via Breizherops (RuBwurm et al., 2019), and multimodal pixel time series classification via CropHarvest (Tseng et al., 2021).
2.3. Galileo's Pretraining Data
We collect a large, global pretraining dataset of 127,155 instances. Figure 3 maps the training points. We include wide range of RS inputs to serve diverse applications. Each instance consists of 4 types of data covering 9 RS data modalities. We select the modalities by their uses in machine learning for remote sensing efforts (Van Tricht et al., 2023; Beukema et al., 2023; Poggio et al., 2021). Section 1. describes our full dataset sampling process.
Comparisons. We benchmark Galileo against all SoTA pretrained RS models (described in Section 5). We report results on the full test set for each task. Input normaliza-
We group the modalities by whether they vary in space, time,
5

---

Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
m-EuroSat
BetBigEarthNet
m-So2Sat
Kill
Top-I I Acc.
Fl Score
Top-I Ace. Training %
Top-I Acc.
Training %
Training %
Training %
Method
Arch.
100%
1%
100%
1%
100%
1%
100%
1%
SatMAE
ViT-Base
84.1
34.8
50.6
29.0
36.0
23.1
86.1
73.5
SatMAE++
ViT-Large
82.7
48.5
50.8
31.6
34.7
23.4
89.6
76.7
CROMA
ViT-Base
85.6
51.3
58.8
44.7
48.8
33.8
92.6
85.1
Table 3. Galileo-Bass is the best model for im- age classification (%) by ANN We show the best architecture per method. We bold and underline the I and 2nd best results across all methods and architectures, as reported in Table 15.
SoftCon
ViT-Small
89.8
27.2
64.7
43.3
51.1
31.4
89.2
77.8
DOFA-vI
ViT-Basc
82.8
49.6
49.4
29,9
41.4
29.4
88.3
78.3
Satlas
Swin-Tiny
81.7
35.8
51.9
29.6
36.6
27.1
88.2
73.0
MMFarth
CNN-atto
81.7
30.0
58.3
39.6
39.8
25.1
89.4
79.7
DeCUR
Small
89.0
46.6
63.8
49.6
45.8
30.9
83.7
74.2
Prithvi 2.0
ViT-Large
80.2
48.0
49.4
28.8
29.5
26.1
87.9
80.6
AnySat
ViT-Base
82.2
47.1
54.9
33.7
39.8
29.0
85.3
72.0
Galileo
ViT-Nano
89.7
41.7
53.8
33,9
50.1
37.4
86.7
79.7
Galileo
ViT-Tiny
90.1
41.3
55.5
34.4
49.7
36.2
86,9
77.3
Galileo
ViT-Base
93.0
56.6
59.0
36.5
54,8
43.2
90.7
78.0
m-EuroSat Top-1 Acc. Training %
E-rigEarthNeet Fl Score Training %
m-So2Sat Top-1 Acc. Training
m-Brick-Kiln
Top-I Acc. Training %
Method
Arch.
100%
1%
100%
1%
100%
1%
100%
1%
SatMAE
VTT-Large
96.6
56.9
68.3
41.8
57.2
36.4
98.4
96.1
SaLMAE++
ViT-Large
96.5
56.4
67.9
45.6
56.0
36.9
98.6
92.5
Table 4. Galileo-Base is the best model for im- age classification (%) by finetuning. We show the best architecture per method. We bold and underline the 1st and 2 best results across all methods and architectures, reported in Table 16.
CROMA
ViT-Large
96.6
52.7
71.9
47.9
60.6
40.9
98.7
96.7
SofCCon
ViT-Base
97.5
56.3
70.3
38.5
61.7
49.2
98.7
97.3
DOFA-vI
ViT-Large
96.9
53.4
68.0
43.5
58.7
37.0
98.6
94.5
Satlas
Swin-Base
97.5
51.9
72.8
25.8
61.9
30.6
98.4
94.7
MMEarth
CNN-atto
95.7
47.5
70.0
43.4
57.2
30.0
98.9
89.2
DeCUR
ViT-Small
97.9
54.2
70.9
44.7
61.7
47.0
98.7
96.9
Prithvi 2.0
ViT-Large
96.5
51.5
69.0
37.1
54.6
31.0
98.6
96.2
AnySat
ViT-Base
95.9
51.3
70.3
13.3
51.8
29.7
98.6
85.6
Galileo
VIT-Nano
94.5
52.6
67.1
23.3
57.4
34,9
96.1
94.2
Galileo
ViT-Tiny
96.9
60.6
69.7
39.5
61.9
43.1
98.7
96.6
Galileo
ViT-Base
97.7
63.5
70.7
40.9
63.3
50.6
98.7
96.8
tion, image sizes, and hyperparameter selections impact performance (Corley et al., 2024), so we therefore re-run evaluations for all comparisons and sweep normalization methods and learning rates (where appropriate). In addition, we resize all images for each model to its pretraining image size. For the image classification and segmentation tasks, we measure results across four training set sizes ("partitions"): 100%, 20%, 5%, and 1%. We use a patch size of for all models with variable patch sizes. When applying single- timestep models to the multi-timestep PASTIS dataset, we additionally sweep pooling methods to pool per-tine -er-tieettep representations. See Appendix C for complete details.
Image results. We compare Galileo to image-specialized models in Tables 3, and These models were pretrained on single-timestep imagery, devoting all their capacity to images, except for Satlas. Nonetheless, Galileo-Bar -Bse out- ranks all of them on image classification and segmentation. Our small (Na-o, Tiny} models also excel at these tasks, and often outperform much larger models, which is valuable for limited computation applications. Furthermore, Galileo's 's variable patch size can exchange computational cost and task performance: we plot this trade-off in Figure 4. By increasing the patch size, an input is split up into fewer tokens, reducing the MACs required for feature extraction.
Besides Galileo, Any is the only model to support both single-timestep images and pixel time series. Of these two generalist models, Galileo is the more accurate on standard benchmarks (by 10.8% on EuroSat for example). (Note: for semantic segmentation, the AnySat features are per-pixel instead of per-patch. For comparable training cost, we sam- ple 6. 25% of its pixel features per image when training, but evaluate with all pixel features when testing. We confirmed the fairness of this evaluation with the the AnySat authors by personal communication.)
4. Results
We present model rankings averaged across all tasks and partitions in Table We evaluate Galileo on common RS benchmarks. While these common RS benchmarks typi- cally consist of a limited set of RS modalities (optical and radar data), Galileo learns from many additional modalities during pretraining. These modalities are readily available to practitioners (Table 1, "Supported Inputs"), and may deliver improvements at test time.
Time series results. We compare Galileo to the generalist
6

---

underline the 1" and 2nd best results across all methods and architectures, as reported Table 17. The F 1 datasct con- sists of labelling floods from SAR data; mod- els which not support this modality have Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
n-CCshee-Plaan
m-SAA-Cro-Type Training S
MADOS
FenIFlo
PASTIS Training S
Truining %6
Training S
Training Sh
Method
Arch.
100%
1%
100%
1%
100%
1%
100%
1%
100%
1%
Table The Galileo models excel at image segmentation measured by % mloU via lin-
SatMAE
ViT-Laarge
30.8
22.7
24.8
16.9
55.6
13.2
NIA
29.6
11.5
SuIMAE++
VIT-Large
29.6
23.3
25.7
16.8
49.9
12.7
N/A
30.5
12.0
ear probing (Galileo-Base obtains an aver-
CROMA
VIT-Base
31.8
26.8
32.0
18.3
64.2
24.4
78.9
77.6
44.4
18.5
age rank of 2.7, Table 18). We show the
SoftCon
VIT-Base
29.6
22.8
30.8
18.5
60.3
16.5
78.0
74.8
31.3
10.5
best architecture per method. We bold and
DOFA-v1
VIT-Large
27.7
23.3
25.4
16.8
51.6
19.1
78.1
77.4
29.8
13.4
Satlas
Swin-Tiny
25.1
18.6
23.4
16.2
45.9
12.4
NIA
28.0
10.9
MMEarth
CNN-atto
24.2
20.3
14.1
34.2
16.1
N/A
24.0
10.5
DeCUR
Smal
26.2
22.8
21.5
15.3
54.8
16.6
74.5
72.2
22.4
11.0
Prithvi 2.0
ViT-Large
26.7
23.2
22.9
15.7
50.0
18.9
N/A
29.3
13.2
AnySar
VTT Base
26.1
21.7
27.1
15.8
50.2
17.0
77.9
76.9
46.2
23.5
Galileo
VIT-Nano
24.4
24.5
19.7
14.5
$4.8
13.9
78.6
77.1
17.5
13.1
Galileo
ViT-Tiny
27.4
27.9
22.5
17.1
60.8
17.5
78.0
77.9
28.1
16.9
the result replaced with N/A,
Galileo
Vi--Base
33.0
30.2
30.1
19.4
67.6
14.7
79.4
78.2
39.2
18.7
EuroSat classification (%) via ANN
Calileo (Base)
92
Table 7. Deep targets combined with structured space-time mask- ing excels in global feature extraction. Segmentation tasks are gray-ed to focus on classification with our global task. We measure % top-1 accuracy via ANN.
90
Galilco (Nano)
Galilco (Tiny)
S SanConn on Small
SoftCon Base
DÉCUR
88
masking
target enc.
loss function
MADOS Floods CropH. EuroSat
stratiggy
computation
86
CROMA I Large
space+time
varied
PatchDisc
58,91
76.92
88.72
89.50
CROMA Basc
rundom
varied
aatchDisc
11.71
69.62
82.12
17:40
SarMAR Large
random spocetteeeeetime
varied
PatchDisc
22.87
71.62
76.53
66.30
84
SaMMAE Base
space+time
O
PatchDisc
61.73
76.66
85,79
86.90
MMEarth
DOFA Large
space+time
PatchDisc
63.83
76.93
88.17
89.20
DOFA Base
space+time
12 varied varied
PatchDisc
60.35
77.19
87.30
87.90
82
Saflas Tiny
SaLMAE
AnySat
space+time
space+time
PatchDise
MSE
25.74
62.35
76.78
71.68
75.30
86.02
87.20
62.50
Satlas Base
80
Prithvi 2.0
10-1
100
10a
102
Table 8. Deep-shallow contrastive learning combined with unstruc- tured random masking excels local feature extraction. Classifica-
MACs (x109)
Figure 4. Accuracy trades off with inference cost as patch size varies {4, 8, 16]. Cost is measured as Hultippy-Accumuti operations (MACs) for encoding one EuroSat input (note the log scale on the x-axis). Galileo model size and patch size offer balance between performance and cost. See Table 14 for full results.
tion tasks are gray-ed to focus on segmentation with our local task.
We measure % mloU (t) of linear prediction on frozen features.
masking
target enc.
loss
MADOS Floods CropH. EuroSat
strategy
computation
function
random
0
PatchDise
71.48
77.39
86.77
86.90
spaccttin
0
PatchDise
68.63
77.82
85,31
88.80
space+time
O
PatchDise
62.25
77.22
86.82
87.00
random
PatchDise
$8.53
75.66
76.58
65.40
random
12
PatchDise
11.65
72.60
71.92
27.50
random
varied
PatchDise
8.25
68.89
77.83
18.40
random
0
MSE
65.34
77.09
86.71
87.40
Table 6. The Galilco models are the best (-Base) and second-best (-Tiny) models for pixel timeseries classification, measured via linear probing. The best result is bolded and the second best underlined. The CropHarvest dataset contains a number of modalities in addition to Sentinel-2 optical imagery, including topography, weather and SAR data. We use all modalities each model can support.
random
0
PatchDisca
70.12
77.26
85.27
88.20
4.1. Ablations
For all our ablation experiments, we pretrain ViT-Tiny mod- els for 200 epochs. We select four diverse tasks for segmen- tation (Sen Floodsl and MADOS), image classification (EuroSat), and time series classification (CropHarvest) us- ing only the validation sets for ablations.
CropHarvest
Method Arch.
Togo
Brazil
Kenya
Breizhcrops
Presto ViT-Presto
75.5
98.8
84.0
63.0
AnySat ViT-Base
73.4
76.7
75.5
66.1
Galileo ViT-Nano
73.5
76.4
84.5
67.3
We first ablate our global and local tasks: while the global task excels at the classification tasks and the local task ex- cels at the segmentation tasks, neither excel at both. We then ablate our combined algorithm, which excels on both the classification and segmentation tasks. We ablate the following specific components of our algorithms:
Galileo ViTyTiny
74.7
97.2
85.4
69.0
Galileo Base
74.8
99.3
84.2
73.0
AnySat and the pixel time series specialist Presto (Tab. 6). Galileo outranks Presto and far exceeds Sat.
Global task ablations. We focus on classification, which
7

---

Self-Supervised Learning. Reconstructing a masked or noisy imput is a common form of self-supervised pretrain- ing, both for natural language (Devlin et al., 2018; Radford et al., 2018; Mikolov et al., 2013) and natural imagery (Xie et al., 2022; He et al., 2022; Vincent et al., 2008). While these methods make predictions in the input space (c.g. re- constructing pixels as done by the MAE He et al. (2022)), recent work has investigated making predictions in the latent Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
5. Related Work and Background
Table Our dual-obeective algorithm excels on both classification and segmentation, and is more consistent than our single-objective algorithms. MADOS and 1 (% mloU) via linear probing. CropHarvest and EuroSat (%& top-1 acc.) via ANN.
global
local
share predictions coniext
target
MADOS Floods CropH. EaroSat
lows
loss
PaichDiscy
PatchDise
mo all
64.37
77.33
87.72
89,70
PatchDiss: PanchDise
no dec.
67.79
77.66
87.87
91.00
PatchDiscs
PatchDise
no
63.54
76.95
86.98
89,30
PatchDiss:
PanchDise
no dec.
36.98
74.21
85.49
83.30
PatchDise
PitchDise
no dec.venc.
63.41
77.36
85.87
89,30
PatchDise
PinchDise
yes all
67.04
78.23
85.23
83.50
PatchDiseg
PatchDiscg
all
67.88
77.08
86.61
space (Assran et al., 2022; Wei et al., 2024). These methods 89,50
MSE
MSE
no all
62.36
77.17
86.28
88.70
predict patch representations computed by the encoder's s exponential moving average. Galileo uniquely predics and learns at different depths of the latent space, ranging from (linear projections of) the input space to the full depth of the latent space.
our global learning is designed for (see Tab. 7). Our global task chooses per-modality depths when computing targets. slightly outperforms models that set all target depths to (half the layers) and 12 (all layers). Using only linear pro- jections for target processing drops by 2.6% on EuroSat and 2.9% on CropHarvest, confirming the importance of target- ing deeper features for classification. Using the PatchDisc loss without our local task fails, and achieves only 62.5% on EuroSat, which may potentially be caused by a shortcut exploiting position embeddings. We fix this by including tokens from other samples in the batch as negatives in the contrastive objective (our PatchDiscg). Finally, unstruc- tured random masking fails when used in our global task.
Contrastive learning (Le-Khac et al., 2020; Oord et al., 2018; Chen et al., 2020; Chopra al., 2005) is a different approach to self-supervised pre-training: it duplicates and transforms inputs, encodes them all, then attracts the representations of the same input (called positives) and repels the representa- tions of different inputs (called negatives). LatentMIM (Wei et al., 2024) applies contrastive learning to latent representa- tions of masked inputs to increase the stability of these meth- ods compared to reconstructive losses: its PatchDise loss attracts patch representations of the same location within an image, and repels patch representations in the same input but at different locations. We adopt the PatchDisc loss but observe it remains prone to collapse. Galileo's 'ss dual losses stabilize pretraining for reliable improvement of the loss.
Local task ablations. We focus on segmentation, which our local learning is designed for (see Tab. 8). Our local targets have a depth of 0, i.e., they are shallow linear pro- jections of the input without any deeper modeling. The choice of shallow targets is highly effective: it achieves 71.5% mIoU on the MADOS dataset, which contains tiny objects such as marine debris, while our global task achieves only 58.9%. Using the PatchDisc loss slightly outperforms PatchDiscg; only targeting linear projections (i.e., without position embeddings) prevents potential shortcuts without using negative tokens from the batch. These contrastive losses outperform the MSE loss by 3+% on MADOS, which demonstrates repelling the pixels from other tokens ampli- fies local features. Ours is the first use of deep-shallow contrastive learning for self-supervised learning for RS in particular and SSL in general. Finally, unstructured random masking outperforms structured masking by 9% on MA- DOS, which confirms our intuition that prediction across shorter spans promotes local features.
Pretrained RS Models. When pretraining models for re- mote sensing data, most methods process a single timestep of data, either of multispectral optical imagery only (Sat- MAE (Cong et al., 2022), MMEarth (Nedungadi et al., 2024)), multispectral optical imagery and SAR data seper- ately (SoftCon Wang et al. (2024b), DOFA Xiong et al. (2024), DeCUR Wang et al. (2024a)) or multispectral optical imagery and SAR data jointly (CROMA (Fuller et al., 2024)). Models which process multiple timesteps of data can either only process multispectral optical imagery (Prithvi 2.0 (Szwareman et al., 2024), Satlas (Bastani et al., 2023)) or ignore spatial relationships and treat the data as pixel time series (Presto (Tseng et al., 2023)). These models employ different self-supervised learning methods during pretraining; we illustrate some of them in Figure 5.
Galileo learns from and processes more modalities than these previous approaches. It can jointly process multi- spectral optical imagery and SAR imagery in addition to many other remote sensing products, including topography, weather, population maps, night-lights and land cover classi- fication maps. These products are commonly used in remote sensing tasks, and are therefore important for the utility of Galileo in a wide range of remote sensing applications.
Full algorithm ablations. Although PatchDisc essen- tial for our global task when used alone, when used with our local task it is unnecessary. Not sharing predictor pa- rameters across objectives is optimal. Interestingly, our dual-objective strategy achieves successful training runs more consistently (e.g. 100% of runs achieve >80% on EuroSat in Tab. 9 vs. 63% of runs in Tabs. and 8).
8

---

Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
La x D
Lg x D
C(af, 25)
predictor
R R(22, x2)
predictor
R R 25)
42
x2
B x
vi
B
D
22
41 D
L1 x D
L2 D
22
1a x
encoder
encoder
encoder
encoder
encoder
222
SimCI R/Barlow/Twiiee Twins style: CROMA, DeCUR, AnySat
MAE-style:
IJEPA-style: AnySat
Sa) CROMA, MMEarth, DOFA
L/ x2x1
D
44 x D
La D
R reconstructive loss
predictor
C(a x
C( 1
predictor
C(a3, x3)
eg. mean squared error
2
Fz
contrastive loss
L1 x D
Ln x
D
22
L1 D
z
La x D
2
C: e.g. InfoNCE
Lg
B batch size
encoder
encoder
encoder
encoder
L : sequence length
D feature dim
LateniMIM styll::
Galileo (ours)
latent 1/2/3: views
Figure 5. for RS. Top left: Contrastive methods attract representations originating from the same sample and repel representations from other samples. Top center: Reconstruction methods predict pixels of hidden patches. Top right: I-JEPA and AnySat predict representations of hidden patches. Bottom left: LatentMIM (which lacks an RS instantiation) attracts representations originating from the same patch and repels representations from other patches. Galileo (ours): Our method simultancously attracts varied-level representations of the same patch and repels elsewhere while it attracts pixel predictions of the same patch and repels elsewhere. This strategy encourages learning global and local features.
In addition, Galileo can flexibly model both the space and time dimensions of this multimodal data to handle inputs as single-timestep imagery, multi-timestep imagery, or pixel time series. This reflects the many multimodal, multi-shape approaches used by remote sensing practitioners (from pixel time series (Van Tricht al., 2023; Kruse et al., 2023) to single- multi- timestep imagery (Beukema al., 2023)), and allows Galileo to fit seamlessly into existing remote sensing workflows.
timodal inputs that vary in both space and time, (ii) our dual local-global SSL algorithm, to encourage the model to learn phenomena occurring at different scales, and (iii) the pretraining dataset used to train the Galileo models.
We run hundreds of evaluations including extensive sweeps of baseline pretrained RS models demonstrate Galileo's performance across wide range of domains and task types. We run thorough ablations of our method. Having confirmed the effectiveness and transfcrability of unified local, global, and multimodal self- supervised learning with Galileo, we note that more research is needed to investigate local and global self-supervision for other data beyond RS.
robustly
AnySar Sat (Astruc et al., 2024) is concurrent with our work and shares the same spirit. AnySat processes data from many satellites, and can also flexibly process the space and time dimensions of this data. However, AnySat is missing many of the other modalities processed by Galileo, which may be necessary model range of remote sensing phenomena (Poggio et al., 2021; Van Tricht et al., 2023)) and make empirical difference across our benchmarks (Table 11),
The ability process many remote sensing modalities is im- portant to remote sensing practitioners, who find that using range of inputs is critical to obtaining good results (Poggio et al., 2021; Rao et al., 2020; Van Tricht et al., 2023). This is rarely reflected in benchmark datasets, which typically only consist of optical or radar data. While many pretrained models can only process the benchmark modalities, Galileo is trained to process additional modalities that common in practice. This functionality, despite not being captured by these standard benchmarks, is valuable to practitioners who need to take full advantage of the available data.
6. Conclusion
We identify requirements for the application of pre- trained models in wide range of RS contexts: (i) the ability to flexibly process different modalities and input shapes, and (ii) the ability to model RS phenomena which occur at different scales. To meet these requirements, we present the Galilco family of pretrained RS models.
The model weights, pretraining code, pretrain- ing data and evaluation code are open sourced at github.comm/sssaarvev/ggllllll
We achieve these requirements by innovating on (i) the Galileo model architecture flexibly process highly mul-

---

Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
Impact Statement
AnySat: An Earth observation model for any resolutions, scales, and modalities. arXiv preprint arXiv:2412. 1411:2242..14/23, 2024.
Applications of machine learning to RS span a range soci- etally important applications. from species distribution mod- elling (Teng et al., 2024) to disaster management (Kansakar & Hossain, 2016). By providing set of RS models which can perform well even when few labels are available, we hope to enable RS practitioners to continue exploring and deploying these applications. We take several steps to en- courage the adoption of these models, including training the models on publicly available RS data and training a diversity of model sizes so that they can be used in compute- constrained environments. In addition, we demonstrate Galileo's performance using both computationally expen- sive transfer learning (with finetuning) and computationally cheap transfer learning (with kNN and linear probing).
Baraka, S., Akera, B., Aryal, B., Sherpa, T., Shresta, F,, Ortiz, A., Sankaran, K., Ferres, J. L., Matin, M., and Bengio, Y. Machine learning for glacier monitoring in the hindu kush himalaya. arXiv preprint arXiv:2012.05013, 2020.
Bastani, F., Wolters, P., Gupta, R., Ferdinando, J., and Kem- bhavi, A. Satlaspretrain: large-scale dataset for re- mote sensing image understanding. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 16772-16782, 2023.
Beukema, P., Bastani, F., Wolters, P., Herzog, H., and Fer- dinando. J. Satellite imagery and ai: A new era in ocean conservation, from research to deployment and impact. arXiv preprint arXiv:2312.03200 2023.
Tuia et al. (2023) highlight that a risk of these models that they can be used collect information about populations so that decisions are made without their involvement. We encourage the deployment of Galileo in collaboration with local communities and stakeholders (Krafft, 2023; Kshir- sagar et al., 2021; Nakalembe & Kerner, 2023).
Beyer, L., Izmailov, P., Kolesnikov, A., Caron, M., Korn- blith, S., Zhai, X., Minderer, M., Tschannen, M., Alab- dulmohsin, I., and Pavetic, F. Flexivit: One model for all patch sizes. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 14496-1450, 2023.
Acknowledgments
iF primarily supported by an NSERC PGS-D. DR and ES are supported by Canada CIFAR AI Chairs. GT is supported by the NSERC-CREATE, LEADS program. GT and HK are supported by the NASA Harvest Grant #8ONSSC23M0032.
Bonafili, D., Tellman, B., Anderson, T., and Issenberg, E. Sen I floodsll: A georeferenced dataset to train and test deep learning flood algorithms for sentinel-1. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 210-211, 2020.
This research was enabled in part by compute resources provided by Mila (mila.quebec), including material support from NVIDIA Corporation. We thank the Beaker team at Ai2 (Guerquin, 2022) for their support on Ai2's cluster.
Brown, C. F., Brumby, S. P., Guzder-Williams Williams, B., Birch, T.,
References
Hyde, S. B., Mazzariello, J., Czerwinski, W., Pasquarella, J., Haertel, R., Ilyushchenko, S., et al. Dynamic world, near real-time global 10 m land use land cover mapping, Scientific Data, (1):2511, 2022.
Abatzoglou, J. T., Dobrowski. S. Z., Parks, S. A., and Hegewisch, K. C. Terraclimate, a high rigg-rreolution global dataset of monthly climate and climatic water balance Scientific data, 5(1):1-12, 2018
from
Chen, T., Kornblith, S., Norouzi, M., and Hinton, G. A simple framework for contrastive learning of visual rep- resentations. In International conference on machine learning. pp. 1597-1607. PMLR, 2020.
Assran, M., Caron, M., Misra, I., Bojanowski, P., Bordes, F,, Vincent, P., Joulin, A., Rabbat, M., and Ballas, N. Masked siamese networks for label-efficient learning. In European Conference on Computer Vision, pp. 456-473. Springer, 2022.
Chen, X., Xie, S., and He, K. An empirical study of training self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 9640-9649, 2021.
Assran, M., Duval, Q., Misra, I., Bojanowski, P., Vincent, P., Rabbat, M., LeCun, Y., and Ballas, N. Self-supervised learning from images with a joint-embedding predictive architecture. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15619- 15629, 2023.
Chopra, S., Hadsell, R., and LeCun, Y. Learning sim- ilarity metric discriminatively, with application to face verification. In 2005 IEFE computer society conference on computer vision and pattern recognition (CVPR'OS) volume 1, pp. 539-55995550 546. IEFE, 2005.
Astruc, G., Gonthier, N., Mallet, C., and Landrieu, L.
10

---

Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
Cong, Y., Khanna, S., Meng, C., Liu, P., Rozi, E., He, Y., Burke, M., Lobell, D., and Ermon, S. Satmae: Pre- training transformers for temporal and multi-spectral satellite imagery. Advances in Neural Information Pro- cessing Systems, 35: 197-211, 2022.
Gorelick, N., Hancher, M., Dixon, M., Ilyushchenko, S.,
Thau, D., and Moore, R. Google earth engine: Planetary- scale geospatial analysis for everyone. Remote sensing of Environment, 002:18-27) 2017.
Guerquin, M. Introducing ai2's beaker. https:
Corley, I., Robinson, C., Dodhia, R., Ferres, J. M. L.,, and Najafirad, P. Revisiting pre-trained remote sensing model benchmarks: Resizing and normalization matters. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp- 3162-3172, June 2024.
wwtp://..r...eeee///e2e22222 htttps:/mmeiii http://wmmoomaamm///a/aaaaa aaakerr-d61175ff45555 2022. Accessed: 2025-05- 15.
Gwilliam, M. and Shrivastava, A. Beyond supervised vs. unsupervised: Representative benchmarking and analysis image representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9642-9999 2222296522222 2022.
Deng, J. Dong, W., Socher, R., L..., Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, 2009. doi: 2001 5206848.
He, K., Chen, X., Xie, S., Li, Y., Dollár, P., and Girshick, R. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 16000-16009, 2022.
Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. Bert: Pre-training of deep bidirectional transformers for lan- guage understanding. arXiv preprint arXiv:1810.04805, 2018.
Helber, P., Bischke, B., Dengel, A., and Borth, D. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Se- lected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217-23)
Dobson, J. E., Bright, E. A., Coleman, P. R., Durfee, R. C.,, and Worley, B. A. Landscan: a global population database for estimating populations at risk. Photogrammetric en- gineering and remote sensing, 66(7):849-857, 2000.
2019.
Hersbach, H., Bell, B., Berrisford, P., Hirahara, S., Horányi, A., Muñoz-Sabater, J., Nicolas, J., Peubey, C., Radu, R., Schepers, D., et al. The era5 global reanalysis. Quarterly Journal of the Royal Meteorological Society, 146(730): 1999-2049, 2020.
Elvidge, C. D., Baugh, K., Zhizhin, M., Hsu, F. C., and Ghosh, T. Viirs night-time lights. International journal of remote sensing, 38(21):5860-588 587000558790008 2017.
Hoffer, E., Ben-Nun, T., Hubara, I,, Giladi, N., Hoefler, T., and Soudry, D. Augment your batch: better training with larger batches. arXiv preprint 19Xiv:/901.000
Frame, J.M. Nair, T., Sunkara, V., Popien, P., Chakrabarti, S., Anderson, T., Leach, N. R., Doyle, C., Thomas, M., and Tellman, B. Rapid inundation mapping using the us national water model, satellite observations, and a convolutional neural network. Geophysical Research Letters, (1(1)))22044LL1094)
2019.
Jean, N., Wang, S., Samar, A., Azzari, G., Lobell, D., and Ermon, S. Tile2vec: Unsupervised representation learn- ing for spatially distributed data. In Proceedings of the AAAI Conference on Artificial Intelligence, pp. 3967- 3974, 2019.
2024.
Fuller, A., Millard, K., and Green, J. CROMA: Remote sens- ing representations with contrastive radar-optical masked autoencoders. Advances in Neural Information Process- ing Systems, 36, 2024.
Kansakar, P. and Hossain, F. A review of applications of satellite earth observation data for global societal benefit and stewardship of planet earth. Space Policy, 2016.
Gamot, V. S. F. and Landrieu, L. Panoptic segmentation of satellite image time series with convolutional temporal attention networks. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4872- 4881, 2021.
Kay, W., Carreira, J., Simonyan, K., Zhang, B., Hillier, C., Vijayanarasimhan, S., Viola, F,, Green, T., Back, T., Natsev, P., et al. The kinetics human action video dataset. arXiv preprint arXiv: / 170.006950 2017.
Kebede, E. A., Abou Ali, H., Clavelle, T., Froehlich, H. E., Gephart, J. A., Hartman, S., Herrero, M., Kerner, H., Mehta, P., Nakalembe, C., et al. Assessing and addressing the global state of food production data scarcity. Nature Reviews Earth Environment, 5(4):295-3) 311, 2024.
Garrido, Q. Assran, M., Ballas, N., Bardes, A., Najman, L., and LeCun, Y. Learning and leveraging world mod- els in visual representation learning. arXiv preprint arXiv:2403. 00504, 2024.

---

Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
Kerner, H., Tseng, G., Becker-Reshef, Nakalembe, C., Barker, B., Munshell, B., Paliyam, M., and Hosseini, M. Rapid response crop maps in data sparse regions. In ACM SIGKDD Conference on Data Mining and Knowledge Discovery Workshops, 2020.
Nedungadi, V., Kariryaa, A., Oehmcke, S., Belongie, S., Igel, C.,, and Lang, N. Mmearth: Exploring multi-modal pretext tasks for geospatial representation learning, 2024.
Noman, M., Naseer, M., Cholakkal, H., Anwer, R. M., Khan, S., and Khan, S. Rethinking transformers pre- pre-training for multi-spectral satellite imagery. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2781 1-27819, 2024.
Kikaki, K., Kakogeorgiou, I., Hoteit, I., and Karantzalos, K. Detecting marine pollutants and sea surface features with deep learning in sentinel-2 imagery. ISPRS Journal of Photogrammetry and Remote Sensing. 210:39-55 54, 2024.
Oord, A. v. d., Li, Y., and Vinyals, O. Representation learn- ing with contrastive predictive coding. arXiv preprint arXiv: (0li:v8807.37/44 2018.
Krafft, A. ASU researcher combats food insecurity with AI https//wwwwsssssuucucd2222000000 deseanche--ommbtt--ooo---i-anee 2023-09-21.
2023. Accessed:
Pedregosa, F, Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., et al. Seikit-learn: Machine learning in python. the Journal of machine Learning research, 12:2825-2830, 2011.
Kruse, C., Boyda, E., Chen, S., Karra, K., Bou-Nahra, T., Hammer, D., Mathis, J., Maddalene, T., Jambeck, J., and Laurier, F. Satellite monitoring of terrestrial plastic waste. PloS one, 18(1):0077997,, 2023.
Poggio, L., De Sousa, L. M., Batjes, N. H., Heuvelink, G. B., Kempen, B., Ribeiro, E., and Rossiter, Soilgrids 2.0: producing soil information for the globe with quantified spatial uncertainty. Soil, 7(1):117-440 2021.
Kshirsagar, M., Robinson, C., Yang, S., Gholami, S., Klyuzhin, I., Mukherjee, S., Nasir, M., Ortiz, A., Oviedo, F. Tanner, D., et al. Becoming good at ai for good. In AAAI/ACM Conference on AI, Ethics, and Society, 2021.
Lacoste, A., Lehmann, N., Rodriguez, P., Sherwin, E., Kerner, H., Lütjens, B., Irvin, J., Dao, D., Alemohammad, H., Drouin, A., al. Geo-bench: Toward foundation mod- els for earth monitoring. Advances in Neural Information Processing Systems, 36, 2024.
Radford, A., Narasimhan, K., Salimans, T., and Sutskever, I. Improving language understanding by generative pre- training. OpenAl technical reports, 2018
Rao, K., Williams, A. P., Flefil, J. F., and Konings, A. G. Sar- enhanced mapping of live fuel moisture content. Remote Sensing of Environment, 245: 11797, 2020.
Le-Kh P. H., Healy, G., and Smeaton, A. F. Contrastive representation learning: A framework and review. leee Access, 8: 113907-193923 2020.
Reed, J., Gupta, R., Li, S., Brockman, S., Funk, C., Clipp, B., Keutzer, K., Candido, S., Uyttendaele, M., and Dar- rell, Scale-mae: A scale-aware masked autoencoder for multiscale geospatial representation learning. In Pro- ceedings of the IEEE/CVF International Conference Computer Vision, pp. 4088-4099 2023.
Lee, J., Brooks, N. R., Tajwar, F., Burke, M., Ermon, S., Lobell, D. B., Biswas, D., and Luby, S. P. Scalable deep learning to identify brick kilns and aid regulatory capacity. Proceedings of the National Academy of Sciences, 1 118 (17):e20188631 18, 2021.
Rußwurm, M., Lefévre, S., and Körner, M. Breizhcrops: A satellite time series dataset for crop type identification. In Proceedings of the International Conference on Machine Learning Time Series Workshop, volume 3, 2019.
Manas, O., Lacoste, A., Giró-i Nieto, X., Vazquez, D., and Rodriguez, Seasonal contrast: Unsupervised pre- training from uncurated remote sensing data. In Pro- ceedings of the IEEE/CVF International Conference Computer Vision, pp. 9414-9423, 2021.
Sumbul, G., Charfuelan, M., Demir, B., and Markl, V. Bigearthnet: A large-scale benchmark archive for remote sensing image understanding. In IGARSS 2019-2019 IEEE International Geoscience and Remote Sensing Sym- posium, pp. 5901-5904, IEEE, 2019.
Mikolov, T., Chen, K., Corrado, G., and Dean, J. Efficient estimation of word representations in vector space, 2013. URL https:ttp/// //aa/aiivvv attpg//a////1..31.//
Nakalembe, C. and Kerner, H. Considerations for ai-eo for agriculture in sub-saharan africa. Environmental Re- search Letters, 2023.
Szwareman, D., Roy, S., Fraccaro, P., Gíslason, T. E., Blu- menstiel, B., Ghosal, R., de Oliveira, P. H., Almeida, J. L. d. S., Sedona, R., Kang, Y., et al. mritvii-co-2.01 A versatile multi-temporal foundation model for earth ob- servation applications. arXiv preprint ar arXiv:2412.02732. 2024.
NASA JPL. NASA shuttle radar topography mission global are second. NASA EOSDIS Land Processes Distributed Active Archive Center, 2000.
12

---

Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
Teng, M., Elmustafa, A., Akera, B., Bengio, Y., Radi, H., Larochelle, H., and Rolnick, D. Satbird: dataset for bird species distribution modeling using remote sensing and citizen science data. Advances in Neural Information Processing Systems, 36, 2024.
Wei, Y., Gupta, A., and Morgado, P. Towards latent masked image modeling for self-supervised visual representation learning, In ECCV, 2024.
Xie, Z.., Zhang, Z., Cao, Y., Lin, Y., Bao, J., Yao, Z.,, Dai, Q., and Hu, H. Simmim: A simple framework for masked image modeling. In Proceedings of the IEEF/CVF con- ference on computer vision and pattern recognition, pp. 1153-9663, 2022.
Tseng, G., Zvonkov, I., Nakalembe, C. I.., and Kerner, H. Cropharvest: A global dataset for crop-type classification. In fhitty-fifth Conference on Neural Information Process- ing Systems Datasets and Benchmarks Track (Round 2), 2021.
Xiong, Z., Wang, Y., Zhang, F., Stewart, A. J., Hanna, J., Borth, D., Papoutsis, I., Saux, B. L.., Camps-Valks G., and Zhu, X. X. Neural passticit-inspiree foundation model for observing the earth crossing modalities. arXiv preprint arXiv:2403. 15356, 2024.
Tseng, G., Cartuyvels, R., Zvonkov, I., Purohit, M., Rol- nick, D., and Kerner, H. Lightweight, pre-trained trans- formers for remote sensing timeseries. arXiv preprint arXiv:2304. 14065, 2023.
Yin, L., Ghosh, R., Lin, C., Hale, D., Weigl, C., Obarowski, J., Zhou, J., Till, J., Jia, X., You, N., et al. Mapping smallholder cashew plantations to inform sustainable tree crop expansion in benin. Remote Sensing of Environment, 295: 13695, 2023.
Tucker, C. J. Red and photographic infrared linear com- binations for monitoring vegetation. Remote sensing of Environment, 8(2):177-154 1979.
Tuia, D., Schindler. K., Demir, B., Camps- Valls, G., Zhu, X. X., Kochupillai, M., Džeroski, S., van Rijn, J. N., Hoos, H. H., Del Frate, F, et al. Artificial intelligence to advance earth observation: perspective. arXiv preprint arXiv:2305.08413, 2023.
Zanaga, D., Van De Kerchove, R., Daems, D., De Keers- maecker, W., Brockmann, C., Kirches, G., Wevers, J., Cartus, O., Santoro, M., Fritz, S., et al. Esa worldcover 10 m 2021 v200. ESA WorldCover Project, 2022.
Zhu, X. X., Hu, J., Qiu, C., Shi, Y., Kang, J., Mou, L., Bagheri, H., Haberle, M., Hua, Y., Huang, R., et al. So2sat lcz42: A benchmark data set for the classifica- tion of global local climate zones [software and data sets]. IEEE Geoscience and Remote Sensing Magazine, 8(3): 76-89, 2020.
Uber. H3: A hexagonal hierarchical geospatial indexing system, https://h3geo.org 2018. Accessed: 2024-1 12-044
Van Tricht, K., Degerickx, J., Gilliams, S., Zanaga, D., Bat- tude, M., Grosu, A., Brombacher, J., Lesiv, M., Bayas, J. C. L., Karanam, S., et al. Worldcereal: a dynamic open-source system for global-scale, seasonal, and re- producible crop and irrigation mapping. Earth System Science Data Discussions, 2023:1-36. 2023.
Vincent, P., Larochelle, H., Bengio, Y., and Manzagol, P.-A. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning, pp. 1096-1103, 2008
Wang, Y.. Braham, N. A. A., Xiong, Z., Liu, C., Albrecht, C. M., and Zhu, X. X. SSL4E0-S12: large-scale multimodal, multitemporal dataset for self-supervised learning in Earth observation. IEFF Geoscience and Remote Sensing Magazine, 1 11(3):98-106, 2023.
Wang, Y., Albrecht, C. M., Braham, N. A. A., Liu, C., Xiong, Z., and Zhu, X. X. Decoupling common and unique representations for multimodal self-supervised learning, arXiv preprint Stt:::23090030000 2024a.
Wang, Y., Albrecht, C. M., and Zhu, X. X. Multi-Label Guided Soft Contrastive Learning for Efficient Earth Ob- servation Pretraining, arXiv preprint MXXv:2405.22462. 2024b.
13

---

Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
A. Methodology details
A.1. The Galilco SSL algorithm
We adopt a latent prediction framework inspired by Assran et al. (2023), Garrido et al. (2024), and Wei al. (2024), which operates as follows: Given batch of samples,
construct two different views of each sample, X E IR l and X2 E R Our "online" encoder computes patch encodings 1 = Exx,), while our "target" encoder exponential moving average of the online encoder computes target patch encodings Z2 = EEMA (x²). A predictor transformer P receives the target view's position, time, month, and channel group embeddings 02 E R D as placeholder queries and predicts patch encodings E TR La D by cross-attending to the online patch encodings, i.e., p = P(e,, 21). @ The predictions p and targets Zy are compared to compute loss C(p. ze) that updates the online encoder.
Table 10. Our global task, with a varying minibatch size. By default, fits in memory on a single H100 GPU (32). We in- clude PatchDiscy results, which are equivalent using PatchDisc with minibatch size of 1.
the largest minibach size which
Minibatch size
MADOS
Floods
CropH.
FurnSat
32
58,91
76.92
88.72
89.50
16
64.24
77.76
88.51
90.10
I (PatchDiscy)
25.74
71.68
75.30
62.50
We adapt this latent prediction framework for learning local and global features. We outline those adaptations below.
1. LEARNING GLOBAL FEATURES
We design this algorithm to learn abstract, lower-frequency features suited for classification applications. View construction involves: O uniformly sampling the number of channel groups N E {2,3, .., 17). ① randomly selecting N channel groups (e.g., RGB, SAR, ERA5), @ repeating steps (a-b) for the target encoder while excluding overlapping channel groups, @ applying either spatial or temporal masking, and tokenizing both views obtain X1 and X2- Space masking samples masks across space while maintaining consistency across time; time masking does the same across time while maintaining consistency across space. In both cases, modalities are selected for encoding or decoding. @ Following our general framework, we compute 2 and p. and compute targets using varied exit depths from the target encoder, É To encourage globally discriminative representations, sampling negative examples from the batch (as opposed to only the instance). This approach ("PatchDisc ")" is defined below (note this is equivalent to PatchDiscy if the Σ, summation is removed):
extend the PatchDisc loss to better discriminate samples in batch by
v Σ L; EJ log exp(sim(u..,
Σ Σ xxpssm((u...xv...))))))
with the softmax temperature т, the sample index i, the batch size B, the token index j. the number of tokens in the ith sample Li, and the l2 normalized dot product sim(u, v) = u v/ ///uullll/// This yields the following global task:
Lgiobal PatchDisc (P(e2, E E(x,)) sgg (EÉMA (x2))
Measuring the effect of batch size within an instance for PatchDiscy. This introduces a dependency on batch size; we measure the effect of batch size Table 10.
PatchDisc samples negative patches from all instances within batch, compared to
A. 1.2. LEARNING LOCAL FEATURES
We design this algorithm to learn fine-grained higher-frequency features suited for segmentation applications. View construction involves: @ tokenizing the entire sample, and ① randomly selecting 5% of tokens for X1 and 50% for X2. O Following our general framework, we compute 2 and p. but compute targets using only the target encoder's linear projection, i.e., Egror skipping transformer blocks such that the predictor targets low-level features. 4 We use LatentMIM's ss PatchDisc loss, tasking the model to discriminate between patches on the basis of low-level features alone:
Llocal = PatchDDic(P(e22. E(x:)), sg( paooj (x2)))
1.1.3. COMBINING LOCAL AND GLOBAL OBJECTIVES
As noted in Section 2.2, our combined method alternates between the local and global objectives during pretraining:
14

---

Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
Galiiee 2 gloobbll + LLocat)
B. Pretraining details
B.1. A globally sampled pretraining dataset
To construct the Galileo dataset, we split the global WorldCover map (Zanaga et al., 2022) into 1000 x 1000 pixels (10km x 10km) tiles. For each tile, we compute two feature sets: 1 the number of pixels within each WorldCover classification class, and @ the latitude and longitude of the tile. We use these features to train a k=150,000 k-means clustering algorithm, and select the tiles closest the centroid of each cluster. This yields 150,000 training points, of which 85% (127,155) are successfully exported using Google Earth Engine (Gorelick al., 2017). By including both the pixel counts and the latitude and longitudes as features the k-mmans algorithm, ensure both the semantic and geographic diversity of the model's ss training points Figure 3 shows chloropleth map of the exported points.
We use this sampling procedure to construct a rich dataset to pretrain our model. This dataset consists of RS inputs, ranging from directly sensed inputs (such as Sentinel-2 optical imagery) to semantically dense maps (such as the Dynamic World landcover maps)- - these are discussed in detail Section 2.3. Table studies the impact of each these modalities on the model's downstream performance, by pretraining the combined global-local model while omitting single data product.
Table 11. Ablating the Galileo dataset. MADOS and Sen Floodsl (% mloU) via linear probing. CropHarvest and EuroSat (% OA) via ANN.
Removed
MADOS SenlFloods CropHarvest
EuroSat
imput
None
67.79
77.66
87.87
91.00
S1
67.67
N/A
85.27
90.20
NDVI
67.89
78.10
88.32
90.00
ERA5
68.10
77.10
87.14
91.20
TerraClim
61.30
74,90
82.78
81.20
VIRS
63.48
74.52
84.10
81.10
SRTM
66.14
77.62
86.74
91.00
SenamicWorld
67.24
77.86
87.80
89.30
WorldCereal
65.94
77.56
87.71
89.60
LandScan
60.74
77.45
87.89
91.10
Location
69.25
77.36
87.14
91.20
B.2. Implementation
All models are trained on single H100 GPUs (model sizes and training times are described in Table 12). We use an effective batch size of 512, which consists of minibatches of instances augmented and repeated times (Hoffer et al., 2019). For data augmentations,
randomly apply vertical and horizontal flipping and 90-degree rotations to each instance. When repeating the data, we first randomly select patch size P E [1, 2,3, 4, 5, 6, 7, 7,8). We then randomly select (size, timestep) combination (S, T') E [[4, 12), (5, 6), (6, 4), (7, 3), (9, 3), 3)]. We then randomly subset spatially height H = P x S, width W = P x S and timesteps T from each instance in the batch.
We use 6 precision, and the Adam optimizer with 31 = 0.9 and B2 = 0.999 with gradient clipping. We warmup our learning rate for 30 epochs to maximum learning rate before applying cooldown via a cosine decay schedule. We use exponential moving averaging (EMA) to update our target encoder with momentum value of 0.996 which linearly increases to throughout pretraining following Assran et al. (2022).
For all ablations (Section 4.1), we pretrain ViT-Tiny model for 200 epochs to maximum learning rate of 2 x 10-3 and use a weight decay of 0.02. For the final Galileo models, we pretrain the models for 500 epochs and conduct a sweep of [learning rate x weight decay]. For the ViT-Nano and ViT-Tiny architectures, we sweep learning rates E 2,3 2,3x 10-21, 2]. For the ViT-Base architecture, we sweep learning rates E [1 x 3 10 10 2 10-3, 3 10 3] and weight decays E [1 10-2 10-2,,) x 10-2 2_3 10-2)
[1x 10 3 3,2 x 10 3x 10 and weight decays E [1 10x 2, 10 2
15

---

Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
Table 12. Configurations of ViT models and associated pretraining costs. GPU-hours describes the number of GPU-hours required pretrain each model for 500 epochs on an H100 GPU,
architecture
blocks
dim
heads
params
GPU-hours
VII-Nano
4
128
0.SM
200
Tiny
12
192
5.3M
259
VIT-Base
12
768
12
85.0M
573
C. Evaluation details
C.1. Implementation
To ensure consistent experimental settings when comparing pretrained models, we rerun all evaluations under identical conditions. For the &NN probing, we follow the implementation of Gwilliam & Shrivastava (2022) - we use the pretrained models to compute representations of the test data (as values) and training data (as keys) - we then use the keys to classify the test data. Following Fuller et al. (2024) and Reed et al. (2023), we use k = 20. When linear probing, we use the pretrained models to compute representations of the training data and use this to train linear probes. We sweep learning rates when training the linear probes (f1, 3, 4, 5) 10f 10f-44-8,-2,11, and apply the trained linear probes to the computed representations of the test data. When finetuning, sweep learning rates when finetuning 3(1,3, 6} 10(-5,-4,-33 and apply the finetuned models to the test data.
C.2. Evaluation Datasets
We evaluate our models on the datasets described below. For all GeoBench-modified datasets (Lacoste et al., 2024) m- m--urosat, m-BigEarthnet, m-So2Sat, m-Brick-Kiln, m- m-Cashew-PPie -lant and m-SA-Crop-Type, we use the training, validation and test splits shared by GeoBench. In addition, we use the 1%, 5% and 20% partitions shared by GeoBench.
m-EuroSat (Helber et al., 2019): The full training set consists of 2,000 images, with 1,000 images in the validation and test sets. Images are 64 64 pixels.
m-BigFarthNet (Sumbul et al., 2019): The full training set consists of 20,000 images, with 1,000 images in the test set. Images are 120 x 120 pixels.
m-So2Sat (Zhu et al., 2020): The full training set consists of 19,992 images, with 986 images in the test set, and images are 32 x 32 pixels.
m-Brick-Kiln (Lee al., 2021): The full training set consists of 15,063 images, with 999 images in the test set. Images are 64 x 64 pixels.
m-Cashew-Plant (Yin et al., 2023): The full training set consists of 1,350 images, with 50 images in the test set. Images are 256 256; we subtile them into 64 x 64 images.
m-SA-crop-type (link): The full training set consists of 3,000 images, with 93 images in the test set. Images are 256 x 256; we subtile them into 64 64 images.
MADOS (Kikaki et al., 2024): The full MADOS dataset consists of 2,804 140 x 140 images, extracted from 174 Sentinel-2 scenes. We use the train/val/test splits from MADOS (50%/25%/255) - each split was created as a representative subset of the entire MADOS dataset. In addition, we subtile each image into 80 x 80 images.
PASTIS (Garnot & Landrieu, 2021): The full PASTIS dataset consists of 2,433 128 x 128 time series, with 38-61 timesteps per time series. We subtile each time series spatially into 64 x 64 images. In addition, we compute monthly aggregations of the time series. Garnot & Landrieu (2021) share 5 folds of the data; we use folds {1. 2,33 for training, for validation and 5 for testing. When applying single-timestep models to this dataset, we additionally sweep pooling methods to pool per-timestep encodings (as described in Section C).
Breizherops (Rußwurm et al.. 2019): The Breizhcrops dataset consists of pixel time series in 4 NUTS-3 regions in Brittany, France. We use 2 for training (FRH01, with 178,613 parcels and FRHO2 with 140,645 parcels). We use FRH03 (166,391 parcels) for validation and FRH04 (122,614 parcels) for testing. The dataset consists of variable sequence lengths; we compute monthly aggregations of the time series.
16

---

Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
CropHarvest (Tseng et al., 2021): The CropHarvest dataset consists of 3 pixel time series tasks: (i) crop vs. non crop in Togo, with 1,319 samples in the training set and 306 samples in the test set, (ii) maize vs. rest in Kenya with 1,345 samples in the training set and 1,942 m² of densely labelled pixels in the test set, and (iii) coffee vs. rest in Brazil with 794 samples in the training set and 4.2 km² of densely lablled pixels in the test set.
C.3. Comparing to baseline models
Corley et al. (2024) found that imput-image sizes and feature scaling methods can have significant impacts on the performance of pretrained RS models. We therefore resize all input images the sizes that the models were pretrained on. In addition, we treat feature scaling methods an additional hyperparameter, and sweep it in addition to the learning rates (where those are applicable, i.e. for linear probing and finetuning). Finally, the PASTIS dataset consists of multiple timesteps of optical imagery. Since all benchmark models (except AnySat) cannot process the full time series natively, we use multiple forward passes. We test a mean and a max to combine the model outputs, following Bastani et al. (2023).
The reported test results are therefore computed by sweeping the cross product of the following hyperparameters, using the validation sets in the downstream datasets:
Table 13. Galilco MADOS classification test perfor- mance (%) function of patch size measured via linear probing for different training set %s.
[Learning Rate] x [Temporal aggregations]
Arch.
patch size
100 %
20 %
5%
1%
In addition to conducting this sweep, we run the linear probes 5 times and average the results. When running the linear probe, we sweep the learning rate and feature scaling method concurrently for the first run. We select the feature scaling method from this first run, and fix it for all subsequent runs. We then select the best other hyperparameters per run, and aggregate these obtain our final results.
2
53.6
43.9
33.5
16.6
ViT-Nano
54.8
41.5
28.9
13.9
2
61.9
49.9
32,6
15.2
ViT-Tiny
4
60.8
50.6
34.0
17.5
2
68.4
53.4
39.0
18.0
ViT-Base
4
67.6
49.0
34.1
14.7
We run this sweep for all evaluation datasets with the exception of the CropHarvest tasks; these consist of small training sets and no validation sets against which the hyperparameters can be selected. We therefore follow Tseng et al. (2023) in using the same feature scaling methods as was used during pretraining, and using scikit-learn's regression algorithm with default parameters (Pedregosa et al., 2011) for all models.
C. FEATURE SCALING
The pretrained models we benchmark against apply either stan- dardization (MMEarth, DOFA, AnySat and Presto) or normal- ization (all other models) during pretraining. We sweep the following normalization statistics, either via standardization on normalization depending on the pre-training procedure: statistics from the downstream datasets, 2 SatMAE pretraining statistics, SSL4EO (Wang et al., 2023) statistics, 4 Galileo pretraining dataset statistics, Presto pretraining dataset statis- tics. For all of these statistics, we additionally sweep standard deviation multipliers. Prithvi 2.0 statistics only cover subset of Sentinel-2 bands; we therefore only include those statistics in the sweeps for the Prithvi 2.0 model.
Table 14. Galileo m-Eurosat classification test performance (%) as function of patch size measured via KNN for different training set %s. MACs required to process a single FuroSat instance are also recorded; by selecting the model size and patch size, practitioners can make trade off's between model performance and inference costs.
Arch.
patch size
GMACs
100
%
20 %
5%
1%
0.25
88.7
81.9
55.0
38.5
VIT-Nano
16
0.06
85.7
79.3
56.0
41.1
1.71
88.3
83.0
59.7
41.3
ViT-Tiyy
16
0.43
83.6
78.4
50.1
33.8
27.20
92.6
88.3
72.4
56.9
ViT-Base
D. Results
16
6.80
88.0
82.4
58.6
48.9
We include full results for the image classification tasks (Table
15) and segmentation tasks (Table 17). In addition, full results for the m-Eurosat dataset with varying patch sizes are recorded in Table 14 - these values are used in Figure 4. Similarly, we measure results for MADOS with varying patch sizes in Table 13 a patch size of 4 is used in Tables 5 and 17.
We rank the models in Table 18. When ranking the models, we compute the average rank of each model across each dataset and partition.
17

---

Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
Table 15. Image classification test performance (%) via &NN. Ranks are calculated by averaging all results and ranking the averages.
m-KuroSat Training
M-BigKarthNee
m-So2Sat
m-Brick-Kill
Top-1 Acc.
Training
Score
Training
Top-1 Acc.
Training Top-1 Acc.
Method
Arch.
100%
20%
5%
1%
100%
20%
5%
1%
100%
20%
5%
1%
100%
20%
5%
1%
SaIMAE (Cong 2022)
ViT Base
73.3
50.1
34.8
35.7
29.0
36.0
29.7
23.1
86.1
80.3
73.5
SaIMAE (Cong et 2022)
ViT-Large
84.3
74.7
53.1
46.4
50.8
42.9
35.6
27.7
36.6
34.3
31.0
24.4
87.9
84.0
80.4
74.7
SaIMAE++ (Noman et al., 2024)
ViT-Large
82.7
75.9
51.1
48.5
50.8
42.8
36.7
31.6
34.7
32.7
29.9
23.4
89.6
87.1
82.8
76.7
CROMA (Fuller
2024)
ViT
66.2
33.8
85.1
CROMA (Fuller al., 2024)
ViT-Large
86.3
78.1
59.9
49.0
56.6
50.6
44.1
38.0
47.6
45.0
43.2
33.7
91.0
86.7
82.9
80.2
SoftCon (Wang et al., 2024b)
VIT-Small
89.8
83.4
55.9
27.2
64.7
58.7
52.6
43.3
51.1
49.9
43.3
31.4
89.2
86.9
80.5
77.8
SoftCon (Wang
2024h)
54.2
35.4
74.5
DOFA-v1 (Xiong al., 2024)
ViT-Base
82.8
72.1
60.9
49.6
49.4
43.6
37.2
29.9
41.4
40.7
37.5
29.4
88.3
86.2
82.0
78.3
DOFA-v1 (Xiong ct al., 2024)
ViT-Large
83.6
72.1
53.5
41.7
49.9
41.6
35.3
27.6
45.4
40.6
35.6
31.8
86.8
85.2
84.8
80.6
Sutlas (Bastani al., 2023)
Swin-Timy
48.3
35.8
37.8
29.6
29.6
27.1
73.0
Suilas (Bastani et al., 2023)
Swin-Base
81.5
69.1
42.1
10.0
47.0
41.1
35.0
25.8
35.8
33.4
29.6
30.4
80.0
76.9
73.3
MMFarth (Nedungadi et al., 2024)
CNN-atto
81.7
73.5
60.3
30.0
58.3
52.2
46.5
39.6
39.8
38.8
36.8
25.1
89.4
K5.4
84.1
79.7
DeCUR (Wang et al., 2024a)
ViT-Small
85.3
72.3
46.6
63.8
59.2
55.4
49.6
45.8
43.1
38.5
30.9
83.7
81.7
77.9
74.2
Prithvi 2.0 (Szwareman al., 2024)
ViT-Large
69.4
54.1
48.0
49.4
35.5
28.8
29.6
26.1
87.9
83.3
80.6
AnySat (Astruc al., 2024)
Base
82.2
73.7
62.5
47.1
54.9
47.2
40.7
33.7
39.8
34.9
32.0
29.0
85.3
81.7
78.0
72.0
Galileo
VIT Nano
89.7
82.4
56.6
41.7
53.8
46.3
41.5
33.9
50.1
50.3
47.5
37.4
86.7
82.2
83.2
79.7
Galilco
ViI-Tiay
83.9
59.5
41.3
416
34.4
44.2
36.2
83.8
77.3
Galileo
ViT-Base
93.0
88.5
71.3
56.6
59.0
51.5
45.4
36.5
54.8
53.8
51.1
43.2
90.7
86.9
85.8
78.0
18

---

Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
Table 16. Image classification test performance (%) via finetuning.
m-EurooSat
m-BigEarth
m-So2Sat Training %. Top-1 Acc.
m-Brick-Ke
Training &, Top-1 Acc.
f
Training %. Fl Score
Training %. Top- Acc.
Method
Arch.
100%
20%
5%
1%
100%
20%
5%
1%
100%
20%
5%
1%
100%
20%
5%
1%
SuIMAE (Cong 2022)
2022)
ViT-Base
96.5
90.8
79.7
55.5
67.8
8
59.3
51.1
39.0
54.5
$2.0
45.2
34.8
98.5
97.4
97.0
94.0
SaIMAE (Cong et al,, 2022)
ViT-I arge
96.6
91.5
82.5
56.9
68.3
61.1
52,4
41.8
57.2
56.2
49.7
36.4
98.4
97.3
97.3
96.1
SatMA+++ (Noman et al., 2024)
ViT-Large
96.5
90.6
56.4
67.9
60.4
51.9
45.6
52.4
46.0
36.9
98.6
97.3
96.0
92.5
CROMA (Fuller 2024)
VII-Base
96.0
53.6
70.0
63.4
54.0
43.4
59.1
54.1
43.3
97.8
96.1
CROMA (Fuller al., 2024)
ViT-Large
96.6
92.9
80.7
52.7
71.9
66.0
58.3
47.9
60.6
57.9
52.9
40.9
98.7
98.0
97.1
96.7
SoftCon (Wang et 2024b)
ViT-Small
97.4
95.4
84,9
57.5
69.5
62.5
53.3
36.0
61.7
60.3
54.2
49.2
98.8
98.1
97.7
97.2
SuftCon (Wang al., 2024b)
70.3
54.2
DOFA-vI-v1 (Xiong et al., 2024)
VIT (T-Base
94.6
86.1
74.2
50.9
68.1
60.3
$1.9
41.9
56.7
49.9
45.8
33.8
98.7
97.3
96.2
95.0
DOFA-vI-v1 (Xiong et al., 2024)
ViT-Large
96.9
91.5
82.2
53.4
68.0
60.3
52.2
43.5
58.7
55.4
47.4
37.0
98.6
96.9
96.1
94.5
Satlas (Bastani et al., 2023)
Swin- Tiny
96.3
78.1
52.9
71.3
63.8
53.6
32.0
$2.7
45.9
30.8
98.5
97.7
96.8
94.7
Suilas (Bastani et al., 2023)
Swin-Base
97.5
92.2
51.9
72.8
65.1
25.8
55.0
47.0
30.6
97.9
94.7
MMFaath (Nedungadi et al. 2024)
CNN-atto
95.7
86.1
73.0
47.5
70.0
62.7
52.6
43.4
57.2
$1.0
44.1
30.0
98.9
98.1)
96.5
89.2
DeCUR (Wang et ul., 2024a)
ViT-Small
97.9
95.3
87.9
54.2
70.9
64.9
54.7
44.7
61.7
61.0
$4.2
47.0
98.7
98.0
97.1
96.9
Prithvi 2.0 (Szwarcman
2024)
VireL
96.5
50.5
AnySat (Astruc 2024)
ViT-Base
95,9
88.2
74.4
51.3
70.3
61.6
46.1
13.3
51.8
49.8
42.0
29.7
98.6
97.2
96.8
85.6
Galilco (ours)
ViT-Nano
94.5
88.3
80.2
52.6
67.1
59.3
44.1
23.3
57.
54.7
47.8
34.9
98.5
97.7
96.1
94.2
Galilco (ours)
VII-Tiny
96.9
94.4
60.6
69.7
62.2
53.4
39.5
61.9
57.2
54.9
43.1
98.7
97.9
97.2
96.6
Galileo (ours)
ViT-Base
97.7
96.0
87.0
63.5
70.7
63.1
53.9
40.9
63.3
57.8
56.7
50.6
98.7
97.5
96.8
19

---

Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
Table Image (and image timeseries) segmentation test performance (%) via linear probing. For semantic segmentation, Any outputs dense per-pixel features instead of per-patch. To keep the training-costs of the linear probes similar other models, sampled 6.25% of pixel features per image when training the linear probe for AnySat. Evaluation used all pixel features in an image.
m-Cashen- Plant Training mloU
m-SA-Cmp-TTy
MADOS
Sen1Fhod:11
PASTIS Training
Training
Training
Training %6
Method
Arch.
100%,
20/
5%
15
100%%.
20%
5%
1%
100%
20%
5%
1%
100%
20%
5%
1%
100%
20%
5%
1%
(Cong 2022)
supported Bot supported
18.5
SaMAB (Cong ct al., 2022)
ViT-Large
30.8
29.7
28.7
22.7
21.8
24.0
21,9
16.9
55.6
41.0
29.9
13.2
29.6
25.3
19.1
11.5
SatMA+++ (Noman et al., 2024)
VIT Large
29.6
28.0
27.5
23.3
15.7
24.3
21.5
16.8
49.9
38.2
27.5
12.7
nott supportd
30.5
26.0
19.3
12.0
CROMA (Paller et al.
ViT-Hase
31.4
18.3
24.4
78-9
44.4
38.4
CROMA (Faller et al. 2024)
ViT-Large
34.3
33.3
32,5
27.9
32
29.9
25.6
18.0
66.3
52.5
36.2
13.9
78.6
78.0
77.1
77.2
42.9
35.9
25.8
16.1
SoftCon (Wang et al., 2024b)
Small
27.0
26.8
25.6
23.0
28.5
27.8
24.3
17.7
$7.1
44.0
29.4
19.1
78.5
78.3
76.9
75.6
28.6
26.1
19.3
11.8
SofCon (Wang al., 2024b)
VIT-Bas
29.6
28.9
27.2
22.8
30.8
29.3
24.7
18.5
60.3
42.4
31.9
16.5
78.0
77.4
74.9
74.8
31,3
26.5
19.3
10.5
DOFA-vi (Xiong 2024)
26.7
77.8
DORAA-v (Xiong et al. 20144
VIT-I ange
27.7
27.4
27.3
23.3
25
23.9
21.3
16.8
51.6
38.5
31.0
19.1
7H.1
77.9
77.3
77.4
29.8
25.5
19.3
13.4
Sailas (Bastani al., 2023)
Swin-Tiny
25.1
24.8
24.2
18.6
23.4
22.7
19.8
16.2
45.9
35.7
26.5
12.4
28.0
24.0
17.4
10,9
(Bastani
supgorted
MMFart (Nedungadi et al., 2024)
CNN-atto
24.2
24.6
24.6
20.3
22.2
21.0
18.7
14.1
14.2
264
19.3
16.1
BOE suppored
24.0
21.6
160
10.5
DeCUR (Wang et al. 2024a)
WT.
26.2
26.2
26.0
2288
21.5
20.8
19.2
15.3
54.8
40.9
30.3
16.6
74.5
74.6
73.5
72.2
22.4
19.7
15.4
11.0
Vit-Large
suppored
Sat (Astruc et al., 2021)
VIT-Base
26.1
26.1
24.9
21.7
27.1
25.2
21,4
15.8
50.2
39.8
30.5
17.0
77.9
77.6
77.1
76.9
46,2
41.
33.7
23.5
Galilco
Namo
24.4
24.6
24.6
24.5
19.7
19.7
17.1
14.5
54.8
41.4
28.9
13.9
78.6
78.5
77.1
77.1
17.5
17.0
15.7
13.1
Gallico
Vii-Tnn
27.4
27.0
78.0
Gallico
32.8
33.1
79.4
36.7
20

---

Galileo: Learning Global & Local Features of Many Remote Sensing Modalities
Table 18. Model rankings, computed against the Image Clasification (Im. Class.) results Table 15, Image Segmentation (Im. Seg.) results in Table 17 and TimeSeries (TS) results in Table 6. We aggregate the Image Classification and Image Segmentation rankings into a single "Image" (Im.) rankings. When we do this, we average the rankings across all the tasks (as opposed to naively averaging the aggregated image classification and image segmentation rankings).
Im. Class.
Im. Seg
Method
Arch.
KNN
FT
LP
Im.
TS
SaMMAE (Cong et al., 2022)
ViT-Base
13.8
12.5
11.7
12.6
N/A
SatMAE (Cong et al., 2022)
Vir-Large
11.9
9.1
10.1
10.4
N/A
SatMAE++ (Noman al., 2024)
ViT-Large
10.9
11.4
10.4
10.9
N/A
CROMA (Fuller al., 2024)
VÝT-
3.6
7.4
2.5
4.3
N/A
CROMA (Fuller al., 2024)
ViT-Large
5.9
5.3
3.5
4.8
N/A
SoftCon (Wang et al., 2024b)
ViT-Small
5.6
4.7
7.7
6.1
N/A
SoftCon (Wang et al., 2024b)
ViT-Base
5.9
4.0
7.3
5.9
N/A
DOFA-v] (Xiong et al., 2024)
ViT-Base
9.4
13.1
9.6
10.6
N/A
DOFA-v1 (Xiong et al., 2024)
ViT-Large
10.6
10.2
7.7
9.4
N/A
Satlas (Bastani et al., 2023)
Swin-Tiny
12.7
10.6
14.9
12.9
N/A
Satlas (Bastani al., 2023)
Swin-Bann Base
15.9
7.9
15.7
13.4
N/A
MMEarth (Nedungadi et al., 2024)
CNN-atto
8.3
11.7
16.1
12.3
N/A
DeCUR (Wang et al., 2024a)
ViT-Small
7.0
3.6
13.0
8.3
N/A
Prithvi 2.0 (Szwareman al., 2024)
VīT-Large
12.0
12.5
10.8
11.7
N/A
AnySat (Astruc et al., 2024)
ViT-Base
11.1
14.5
8.3
11.1
4.5
Presto (Tseng et al., 2023)
ViT-Presto
N/A
N/A
N/A
N/A
3.0
Galileo
ViT-Nano
7.0
13.1
12.2
10.9
3.5
Galileo
ViT-Tiny
6.6
5.8
6.8
6.4
2.3
Galileo
ViT-Base
2.9
3.5
2.7
3.0
1.8
21


Reviews:
[{"review_id": "JdhQgNr8f5", "reviewer": "Reviewer_VuYX", "summary": "This paper introduces *Galileo*, a family of pretrained ViTs that flexibly encode multi-source Earth observation (EO) data of varying spatial and temporal scales for various downstream tasks. To address limitations in existing pretrained EO \"foundation models\", *Galileo* uses a self-supervised learning (SSL) recipe inspired by I-JEPA to simultaneously learn large-scale global features suitable for coarse-grained tasks like image classification and small-scale local features ideal for dense prediction tasks. The proposed approach includes a latent prediction task formulated in a contrastive manner to achieve discriminative intra-image and inter-image patch representations. Through additional techniques such as *FlexiViT*, dynamic encoder depth, and structured masking, *Galileo* achieves flexibility in input resolution, representation granularity, and available input sources. Extensive experiments demonstrate *Galileo*'s performance compared to various baselines across multiple EO tasks, including image classification, timeseries classification, semantic segmentation, and timeseries segmentation.\n\n## Update after Rebuttal\nAfter the author's response to my questions, my main concerns about the construction of the proposed method and the evaluation protocols have been resolved. In addition, given the comprehensiveness of the experiments, I raised my score to 3.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the thorough review and excellent suggestions.\n\n## Claims and Evidence\n### 1. Flexible input shapes\nFig 3 and Tab 11 show Galileo performs well across varying patch sizes. Tabs 3 and 5 show Galileo takes both pixel timeseries and images. We also run MADOS segmentation with a smaller patch size of 2 [here](https://imgur.com/a/JMEjrvb).\n\n### 2. Benchmark task\nWe agree that EuroSat alone is insufficient. We showcase SoTA results on **ten diverse datasets** incl. FloodBase Sen1Floods11 and NASA CropHarvest, created by real-world practitioners.\n\nFTW [AAAI 25] is concurrent per ICML policy. FTW cite PASTIS as a “relevant dataset for comparison\"; we evaluate on PASTIS in Tab 4.\n### 3. Baselines\nWe show that Galileo is the best on average against 16 other pretrained RS models, which in turn gain over specialized models, e.g. SatMAE outperforms a fully supervised ResNet50 (their Tab 9) and Presto outperforms a SITS Former (their Tab 6).\n\nWe run new finetuning results [here](https://imgur.com/a/NkJlUOD) with a SwinViT (Satlas) as requested. We update our rankings (Tab 14) [here](https://imgur.com/a/Dkz8Lgk) w/ finetuning: Galileo-Base has the best average rank.\n\nSatCLIP [AAAI 25 = concurrent work] is a location encoder (a function of lat/lon coordinates) while Galileo is a visual encoder (a function of image/pixel spatial/temporal inputs). SatCLIP only compares itself to other location encoders.\n### 4. Contrastive learning in the pixel space\nThis describes our use of targets from linear projections of input pixels (this work) instead of from deep representations (prior work). See Sec 2.2.2 at “Target Depth” and Fig. 2 (step 4). We thank you for highlighting the insufficient clarity of this novelty: we will edit Sec. 2.2.2 to explain. Galileo is the first model to exploit contrastive learning on _multiple depths_ in SSL - we show its effectiveness via extensive ablations (e.g. 8% gain on MADOS: Tab 7).\n### 5. Proposed loss\nPatchDisc losses outperform MSE losses by 5.4% on MADOS, 0.5% on Sen1Floods11, 1.6% on CropHarvest, and 2.3% on EuroSat (Tab 8). These controlled ablations verify the value of our loss for pretraining on remote sensing data.\n\n### \"Extension of FlexiViT and I-JEPA\"\nGalileo is not simply an extension: it introduces 1) a first-of-its-kind encoder for our set of more general inputs, 2) a new latent masked modelling method with _multiple depth_ targets, 3) a novel combination of two methods, which are themselves novel (our local and global losses). We describe these in detail [here](https://openreview.net/forum?id=gqZO3eSZRy&noteId=SQm6v8ByKG). \n\n## Questions\n1. **Local task**: Please see \"contrastive learning in the pixel space\" above.\n2. **Patch size**: Please see “flexible input shapes” above. We provide new results for patch size = 2 as an alternative to patch size = 1 ( size = 1 results in too many embeddings and excessive computation).\n3. **Galileo vs. DeCUR, CROMA**: No one model is best for all tasks, but Galileo is best on average (please see the rankings [here](https://imgur.com/a/Dkz8Lgk)). CROMA and DeCUR’s image specialization may help on m-BigEarthNet and m-Brick-Kiln. \n4. **AllDisc loss temp**: We fix the temp and will note this in Sec. 2.2.1.\n5. **Contrastive shortcuts**: Great observation. Our local loss prevents this shortcut, as position/channel/time embeddings are not added to our input projections. Our combined algorithm stabilizes training (100% of runs achieve >80% on EuroSat in Tab 8 vs. 63% in Tab 6). We emphasize these are not simply I-JEPA losses ([link](https://openreview.net/forum?id=gqZO3eSZRy&noteId=SQm6v8ByKG)).\n6. **Location inputs**: Galileo takes optional location inputs and achieves top or near-top benchmark results (Tab. 3-5) without them (only CropHarvest has location inputs); see our new result (last row) in [link](https://imgur.com/a/RJk1BAJ). We include locations because they are included in real world applications [1]. \n7. **Batch size effects**: AllDisc experiments used the largest batch size that fits in memory (a common heuristic). As requested, we run our global feature learning algorithm with a smaller batch size ([link](https://imgur.com/a/RvkPB9w)). A smaller batch size doesn't hurt performance.\n8. **Inputs and targets**: Correct, all data sources are inputs and targets for both the global and local tasks. Patch sizes are randomly sampled from the same distribution for the global and local tasks. We will edit Sec. 2.2.1 and 2.2.2 to reinforce these points.\n\n## Other comments\n1. **Hard to parse Figs 1, 2**: Thank you for highlighting this. We will edit Fig 1 to relate the (x_1, x_3) paths to the local and global tasks in Fig 2 and simplify Fig 2 by removing the grid in step 5\n2. **[terms] are used almost interchangeably**: Thank you for highlighting this potential confusion. We will standardize terms to use “local/global” for loss targets and branches vs. “shallow/deep” for target depths.\n\n[1] https://essd.copernicus.org/articles/15/5491/2023", "claims_and_evidence": "1. **Flexible input shapes**: Although FlexiVit has been validated to perform well for various patch sizes through ImageNet benchmarks, the author’s claim on input shape and patch size flexibility can be greatly improved by adding evaluations to demonstrate the model’s performance change with respect to multiple input sizes. An example of this is the Figure 4 of FlexiViT.\n2. **Benchmark Task**: Although EuroSat has been widely used in various prior works about “remote sensing foundation models,” I find this benchmark unconvincing, as a simple ResNet can easily achieve 99% accuracy [1, 2]. Evaluating Galileo against other models on a challenging and realistic benchmark, such as Fields of the World (FTW) [2], in Figure 3, can provide more convincing evidence for Galileo’s usefulness.\n3. **Baseline methods**: I recognize the authors for the extensive comparisons with prior works. However, to demonstrate the usefulness of the work, it would be helpful to compare Galileo with a few specialized models (such as ResNet and SwinViT for monotemporal tasks and TSViT [3] for timeseries tasks) trained from scratch on the evaluation task similar to GeoBench [2]. In addition, a comparison to SatCLIP [4] on certain image-level tasks would also be beneficial.\n4. **Contrastive learning in the pixel space**: As Galileo still performs patchification in the local I-JEPA pretraining task, I find the claim of being “the first SSL algorithm to perform contrastive learning in the pixel space” unsubstantiated, although a patch size of one can be randomly selected as suggested in Appendix B.2. In addition, the downstream evaluation uses “a patch size of 4 for all models with variable patch sizes”, which also does not help justify the usefulness of “contrastive learning in the pixel space.” \n5. **Effectiveness of the proposed loss**: In Table 8, PatchDisc only has marginal improvements over MSE, so I think this is a rather weak signal for practitioners to choose an I-JEPA-like paradigm against paradigms like BYOL and SimSiam in which features are matched with MSE without the overhead of introducing negative examples.\n\n[1] https://paperswithcode.com/sota/image-classification-on-eurosat\n\n[2] Lacoste, A., Lehmann, N., Rodriguez, P., Sherwin, E., Kerner, H., Lütjens, B., Irvin, J., Dao, D., Alemohammad, H., Drouin, A. and Gunturkun, M., 2023. Geo-bench: Toward foundation models for earth monitoring. Advances in Neural Information Processing Systems, 36, pp.51080-51093.\n\n[2] Kerner, H., Chaudhari, S., Ghosh, A., Robinson, C., Ahmad, A., Choi, E., Jacobs, N., Holmes, C., Mohr, M., Dodhia, R. and Ferres, J.M.L., 2024. Fields of the world: A machine learning benchmark dataset for global agricultural field boundary segmentation. arXiv preprint arXiv:2409.16252.\n\n[3] Tarasiou, M., Chavez, E. and Zafeiriou, S., 2023. Vits for sits: Vision transformers for satellite image time series. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 10418-10428).\n\n[4] Klemmer, K., Rolf, E., Robinson, C., Mackey, L. and Rußwurm, M., 2023. Satclip: Global, general-purpose location embeddings with satellite imagery. arXiv preprint arXiv:2311.17179.", "methods_and_evaluation_criteria": "The proposed method extends prior works in computer vision such as FlexiViT and I-JEPA. They make sense on a high level. The evaluation criteria follow widely recognized benchmarks in the domain.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "Yes, kNN and linear probing with varying training dataset size is a common approach to evaluate pretraining image encoders. I left other comments about the selection of downstream tasks in *Claims and Evidence*.", "relation_to_broader_scientific_literature": "Existing pretrained remote sensing models (e.g., SatMAE (Cong et al., 2022), CROMA (Fuller et al., 2024), MMEarth (Nedungadi et al., 2024), AnySat (Astruc et al., 2024)) focus on single or a specific combination of modalities or limited input shapes. Galileo explicitly designs the transformer architecture to handle varied input modalities (multispectral optical, SAR, topographic, temporal data) and input dimensions by using a customized tokenization approach that can handle arbitrary combinations of modality, temporal steps, and spatial resolutions.", "essential_references_not_discussed": "N/A", "other_strengths_and_weaknesses": "The strength of the paper lies in the extensive experiment design. The clarity of the writing, notations, and figures should also be greatly improved before being accepted for publication. I outline other weaknesses in *Claims and Evidence*, *Other Comments Or Suggestion*, and *Questions For Authors*.", "other_comments_or_suggestions": "1. I find it hard to parse Figure 1 and Figure 2 together. For example, it took me a while to figure out x1 is for the “global” task with early exit in Figure 2 and  x3 is the label for the “local” task.\n2. Terms such as \"local/global,\" \"shallow/deep,\" and \"high-level/low-level\" are used almost interchangeably. Given that the local and global features/tasks in this paper (feature abstraction levels determined by the depth of the encoding modules) are different from those in other literatures (spatial locations), consider choosing and consistently sticking to one pair of terms to avoid confusion and provide the reader with more explanation about the terms.", "questions_for_authors": "1. The authors claim that “ours is the first SSL algorithm to perform contrastive learning in the pixel space.” Does this mean that the patch size for the local pretraining task is always one? \n2. Can we use a patch size of one for segmentation tasks? Will it yield substantial improvements over a patch size of four?\n3. In Table 12, Galileo does not outperform DeCUR on m-BigEarthNet and does not outperform CROMA on m-Brick-Kiln. Does the author have any explanation or intuition for this observation?\n4. In the definition of AllDisc, is the softmax temperature learned, following a schedule, or fixed? I could not find other references to the temperature in the paper except for this definition.\n5. In the local and global I-JEPA tasks, both the predictor and the encoder receive the same (position, time, and channel group) embeddings (in the predictor cross attention and after patch embedding layers). Does this shared information create potentially shortcuts in the contrastive loss?\n6. Could the authors further justify adding the raw location of the image in training? Could it increase the risk of overfitting? Is using raw coordinates instead of functional positional encoding optimal for encoding locations? [1]\n7. With the modifications to I-JEPA, do the authors have any observations about the effect of batch size on performance since the negative examples now include patches from other training examples?\n8. Could the authors clarify which input sources are considered targets? Is the online encoder receiving the same inputs but with different patchification and masking strategies for the global and local tasks?\n\n[1] Klemmer, K., Rolf, E., Robinson, C., Mackey, L. and Rußwurm, M., 2023. Satclip: Global, general-purpose location embeddings with satellite imagery. arXiv preprint arXiv:2311.17179.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TwDHycs3Ee", "reviewer": "Reviewer_ZDMq", "summary": "This paper proposes the \"Galileo\" family of pre-trained remote sensing models, which aims to learn both global and local features to cope with the multimodal, variable input size, and large-scale span characteristics of remote sensing data. The authors improve the ViT architecture to enable the model to flexibly handle inputs from multiple sensors, and design a self-supervised learning algorithm that uses global and local objectives to learn coarse-grained and fine-grained features, respectively. The paper provides a large number of benchmark experiments, ablation studies, and comparative experiments.\n\n## update after rebuttal\nThank you for the clarifications. I will keep my score.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough review; we are glad you recognize the meaningful challenges we address with Galileo, since this was one of the primary objectives of this work.\n\n### Balancing the losses\nWe combine the global and local objectives via a simple average (Section 2.2.3), and demonstrate via ablations that Galileo can learn from this combination effectively (Table 8). There is no balancing of the losses, and no multi-objective tuning to explain. \n\nWe also measure token representation similarities in Table 2: combined retains the within-sample diversity of local and achieves between-sample diversity between global and local. The optimal diversity within or between samples is unknown and likely task-dependent; we offer these measurements to complement our benchmark evaluations.\n\n### Operations on input scale and shape / resizing + image processing\nGalileo is the first to harmonize multi-modal and multi-resolution inputs for remote sensing in this way. Relative to FlexiViT, we modified the model architecture (“Flexible input shapes”, Section 2.1.2) and the training recipe (described in detail in Appendix B.2). Remote sensing analyses may require data as pixel timeseries [1] or imagery [2] across many sizes and shapes; Galileo is the first model that can process any of these diverse inputs for a range of spectral bands and other products (e.g. weather, topography).\n\n### Lack of related research in the image frequency domain\nWe cite remote sensing papers that aim to learn both high and low frequency features, including ScaleMAE and SatMAE++ (which include in our evaluations in Tables 3 and 4). We welcome the suggestion of additional references.\n\n## Answers to questions\n\n### 1.a. Does this construction match the real world?\nYes: different real world applications of machine learning for remote sensing use very different input shapes, which reflects our subsampling. For example, Skylight [2] uses single-timestep S2 imagery or multi timestep S1 imagery, WorldCereal [1] uses highly multimodal (S2, S1, topography, weather) pixel timeseries, and Global Plastics Watch [3] uses both images and pixel-timeseries. **Galileo is the first model that can support these diverse, real world use cases**.\n\n### 1.b. Could the authors explain the scale issue semantically?\nGalileo is the first pretrained model to incorporate inputs of significantly different resolutions (e.g. ERA5 at ~30km/pixel vs. Sentinel-2 at 10m/pixel). For inputs significantly coarser than 10m/pixel, we treated them as unchanging in space (i.e. as a pixel timeseries). \n\nWe use the term “multiscale” to describe the scale of the targets (e.g. vessels, which occupy a few pixels, vs. glaciers which span kilometres). We apologize for the confusion and will clarify this in the preamble to Section 2.\n\n### 2.a. Is \"time\" important in this field?\nYes: previous work has found the temporal dimension to be critical, e.g. for agricultural land cover mapping like our PASTIS benchmark (and even more important than the spatial dimension) [4]. Many large scale mapping efforts (i.e. segmentation) model pixel timeseries to focus on the time dimension instead of the spatial dimension [2, 5, 6]. \n\n### 2.b. Are there any experiments on tasks related to time series? \nOur paper contains 3 evaluation datasets with a temporal dimension, two of which only contain the temporal dimension. Results for PASTIS (agricultural land cover segmentation) are in Table 4. Table 1 of [7] found that ignoring PASTIS’s time dimension performed significantly worse. Results for the CropHarvest and Breizhcrops pixel timeseries tasks are in Table 5. Galileo is the best or second-best method across time-series datasets. \n\n[1] https://essd.copernicus.org/articles/15/5491/2023/\n\n[2] https://arxiv.org/abs/2312.03207\n\n[3] https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0278997\n\n[4] https://arxiv.org/abs/1901.10503\n\n[5]  https://soil.copernicus.org/articles/7/217/2021/\n\n[6] https://esa-worldcover.org/en\n\n[7] https://arxiv.org/abs/2107.07933", "claims_and_evidence": "The authors claim that Galileo model can flexibly handle different modalities, inputs of different sizes, and targets of different scales.\nTo demonstrate this, the authors provide detailed comparative experiments and ablation experiments.", "methods_and_evaluation_criteria": "This part makes sense.", "theoretical_claims": "This paper proposes a pre-trained multimodal framework. The main improvements are based on the data and network structure. There is no problem with the theoretical algorithm.", "experimental_designs_or_analyses": "The experimental part is sufficient and complete.", "relation_to_broader_scientific_literature": "This paper is motivated by the input shape and scale issues of remote sensing images, which inspires the study of foundation models in remote sensing.", "essential_references_not_discussed": "This part makes sense.", "other_strengths_and_weaknesses": "Advantages:\n1. The issues and challenges that the paper focuses on are meaningful.\n2. There is a wealth of experimental support for this model.\nWeakness:\n1. The global and local pre-training objectives are simple and clear. The algorithm does not seem to be able to balance the two well, and the optimization for multiple objects is not well explained.\n2. The operations on input scale and shape seem to be resizing techniques and image preprocessing techniques.", "other_comments_or_suggestions": "The author proposed an SSL algorithm that focuses on low-frequency and high-frequency features. The introduction, methods and appendix do not reflect related research in the image frequency domain.", "questions_for_authors": "1. The author subsampled the modalities and input shapes from the dataset to construct realistic scenes. Does this construction match the real world? After that, patchification and sampling related project methods are used for different shapes and scales. The whole process seems to be downsampling the dataset and then upsampling. Could the authors explain the scale issue semantically?\n\n2. The author focuses on processing timesteps-related data. However, the paper is about classification and semantic segmentation. Is \"time\" important in this field? Are there any experiments on tasks related to time series such as change detection?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GVHW1vVDIS", "reviewer": "Reviewer_yXVW", "summary": "The paper proposes a multimodal geospatial foundation model called Galileo. The authors also propose a new joint dataset combining various modalities with temporal, spatial, and spatiotemporal variations. As the architecture is ViT based the authors also provide methods for generating patches for diverse resolutions and also for adding spatiotemporal embedding. Representation learning is done on the latent space and the pixel space. The losses are based on the patch discrimination as in LatentMIM.\nEvaluation is done on GeoBench and the method is compared to various other geospatial foundation models.\n\n## update after rebuttal\nThank you for the clarifications. I will keep my score and still opt to accept the paper.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your review, and your detailed questions. \n\n## Weaknesses\n\n### 1. Evaluation on only Sentinel 2 tasks\nWe agree that benchmark datasets over-represent Sentinel-2 based tasks, providing few opportunities to test the value of Galileo’s many modalities that reflect the wide diversity of input sensors used in the real world [1,2,3]. We benchmark on Sen1Floods11 (includes Sentinel-1 inputs), and CropHarvest (includes highly multimodal inputs, including topography and weather). Galileo excels on both of these datasets.\n\nWe ablate all of our input modalities in Table 9 and find that - even for Sentinel-2 tasks - diverse modalities significantly helps performance (e.g. MADOS gains 4% with a model trained on VIIRS night lights compared to without).\n\nThe above points show that the variety of modalities is useful, not a hindrance.\n\n### 2. Technical novelty\nOur work introduces several technical innovations for self-supervised learning in general and pretraining models for remote sensing specifically:\n\n1. We are the first to successfully fuse a diverse set of multiple sensors and products (e.g. weather) across space and time in a pretrained model. This fusion broadens Galileo's real-world applicability — as these inputs make a difference to downstream tasks [2,3,4] — and offers important empirical findings for multimodal self-supervision (e.g. the contributions of modalities to performance in Table 9). \n2. We introduce contrastive learning with _varied depth_ targets, which we show is highly effective (8% improvement on the MADOS benchmark, Table 7). Our algorithms construct target tokens from (i) linear projections of the inputs (in our local loss) and (ii) a varying number of target encoder layers (in our global loss). Galileo is the first to exploit early-exited targets in SSL - we show its effectiveness via extensive ablations.\n3. Galileo constitutes a novel combination of our dual local and global SSL losses which are each novel on their own. We demonstrate the efficacy of the algorithms in isolation (Tables 6, 7) and when combined (Table 8). \n4. A novel combination of existing methods can also constitute technical novelty. For example, CROMA [NeurIPS ‘23] combined MAE and radar-optical contrastive learning: these SSL methods, _independently_, were not novel to remote sensing or ML in general. However, its _joint_ reconstructive and contrastive multi-modal self-supervisory method was novel. \n5. We note that this work is submitted as “Application-Driven Machine Learning” as implemented by the [ICML 2025 process](https://icml.cc/Conferences/2025/ReviewerInstructions). This has different reviewing criteria including “Originality need not mean wholly novel method: It may mean a novel combination of existing methods to solve the task at hand…so as to match the needs of the user”. This is the case for Galileo and users of remote sensing in (1) the flexibility across possible inputs, (2) the accuracy across 10 diverse benchmarks, and (3) the effectiveness in the fine-tuning regime (for resource-rich groups) and kNN & linear regime (for resource-poor groups).\n\n## Questions\n\n### 1. AllDisc vs. PatchDisc\nAllDisc samples negative examples from all patches in a _batch_, as opposed to within an _instance_. AllDisc yields significant empirical gains compared to PatchDisc (e.g. a 27% improvement on EuroSat, Table 6). We will make the following change to the “Loss function” part of Sec. 2.2.1 to clarify this (updates in italics): \"To encourage globally discriminative representations, we extend the PatchDisc loss to better discriminate samples in a batch. _We achieve this by sampling negative examples from the entire batch, as opposed to within the sample._\"\n\n### 2. Maximal input tokens\nWe trained all the final Galileo models on a single GPU with 80Gb of memory, but smaller mini-batches allowed us to run many of our pretraining experiments on a consumer-grade 24Gb GPU. We agree that a large number of tokens is a challenge when incorporating many modalities, timesteps and spatial dimension: when subsampling the inputs (Appendix B.2) we selected (size, timestep) combinations so that the maximum number of input tokens was 1500. The masking percentage during training was 10% of tokens unmasked and 50% of tokens decoded; this was consistent for both the global and local branches. \n\n## Other comments\n\n### 1. Relating strategies to past papers\nThank you for this feedback. To better balance our own descriptions with relations to other work, we will first provide a self-contained summary of our method at the start of Sec. 2 (and we will move the preamble on motivations to the supplementary material to make space for this suggestion). \n\n[1] https://arxiv.org/abs/2312.03207\n\n[2] https://essd.copernicus.org/articles/15/5491/2023/\n\n[3] https://soil.copernicus.org/articles/7/217/2021/", "claims_and_evidence": "The general claims are supported by the evaluation.", "methods_and_evaluation_criteria": "The paper is evaluated on GeoBench which is the common benchmark for this type of models.", "theoretical_claims": "The paper does not contain any theoretical claims that need to be proven.", "experimental_designs_or_analyses": "The experimental design seems appropriate", "relation_to_broader_scientific_literature": "Geospatial foundation models are currently trending and as far as I could see the authors relate their work to the SOTA in the area and uses the acknowledged Benchmark for this direction.", "essential_references_not_discussed": "I am not aware of any missing paper in the area.", "other_strengths_and_weaknesses": "Strong points:\n* The proposed method achieves convincing results on Geobench.\n* The combination of modalities using varying embedding to generate uniform input token size for the ViT is elegant.\n* The paper examines if mapping latent space representation for SSL makes sense for remote sensing tasks.\n\nWeak points:\n* It is unclear whether adding the variety of modalities is useful or hindering as the models were only evaluated on Sentinal2 tasks.\n* The technical contribution is kind of using two known methods instead of just one which are alternated in training. Besides this, losses, masking strategies, and training methods are well-known from previous works in representation learning", "other_comments_or_suggestions": "The paper often relates what is done to the strategies of other papers, which sometimes makes it harder to grasp what you have actually done. As you mostly justify your choices with the experimental results presented later on anyway, it sometimes would be more helpful to describe what you have in more detail.", "questions_for_authors": "You propose AllDisc which is equivalent to PatchDisc except for averaging over the batch. I am wondering how PatchDisc computes a loss over a complete batch if not by averaging the loss. What is the exact difference for computing the loss over batch B?\n(on 2nd thought, I guess the difference is the averaging in the denominator which is either over the batch or the patches within on image)\nIf you average over the batch, do you average over all the patches in the image as well?\n\nThe combination of patches from a large selection of modalities, times, and spatial patches seems to result in rather large amounts of maximal input tokens. How many tokens were used as maximal input and what were the GPU memory requirements resulting from it?\nWhat was your masking percentage in training? Was it the same for both branches of your encoder?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "JbvaIj6j2N", "reviewer": "Reviewer_1X8T", "summary": "This work introduces a vision foundation model for remote sensing data based on self-supervised learning. The proposed method features two key technical designs: 1) A flexible encoder architecture that supports space-time, spatial, temporal, and static data. 2)A new training objectives that incorporate both global and local features to better capture representations for objects of varying scales and types. Extensive experiments conducted on multiple benchmark datasets demonstrate promising performance.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thoughtful feedback, for your attention to detail and for acknowledging the strengths of our submission.\n\n### 1. Technical Novelty\nOur work introduces several technical innovations for self-supervised learning in general and pretraining models for remote sensing specifically:\n\n1. We are the first to successfully fuse a diverse set of multiple sensors and products (e.g. weather) across space and time in a pretrained model. This fusion broadens Galileo's real-world applicability — as these inputs make a difference to downstream tasks [2,3,4] — and offers important empirical findings for multimodal self-supervision (e.g. the contributions of modalities to performance in Table 9). \n2. We introduce contrastive learning with _varied depth_ targets, which we show is highly effective (8% improvement on the MADOS benchmark, Table 7). Our algorithms construct target tokens from (i) linear projections of the inputs (in our local loss) and (ii) a varying number of target encoder layers (in our global loss). Galileo is the first to exploit early-exited targets in SSL - we show its effectiveness via extensive ablations.\n3. Galileo constitutes a novel combination of our dual local and global SSL losses which are each novel on their own. We demonstrate the efficacy of the algorithms in isolation (Tables 6, 7) and when combined (Table 8). \n4. A novel combination of existing methods can also constitute technical novelty. For example, CROMA [NeurIPS ‘23] combined MAE and radar-optical contrastive learning: these SSL methods, _independently_, were not novel to remote sensing or ML in general. However, its _joint_ reconstructive and contrastive multi-modal self-supervisory method was novel. \n5. We note that this work is submitted as “Application-Driven Machine Learning” as implemented by the [ICML 2025 process](https://icml.cc/Conferences/2025/ReviewerInstructions). This has different reviewing criteria including “Originality need not mean wholly novel method: It may mean a novel combination of existing methods to solve the task at hand…so as to match the needs of the user”. This is the case for Galileo and users of remote sensing in (1) the flexibility across possible inputs, (2) the accuracy across 10 diverse benchmarks, and (3) the effectiveness in the fine-tuning regime (for resource-rich groups) and kNN & linear regime (for resource-poor groups).\n\n### 2. Technical Distinction between AllDisc and PatchDisc \nAllDisc samples negative examples from all patches in a _batch_, as opposed to within an _instance_. AllDisc can yield empirical gains compared to PatchDisc (e.g. 27% improvement on EuroSat, Table 6). We agree that AllDisc is a simple modification of PatchDisc. This simple modification that gives significant improvement is a strength of our method. We will update Section 2.2.1 to clarify this distinction. \n\n### 3. The rationale behind the choice of PatchDisc and AllDisc \nWe train Galileo with PatchDisc for local and global learning (as discussed in Section 2.2.3), but we observe significant improvements when using AllDisc for global learning (Table 6), so we include both in the text. We thank the reviewer for the feedback on the choice of PatchDisc vs. AllDisc in the final combined method, and we will make this clearer in the method section. \n\n### 4. Masking Strategy and Depth Details\nThank you for your attention to these important details. While space-time masking is best when learning global features (Table 6), random masking is best when learning local features (Table 7). Note that the target encoder depth also matters: prior work always used the full encoder to construct targets, but we find that a target depth of 0 is best for learning local features, and depth that varies per modality is optimal for learning global features. We are the first to vary target depth in this way, and the first to show its importance for remote sensing. We will clarify the use of random masking in Section 2.2.2 per this comment.\n\n### 5. Performance and Complexity vs. CROMA\nGalileo-Base outperforms CROMA-Base on image tasks (Table 1) and is less complex (CROMA-Base has 60% more parameters than Galileo-Base). Galileo is architecturally simpler than CROMA, which requires 3 encoders to process a Sentinel-1&2 image pair (an MS optical encoder, a SAR encoder, and a fusion encoder). Galileo leverages a single encoder to process inputs across space, time, spectral bands, modalities, etc. Re: dataset size, CROMA was pretrained on SSL4EO with 1M samples of 264x264 pixel images while Galileo was pretrained on 127K samples of 96x96 pixels at 24 timesteps. In total, CROMA was pretrained on >2x as many Sentinel-2 pixels, so Galileo’s performance is not explained by dataset size.", "claims_and_evidence": "The claimed contributions are well-supported by the proposed method and experimental results.", "methods_and_evaluation_criteria": "The method combines the strengths of contrastive learning and masked image modeling to learn features at both token and pixel levels. The approach is technically sound, and the evaluation metrics are reasonable.", "theoretical_claims": "No theoretical proof in the paper.", "experimental_designs_or_analyses": "The literature review appears comprehensive, and I am quite familiar with the topic.", "relation_to_broader_scientific_literature": "Vision foundation models are of great interest to people in remote sensing and computer vision.", "essential_references_not_discussed": "No. The literature review looks comprehensive.", "other_strengths_and_weaknesses": "Strengths:\n1) The idea of building a unified foundation model for diverse types of remote sensing data is compelling. Previous attempts have focused on unifying models from the perspective of spatial, temporal, or spectral characteristics. This work integrates several recent advances to create a more comprehensive and unified foundation model.\n\n2) The experimental section is thorough and well-executed. Extensive experiments across multiple benchmarks effectively demonstrate the efficacy of the proposed method.\n\nWeaknesses:\n1) The proposed method combines training objectives from contrastive learning and masked image modeling with only minor modifications. While effective, the technical novelty is somewhat limited due to this straightforward and intuitive combination.\n\n2) The technical distinction between PatchDisc and the proposed AllDisc is minimal.\n\n3) The rationale behind the choice of PatchDisc and AllDisc is unclear. In Section 2.2.3, the authors state, \"PatchDisc outperforms AllDisc when combining global and local objectives, so we use PatchDisc for both objectives.\" If this is the case, the necessity of introducing AllDisc in Section 2.2.1 becomes questionable. Additionally, Table 7 shows that the performance difference between the two is marginal.\n\n4) The masking strategy is frequently mentioned in the experimental section but is less emphasized in the method description. If space-time masking consistently performs best, its discussion in the results section may be redundant. I also recommend removing discussions on target encoder exit depth in some tables, as they may distract readers from the main findings.\n\n5) From Tables 3 and 4, it appears that the proposed method does not outperform CROMA on several benchmarks. Given that the proposed method is significantly more complex than CROMA and likely trained on a larger dataset, the performance gains do not seem substantial enough to justify the added complexity.", "other_comments_or_suggestions": "No", "questions_for_authors": "See the weakness part.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces Galileo, a family of trained ViTs that flexibly encode multi-source Earth observation (EO) data of varying spatial and temporal scales for various downstream tasks. Extensive experimental evaluations have been conducted against baselines on multiple datasets. The main contribution is a pre-trained model for remote sensing datasets in various tasks. The authors' rebuttal has addressed reviewers' main concerns in model evaluation and comparison with related works. The authors are suggested to incorporate these revisions.
Decision: Accept (poster)