MAESTRO: Masked AutoEncoders for Multimodal, Multitemporal, and
  Multispectral Earth Observation Data
Authors: name: Antoine Labatie, name: Michael Vaccaro, name: Nina Lardiere, name: Anatol Garioud, name: Nicolas Gonthier
Organization: Institut national de l'information géographique et forestière

Self-supervised learning holds great promise for remote sensing, but standard
self-supervised methods must be adapted to the unique characteristics of Earth
observation data. We take a step in this direction by conducting a
comprehensive benchmark of fusion strategies and reconstruction target
normalization schemes for multimodal, multitemporal, and multispectral Earth
observation data. Based on our findings, we propose MAESTRO, a novel adaptation
of the Masked Autoencoder, featuring optimized fusion strategies and a tailored
target normalization scheme that introduces a spectral prior as a
self-supervisory signal. Evaluated on four Earth observation datasets, MAESTRO
sets a new state-of-the-art on tasks that strongly rely on multitemporal
dynamics, while remaining highly competitive on tasks dominated by a single
mono-temporal modality. Code to reproduce all our experiments is available at
https://github.com/ignf/maestro.