Adapting Vision Foundation Models for Robust Cloud Segmentation in
  Remote Sensing Images
Authors: name: Xuechao Zou; name: Shun Zhang; name: Kai Li; name: Shiying Wang; name: Junliang Xing; name: Lei Jin; name: Congyan Lang; name: Pin Tao
Published: 2024-11-20 08:37:39

Abstract: Cloud segmentation is a critical challenge in remote sensing image
interpretation, as its accuracy directly impacts the effectiveness of
subsequent data processing and analysis. Recently, vision foundation models
(VFM) have demonstrated powerful generalization capabilities across various
visual tasks. In this paper, we present a parameter-efficient adaptive
approach, termed Cloud-Adapter, designed to enhance the accuracy and robustness
of cloud segmentation. Our method leverages a VFM pretrained on general domain
data, which remains frozen, eliminating the need for additional training.
Cloud-Adapter incorporates a lightweight spatial perception module that
initially utilizes a convolutional neural network (ConvNet) to extract dense
spatial representations. These multi-scale features are then aggregated and
serve as contextual inputs to an adapting module, which modulates the frozen
transformer layers within the VFM. Experimental results demonstrate that the
Cloud-Adapter approach, utilizing only 0.6% of the trainable parameters of the
frozen backbone, achieves substantial performance gains. Cloud-Adapter
consistently attains state-of-the-art (SOTA) performance across a wide variety
of cloud segmentation datasets from multiple satellite sources, sensor series,
data processing levels, land cover scenarios, and annotation granularities. We
have released the source code and pretrained models at
https://github.com/XavierJiezou/Cloud-Adapter to support further research.