  Image classification is one of the main drivers of the rapid developments in
deep learning with convolutional neural networks for computer vision. So is the
analogous task of scene classification in remote sensing. However, in contrast
to the computer vision community that has long been using well-established,
large-scale standard datasets to train and benchmark high-capacity models, the
remote sensing community still largely relies on relatively small and often
application-dependend datasets, thus lacking comparability. With this letter,
we present a classification-oriented conversion of the SEN12MS dataset. Using
that, we provide results for several baseline models based on two standard CNN
architectures and different input data configurations. Our results support the
benchmarking of remote sensing image classification and provide insights to the
benefit of multi-spectral data and multi-sensor data fusion over conventional
RGB imagery.
