###### Abstract  
In order to solve the problem of manual labeling in semi-supervised tree species classification, this paper proposes a pixel-level self-supervised learning model named M-SSL (multisource self-supervised learning), which takes the advantage of the information of plenty multisource remote sensing images and self-supervised learning methods. Based on hyperspectral images (HSI) and multispectral images (MSI), the features were extracted by combining generative learning methods with contrastive learning methods. Two kinds of multisource encoders named MAAE (multisource AAE encoder) and MVAE (multisource VAE encoder) were proposed, respectively, which set up pretext tasks to extract multisource features as data augmentation. Then the features were discriminated by the depth-wise cross attention module (DCAM) to enhance effective ones. At last, joint self-supervised methods output the tress species classification map to find the trade-off between providing negative samples and reducing the amount of computation. The M-SSL model can learn more representative features in downstream tasks. By employing the feature cross-fusion process, the low-dimensional information of the data is simultaneously learned in a unified network. Through the validation of three tree species datasets, the classification accuracy reached 78%. The proposed method can obtain high-quality features and is more suitable for label-less tree species classification. self-supervised learning; tree species classification; multisource remote sensing images +
Footnote †: journal:  
computer vision, the potential of forest remote sensing is not yet realized. Until now, there was no record in the literature of multisource remote sensing images used to classify tree species with the self-supervised learning method, especially HJ-1A and Sentinel-2 images. Therefore, it is a challenge to study tree species classification based on self-supervised learning for multisource forest remote sensing images. Self-supervised learning can extract effective information from unlabeled samples [16; 17; 18; 19; 20], and tree species classification can be accomplished by extracting features from multi-source remote sensing images through self-supervised learning methods. Self-supervised learning methods were studied as an extended taxonomy: generative, contrastive, and joint methods. Generative learning aims to map the training data to a particular distribution pattern [21]. According to the input distribution, beneficial features or fake sample extractors similar to the actual data are obtained. For HSI/MSI classification, stacked sparse autoencoder (SSAE) uses the autoencoder to extract sparse spectral features and multiscale spatial features. 3D convolution autoencoders (3 DCAE) and CapsNet are assisted by GAN (TripleGAN) [22; 23]. The contrastive learning method aims to distinguish from different samples. After training by comparing different samples, the discriminator is employed in image classification tasks [24]. The enhanced views of the same input are embedded next to each other while trying to separate embeddings from different inputs. Adding all features generated in different ways from the same class is known to improve the similarity between them, which tends to cluster their representation in the feature space so that the computation cost is less than generative learning methods. The cascade network and the contrastive loss method are often used in contrast learning [25], and the difference between them lies mainly in their different ways of collecting negative samples. SimCLR [26] uses a large number of different scales data for training to obtain a contrast loss value more representative of the image domain. MoCo [27] uses contrastive learning as the dictionary lookup, where dynamic dictionaries with queues and average-shift encoders are constructed to make the large and consistent dictionaries to be built for facilitating contrast unsupervised learning. SimSiam [28] collects negative samples by reducing the batch size.