dataesr/scientific-paragraphs-categorization huggingface dataset:
An increasing availability of remote sensing data in the era of geo big-data makes producing well-represented, reliable training data to be more challenging and requires an excessive amount of human labor. In addition, the rapid increase in graphics processing unit (GPU) processing power has enabled the development of advanced deep learning (DL) algorithms, which achieve impressive results in the field of satellite image processing. However, they require a huge and comprehensive training dataset to avoid overfitting problems and to represent a generalizable model. Thus, moving toward the development of non-supervised deep learning (NSDL) models in different remote sensing applications is an inevitable need. To provide an initial response to that need, this paper performs a comprehensive review and systematic meta-analysis of recently published research articles focusing on the applications of NSDL for remote sensing data processing.
"geo big data". Therefore, developing processing tools to effectively use remote sensing dataset have been always one of the hot topics in the research community. On the one hand, the main processing tasks can be divided into classification, object detection, change detection (CD), and image fusion. On the other hand, many remote sensing applications require preprocessing steps, such as noise reduction, data fusion, image registration, and super-resolution (SR). Hence, diverse methods from statistical parametric approaches to machine learning (ML) and recent deep learning (DL) methods have been developed or evaluated to address remote sensing data processing challenges  [1]. Despite the significant improvements of DL algorithms on various remote sensing processing tasks, they require extensive labeled training data.
In other words, the excellence of DL models is highly dependent on the quality and amount of training samples. However, acquiring high-quality annotations on a large scale and in some applications could be laborious and costly. In particular, for earth observation (EO) applications, several field works are required to collect proper training samples. Additionally, the large volume and variety of remote sensing datasets and the number of input data layers directly increase the requirement of training samples due to the Hughes phenomenon in higher dimensions  [2]. Furthermore, some imaging scenes, such as complex and heterogeneous landscapes, could add various challenges and thus, affecting the generalization capability of the DL model due to their mixed distribution. To sum up, DL models' performance and generalization capability extremely relies on the large-scale training samples.
Despite annotated training data, unlabeled data are more accessible and easier to collect. Accordingly, exploiting unlabeled data in order to avoid expensive training data collection and achieving the highest performance of DL models can be considered an attractive solution. Hence, non-supervised > REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) < learning methods can be adapted to maximize the applicability of unlabeled data for training the DL model. However, due to the inherent complexity of remote sensing data and existing challenges in non-supervised learning, precise learning schemes and specific model designs are required. This paper provides a meta-analysis and a systematic review of the latest deep non-supervised learning methods offered by researchers in the field of remote sensing.
There are various approaches for categorizing non-supervised learning methods. However, this study divides them into five main categories: transfer learning (TL) or fine-tuning, weakly supervised learning (WSL), semisupervised learning (Semi-SL), self-supervised learning (Self-SL), and unsupervised learning (USL). While considering that supervised learning is performed in an ideal condition with abundant and accurate labeled samples, the main capability of non-supervised learning is leveraging unlabeled or weakly labeled data to improve the trained model's generalization. TL addresses the problem of limited training data and accelerates the learning process by using a pre-trained model from another domain and fine-tuning it with labeled samples from the main dataset. Unlike TL, other learning methods train a base model without pre-trained weights.
The key idea behind WSL is training a model with low quality of training data, while the quantity can be sufficient, such as, semantic segmentation with image-level labeled data. Semi-SL algorithms, generally, take advantage of unlabeled data to increase the size of training dataset by iteratively generating pseudo samples. Self-SL uses unlabeled data for unsupervised representation learning and then fines-tunes the model by small number labeled data. Despite these learning methods, USL fully utilizes unlabeled data for modelling. Several review papers have been published concerning DL applications in remote sensing data processing. However, these studies mainly focus on a few topics, such as general applications of the DL models to remote sensing data  [3] ,  [4] , reviewing a specific DL architecture  [5] -  [7] , or a specific application or dataset  [8] ,  [9].
Hence, there is no attention on either learning methods or training strategies in most of these studies. One of the very first comprehensive reviews of DL in remote sensing was published by Zhu et al. in 2017  [4]. They reviewed well-known models, applications, and benchmark remote sensing datasets for developing DL algorithms. Ma et al. (2019)  gathered a systematic review and meta-analysis of DL in various remote sensing applications and reviewed related papers in the topic, created a meta-analysis by categorizing the major inter-connections between DL and remote sensing data, such as DL architecture, study target, and data resolution  [3]. Hoeser et al. (2020)  published a comprehensive overview of DL models focusing on object detection and segmentation using remotely sensed images  [10] ,  [11].
II. METHOD A comprehensive review process in this paper was performed by preparing a systematic literature search query using the SCOPUS, IEEEXplore, and Science Direct databases, which cover almost all the well-known scholarly literature in the field of remote sensing. Preferred Reporting Items for Systematic > REPLACE THIS LINE WITH YOUR MANUSCRIPT ID NUMBER (DOUBLE-CLICK HERE TO EDIT) < Reviews and Meta-Analyses (PRISMA)  [20]  methodology was followed for the selection of articles. Accordingly, after a few trials, three groups of keywords, including the "Field of study", "Learning method", and "Modeling method", were selected to extract relevant research articles from the databases. Thus, the query design for searching the databases is as follows: ("transfer learning" OR "fine tuning" OR "semi-supervised" OR "weakly supervised" OR "active learning" OR "selfsupervised" OR "unsupervised") AND (Deep" OR "Deep learning") AND ("Remote sensing" OR "Earth observation") The database searching and data extraction process were performed until December 31, 2022.
III. RESULTS AND BRIEF OVERVIEW The generated database of the reviewed papers and their corresponding data is used for meta-analysis of the NSDL methods in the remote sensing data processing. The essential information about a remote sensing dataset could be electromagnetic band, spatial resolution, and type of study area, while the main parts of the modeling process are learning method, training data, and model architecture. Finally, the trained model is evaluated with proper metrics such as accuracy. Therefore, this section provides the results of metaanalysis of NSDL methods in remote sensing in the following subsections: first, a general characteristic of evaluated studies, such as a simple statistical analysis of published papers in different years and journals, is provided. The Next, deep architectures used within the non-supervised learning scheme in the remote sensing studies are reviewed in the second subsection.
This is followed by an analysis of results attained from different training sizes and datasets in the third and fourth subsections, respectively. In the last part, the statistical results of NSDL methods applied to various study targets of remote sensing are shown and discussed. 232 ) demonstrates an increasing interest in applying non-supervised learning for remote sensing problems within a less than a decade. We also showed the number of open access codes regarding the published papers in each year. The results show that as interest in developing NSDL methods increases, researchers are more willing to share their developed code publicly. Furthermore, a list of most frequently publishing journals, which issued more than two papers evaluated herein, is provided in Fig. 3. As demonstrated in this figure, Remote Sensing MDPI and IEEE Transactions on Geoscience and Remote Sensing (TGRS) are two top journals that have the largest number of publications in this field.
C. Training data Amount or fraction of available labeled training data is an important factor in order to choose a suitable learning method in DL modelling process. According to the definition of USL in this review, we assume that USL does not use any annotated samples and thus, the training fraction is zero. Fig. 6  shows the fraction of training samples used for training the model based on the other learning methods. Accordingly, Semi-SL and Self-SL algorithms have used the lowest training percentage from the available ground-truth with 35. 9%, and 36. 4%, while TL and WSL algorithms, respectively, have used 57. 2% and 68. 9% of the available ground-truth dataset for training their model. Thus, TL methods consumed more training data compared to Semi-SL and Self-SL, while the papers based on these methods cover a wider range of training data.
D. Data type Selecting a suitable data among the existing extensive datasets is a crucial step in solving a remote sensing problem. According to the database collected in this review study, until now, the main goal has been centered on the algorithm development without considering data type. However, this subsection categorizes the applied datasets in the reviewed articles based on the type of sensors and spatial resolution. Results in Fig. 7  show that three band RGB images were the most used dataset in all four learning methods with 162 studies, followed by hyperspectral (HS) and multispectral (MS) datasets with 103 and 105 studies, respectively. In contrast, only few studies dedicated to active sensors, where SAR have been used in 40 studies, and only 7 papers used LiDAR as their data source.
Notably, 41 papers used multiple sources of data in their research work. Fig. 8  depicts the frequency of used sensors based on their spatial resolutions, for each learning method. Accordingly, high portion of studies (232 papers) have used very high resolution (VHR) images, while fewer studies have been dedicated to high-, medium-, or low-resolution datasets. This is aligned with the results obtained for investigating the sensor type (Fig. 7 ), where lower spectral discriminability can bring higher spatial resolution. Table  I  summarizes the most commonly used benchmark datasets in the field of NSDL in remote sensing, along with their characteristics and main applications.
There is a high number of papers that utilized TL or Semi-SL in scene classification and TD by remote sensing images using DL. On the other hand, USL demonstrates its usage in applications that having annotated data is not a common approach, such as anomaly detection (AD) and CD, where automated frameworks are favorable. Similarly, USL is also applied in data preprocessing or enhancement related applications such as denoising, dimension reduction (DR), super resolution (SR), spectral unmixing, data fusion, and registration. Although the WSL method has been less frequently used compared to other learning methods, it has mainly been employed in classification and TD.
IV. NON-SUPERVISED LEARNING METHODS Deep supervised learning methods have obtained state-ofthe-art results in many fields, including several remote sensing applications  [3] ,  [4]. However, these methods necessitate a considerably huge and comprehensive training dataset in order to present their best performance. Furthermore, they are prone to overfit on training data, and their performance extremely degrades on the new domain of test datasets  [18]. In order to address the mentioned drawbacks, non-supervised learning methods are proposed. There is no unique categorization of non-supervised learning methods in the literature, and it is not feasible to find hard boundaries to separate these methods. However, in this paper, we formulate them based on the most commonly used methods in remote sensing.
A. Transfer Learning In transfer learning (TL), a model is first trained on a dataset from one domain and then reused for a dataset from another domain. TL is also referred to fine-tuning or pretraining strategy in the literature  [21] ,  [22]. Generally, training a deep model with thousands of learnable parameters from scratch requires a large-scale training dataset. Hence, the main goal of TL is alleviating the problem of limited training data and accelerating the learning process by using the pre-trained model from another domain and fine-tuning the model with the existing labeled samples of the main dataset. The most prevalent approach in TL is to utilize base models that have been pretrained on large-scale natural benchmark datasets, such as ImageNet  [23] , or COCO  [24]. These models are then finetuned on the desired dataset.
While this approach is effective in reducing the need for a large number of training samples in computer vision applications, it is not as effective in most scenarios for remote sensing due to the different geometry and nature of the acquired data, and still large number of training data is required for these applications. Therefore, it is important to note that we did not consider this approach as a nonsupervised TL method for remote sensing applications. Data augmentation by generating synthetic or simulated data can be considered as another approach in TL  [25] -  [28]. Data augmentation by generating synthetic or simulated data is another TL approach [25]-  [28]. One primary advantage of using synthetic data is that the precise annotation is become available for free. On the other hand, collecting and annotating data for a large number of problems is not only a tedious process but also prone to human errors  [18].
Recent studies have shown that GAN models are effective in increasing the number and diversity of training data, and balancing the imbalanced labeled data by synthesizing new samples (Fig. 11 ). The main drawback of TL would be the domain shift between the synthesized training data and the test dataset  [29] ,  [30]. More specifically, in remote sensing problems, the existence of different sensors for multi-sensor or multi-temporal problems or different illumination, reflectance, and topographic conditions on large scales can be known as the main sources of domain inconsistencies  [31]. Consequently, this degrades the generalization capability of the trained model on test samples  [6]. Domain adaptation can address the mentioned limitations in remote sensing data processing by minimizing the distribution gap between source and target domains  [18] ,  [31].
Meta-learning comprises multiple tasks, where each task contains a pair of support and query sets. The learning processing accomplishes in two different inner-and outer levels. At the inner level, the base learner concerns with plenty of single tasks to learn from the new observations. In contrast, at the outer level, the meta learner adapts the knowledge gathered from previous tasks to generalize a new task  [34] -  [39]. Accordingly, the model's learning improves by the time and with more experience. Thus, Meta-learning learns the learning rule or update function, which helps to converge the model with very few training samples and improves the model's generalization by updating the learning function. However, as mentioned in  [39] ,  [40] , one limitation is that this method requires a meta-training dataset rich in class diversity to attain a generalizable meta-model.
C. Semi-Supervised Learning The main idea behind the semi-supervised learning (Semi-SL) method is to improve the model's performance by making use of unlabeled data in the model's learning process  [15] ,  [49]. More specifically, Semi-SL mainly aims to leverage the power of thousands or millions of unannotated samples in a dataset to reinforce the training process and improve the generalization of the final model. So compared to the other non-supervised approaches, Semi-SL's primary strategy is to generate pseudo labels for test samples  [15] ,  [49]. Mainly, few criteria or conditions are needed to select high-quality pseudo labels and ignore the uncertain labels to be evaluated in the subsequent iterations. Accordingly, apart from human interpretation and annotation of candidate samples  [19] , some of the suggested approaches in the literature could be evaluating the predictions of ensemble models  [50]  or Bayesian modeling evaluation, which is estimating the class assignment confidence based on the posterior probabilities  [51].
Active learning (AL) can also be regarded as a sub-division of Semi-SL. In essence, AL commences with training a learner using a limited-sized training dataset. Following the initial model training, unlabeled data is employed for uncertainty sampling. Samples exhibiting high levels of uncertainty as predicted by the model are chosen for human annotation. Subsequently, these newly labeled samples are incorporated into the training dataset in order to retrain the model  [15] ,  [42] ,  [49] -  [54]. Fig. 12  Demonstrates the general pipeline for Semi-SL and AL.
D. Self-Supervised Learning Self-supervised learning (Self-SL) is a relatively new approach that has gained much attention in recent studies, due to its capability in reduction of required labeled data with representative feature extraction from unlabeled data  [33] ,  [41]. General pipeline of Self-SL consists of two main steps: First unsupervised representation learning, and then supervised finetuning of the pre-trained model for the corresponding main application  [55]  (Fig. 13-a ). In unsupervised representation learning, underlying high-or low-level features of the dataset are extracted by leveraging the huge number of unlabeled samples. This step is mainly performed based on a pretext learning task, which aims to learn generalizable representations of the under the test dataset. At this point, model output is mainly defined from the input data properties, such as signal reconstruction, predicting a pretext task label, or contrasting the similarity between the input signals  [33] ,  [41].
In the second  The main difference between Self-SL and USL can be in term of their methodology. In unsupervised learning no external labels exist to aid the learning process  [33]. Thus, USL mainly find patterns by utilizing the statistical properties of the data samples  [41]. Whereas, Self-SL tries to extract representative features by self-training. Thus, it uses far more feedback signals than USL during the training processing  [33] ,  [41]. Main steps of Self-SL might resemblance TL, where it fine-tunes a pretrained model. However, in contrast to TL, pre-training stage needs no training data in Self-SL, and the whole training process needs much smaller training samples than TL. Moreover, domain gap is much less than TL, due to using a same dataset for pre-training and fine-tuning  [41] ,  [55].
In recent studies, adversarial models, such as GANs and contrastive learning models, were utilized to extract inherent data patterns or generate pseudo-training datasets by leveraging the min-max game of two contrastive loss functions in the training stage  [18] ,  [33] ,  [42] ,  [64] -  [68]. With the provided overview of the most prominent NSDL methods in remote sensing, Table  II  highlights their main advantages and disadvantages. This aims to make easier to choose a suitable learning method for solving remote sensing problems.
V. NON-SUPERVISED DEEP LEARNING IN REMOTE SENSING Applications and recent advances of the DL algorithms in remote sensing have been discussed in previous review papers  [3] ,  [4] ,  [10]. Thus, following the taxonomy provided in the previous literature, this section aims to discuss and give some insights into the applications of NSDL algorithms in remote sensing data processing. According to the results shown in Fig. 10  of this review, the primary focus of studies is on the development of NSDL algorithms for remote sensing image classification (including segmentation and land-use land-cover mapping) followed by target detection and change detection studies. Although small portion of recent studies are focused on other applications, such as anomaly detection, and data enhancement and preprocessing applications such as fusion, denoising, registration, they are still worth mentioning in this section in order to draw more attentions for future studies.
A. Classification and semantic segmentation Image classification and semantic segmentation have the most application in using DL models, due to their importance and also availability of various benchmark datasets. Remote sensing classification tasks can be object classification from small patches such as classifying airplanes, buildings, or cloudy images or pixel-wise scene classification or semantic segmentation such as land use land cover, crop types, or wetlands. However, as mentioned earlier, collecting a rich ground truth in order to produce a generalizable model is an expensive task. According to the results, NSDL algorithms offer various solutions to these issues. As shown in Fig. 10 , the most common approaches are TL and Semi-SL. The basic idea of utilizing TL for remote sensing in the early stages of DL involved fine-tuning pretrained base DL models, such as the ResNet family  [69] , VGGs and AlexNet  [70] ,  [71].
However, this approach relies on models that were originally pretrained on large-scale RGB computer vision datasets, such as ImageNet  [23] , or COCO  [24]  , which are not well-suited for remote sensing datasets. Remote sensing datasets typically have different imaging geometry and may contain more than three channels, such as multispectral and hyperspectral images. As a result, a significant number of training samples are still required to obtain a reliable model using this approach, which does not differ significantly from supervised training. Recent developments addressed the issues related to TL from computer vision pretrained datasets by releasing large remote sensing datasets such as BigEarthNet S2  [72] , S1, and MM  [73] , which are labeled image patches from Sentinel-1, and 2 for image classification.
Furthermore, Yang et al. , (2017) investigated the TL between two hyperspectral datasets, as source and target domains  [74]. Generating simulated or synthetic samples can also provide more customizable and diverse training data with a lower domain distance from the target dataset. The feasibility of this process has been increased significantly with the emergence of generative models. Numerous recent studies have observed the effectiveness of GAN models for remote sensing data classification by addressing the imbalanced or scarce training datasets by generating synthetic data. This approach has been successfully implemented in classification tasks across diverse fields  [25] ,  [28] ,  [75]. Recent studies in the past few years tried to develop novel methods in TL with very few annotated data.
To this end, GNN model was presented in  [76]  for few-shot VHR RGB image classification, while Cao et al. ,  [77]  investigated 3D convolution Siamese model with a contrastive loss for • The pre-trained model may not be suitable for the target task, leading to suboptimal performance. • There might be a domain shift between the source and target domains, affecting the transferability of learned features. • Different preprocessing steps between the source and target datasets might affect the finetuning performance.