# An Introduction to Machine and Deep Learning Methods for Cloud Masking Applications
## 4 Deep Learning Methods  
### Deep Learning for Semantic Segmentation  
Deep Learning enables the development of computational models structured with several layers of processing. These models are capable of learning and identifying data representations through various levels of abstraction [59]. Deep Learning architectures such as CNNs (Convolutional Neural Networks) and Fully Convolutional Neural Network (FCN) [60; 61] have significantly improved the results in cloud detection and masking [39; 62]. Specifically, their ability to effectively capture spatial patterns and structures from images, in addition to spectral features, has led to more accurate image classification and segmentation algorithms [63; 64; 65]. In this subsection it is a brief overview of state-of-the-art Deep Learning methods for semantic segmentation. The hierarchical structure of CNNs, including convolutional, pooling, and fully connected layers, enables them to learn increasingly complex features from the input data by increasing the number of convolutional layers, automatically capturing in the cloud case, spatial, and spectral characteristics, with the notable advantage of not depending on domain experts for manual feature selection. The loss of resolution, i.e., of local and global information due to pooling and subsampling operations of a basic CNN, make it less suitable for semantic segmentation, which requires knowledge of more details to achieve accurate pixel-level classification. However, the research in this area has produced CNN-based architectures in the last decade [65] that minimize the loss of global, local, and spatial information. Considerable progress has been made in Fully Supervised methods whose main limitation is the need for large volumes of annotated data for training purposes. Weakly supervised learning approaches are a growing area of research that tends to overcome this limitation and speed up the process, although their performance is currently inferior to that of supervised methods, partly due to the coarser granularity of data annotations [66; 67; 68]. Generative Adversarial Networks or GANs are generative/discriminative models capable of generating input-consistent images and acting as [69; 70] classifiers, used as a weakly supervised method to tackle the laborious task of generating pixel labels. Unsupervised Domain Adaptation method, UDA, can be considered a special case of transfer learning, and the strategy consists of adapting existing models from a source domain to a target domain for which no labelled data exists. UDA approaches differ according to the adaptation techniques used, e.g., self-training or Generative Adversarial Networks-based [65; 71; 72; 73]. In supervised learning, fully convolutional neural networks (FCNs) represent a milestone for pixel-wise end-to-end prediction masks. In FCNs, the substitution of fully connected layers into convolutional layers enabled to obtain segmentation maps with the same size as the input image but in general, spatially detailed information is lost. The encoder-decoder two-stage structure is extensively used where the spatial information is recovered after having downsampled the feature map size, by gradually resizing the spatial features in the upsample stage. Classic examples are the well-known architectures U-Net [74] (Figure 2) and SegNet [75], designed for segmenting biomedical images and natural images, respectively, and the more recent SFANet [76] and CANet [77]. The U-Net model has given rise to several variations in different fields such as UNet++, a nested U-Net architecture proposed in [78] for medical images. High-Resolution Network, HRNet, and its evolution Hmetv2, which is more specific for semantic segmentation applications [79], are designed with two multiresolution connected sub-nets to maintain the high-resolution image information through the parallel high-to-low convolution streams.