# An Introduction to Machine and Deep Learning Methods for Cloud Masking Applications
## 4 Deep Learning Methods
### Deep Learning for Cloud Masking  
The problem of generating pixel-wise cloud masks can be regarded as a semantic segmentation task, whose algorithms are highly concerned with the semantic content of an image and aim to classify each pixel into meaningful object classes (specifically cloud, clear  
Figure 2: U-Net architecture overview. The U-Net design, characterized by a contracting path on the left and an expansive path on the right, forms a U-shaped structure. The contracting path involves convolutional and pooling layers for feature extraction, while the expansive path employs transposed convolutions to enable precise localization and reconstruction. Skip connections bridge corresponding layers, aiding in the retention of fine-grained details. Courtesy of [PERSON] [74]. sky, land, water, snow, haze and so on). Deep Learning architectures developed to segment natural images form the basis for addressing this topic. Some early works proved the potentiality of CNN for this task. For instance, the article in [91] is an example of a study for the application of a simple CNN architecture to the task of cloud masking; performed on multispectral Proba-V images, it was shown that the approach was promising. Generally, CNNs are used for classification aims, but also for segmentation tasks approached as a classification task when a pixel level is not required. As an example, in [92] an improved version of the Simple Linear Iterative Clustering (SLIC [93]) method is used to generate superpixels, on the basis of value similarity and proximity. Patches extracted from the superpixels are fed into a specifically designed deep CNN, dual-branch PCANet in [94] combined with an SVM classifier, responsible for predicting their categorization a cloud/non-cloud. This approach strongly relies on the accuracy of the image segmentation in superpixels. Deep Learning algorithms for cloud detection at pixel-level provide labelled masks typically employing techniques that learn and infer class labels based on high-level semantic features directly from the raw input image. See Figure 3 as an example of masks. U-Net-like architectures are well-suited for segmenting large images such as satellite data and they are extensively used in cloud masking. U-Net architecture involves feature detection and up-sampling, with the feature extraction phase growing in scale with each max-pooling operation. In recent years, a considerable amount of work has been presented in the literature where several methods for cloud masking are based on improved U-Net and different semantic segmentation models. CS-CNN, a Cloud Segmentation model for cloud masking was trained on MSG-SEVIRI images, in [96], and compared with RF models; U-Net variations were proposed for cloud/snow and cloud/cloud-shadow segmentation, respectively, in [97,98] and in [99] where the RS-Net model performance, for Landsat 8 images, was comparable to the reference FMask algorithm. In [100], the authors proposed  
Figure 3: Cloud masks for different scenes (rows) with varying conditions, obtained by three different methods (columns). First column shows the original RGB images. Predicted masks and ground truth are overlapped. Color legend for predicted cloud masks vs ground truth: yellow indicates areas where cloud mask agrees with the ground truth, designating cloudy pixels (True Positives). Orange denotes regions where cloud mask erroneously identifies cloud-covered pixels as clear land (False Negatives). Blue highlights zones where cloud mask inaccurately classifies clear land as cloud-covered areas (False Positives). Reflective pixels are left white in masks and ground truth when they are not misclassified. Courtesy of [PERSON] [95]. the specialized architecture Cloud-Net, trained on Landsat 8 38 Cloud dataset (labelled by the authors), to capture global and local cloud features using specifically designed convolutional blocks, leading to a better Overall Accuracy (proportion of correct predictions out of all the predictions made by a model) than FMask. [PERSON] et al.