# ReFuse: Generating Imperviousness Maps from Multi-Spectral Sentinel-2 Satellite Imagery
## 3 Materials and Methods
### Multi-Spectral Bands Fusion Network  
Semantic labelling of satellite images requires a dense pixel-wise image classification. In this work, we started exploring some popular neural networks' capabilities for pixel-wise extraction of impervious surfaces. However, it is crucial to consider that Remote-Sensing (RS) image data are more than a picture since they include electromagnetic wavelengths ranging beyond the three RGB bands of natural images. In general, a CNN can take an arbitrary number of spectral bands as input, modifying the depth of the first input convolutional layer. However, exploiting the multi-spectral content of RS images is not as simple as presenting more bands as input to the network. Indeed, increasing the number of spectral input bands eventually results in the need for more extensive training datasets, consisting of satellite images and corresponding ground-truth data, to avoid incurring in overfitting. Moreover, this approach does not allow for leveraging _transfer learning_, which instead proved to be effective for remote-sensing image analysis [35]. In a multi-sensor setup, with more than three bands available, a possible approach is to employ two or more neural network branches to analyse some of the bands separately, fusing the features at a later stage in the network. However, the main drawback of this late fusion procedure is that the number of weights doubles, thus requiring more computation time for both the  
Figure 5: Localization of the Sentinel-2 tiles used in dataset generation. training and inference phase. Considering the extension of satellite images, this limitation might be too demanding to make the approach feasible in a real production environment. In this paper, we thus focus on designing a CNN able to efficiently combine features from multi-spectral bands, providing a good balance between the number of parameters and segmentation performance. The proposed architecture, shown in Figure 6, comprises three state-of-the-art ideas:  
1. A **U-Net** as backbone architecture. Its encoder-decoder paradigm with cross-connections for pixel-wise labelling and skip connections between same-sized parts in down-sampling\(\backslash\) up-sampling paths help to address the loss of fine detail during up-sampling [7];
2. To achieve better results as the depth of the network increases, _the building blocks of the standard U-Net encoder part were replaced with residual blocks_. More in detail, a ResNet-50 model has been used to replace the U-Net encoder down-sampling section. The idea is to leverage residual blocks' ability to reduce the problems associated with the vanishing gradients strongly;
3. To exploits multi-spectral content, beyond classic RGB wavelengths, a **FuseNet**[9] approach has been used. The FuseNet model jointly encodes both the RGB and depth information using two encoders (in this case, two ResNet-50, as described in the previous point) whose contributions are summed after each convolutional block. We adapted the standard fusion approach to the use of residual networks by summing contributions from different branches after each residual block. The result is a U-Net-like network with the encoder replaced by two parallel ResNet-50 networks where the main branch inputs the RGB bands while the second uses bands B07, B08, and B11. Every residual block output from the second branch is fused into the main branch by employing feature map summation. After that, the fused map is connected to a convolutional layer for the decoding part through concatenation, implementing the classical U-Net cross-connections. _All these solutions allow the proposed architecture to benefit from the combination of short (i.e., residual skip connections) and long skip connection (i.e., U-Net cross-connections) during the training strategy._ We named this approach **ReFuse** after its two core components: **RE**sidual blocks and **FuseNet**. Some minor changes were applied to the ResNet-50 encoder, following the minor adjustments presented in [36].