# ReFuse: Generating Imperviousness Maps from Multi-Spectral Sentinel-2 Satellite Imagery
## 4 Experimental Setup  
We implemented the network in PyTorch, and the training execution was on ad hoc AWS EC2 instances. We trained all our models using the Adam optimizer [37] with \(\beta_{1}=0.9\), \(\beta_{2}=0.99\), \(\epsilon=1^{-5}\), a weight decay of \(1^{-2}\) and a batch size of 8. The considered loss function is a combination of dice and pixel-wise cross-entropy loss:  
\[L_{total}=L_{dice}+L_{ce}\]  
with  
\[L_{ce}=-(ylog(p)+(1-y)log(1-p))\] \[L_{dice}=1-\frac{2|A\cap B|}{|A|+|B|}\]  
where \(A\) is the predicted segmentation mask and \(B\) is the ground truth, \(y\) is the ground truth, and \(p\) is the probability for that class. The choice of also including the dice loss is to help to regularise results in the case of unbalanced data chips (e.g., 95% bare soil and 5% impervious surface in a single chip).  
One of the most critical hyperparameters is the learning rate (LR): a big LR causes the model to diverge, while a small LR causes the model to converge slowly or stack in an unfavourable solution. We used two methods, the _learning rate range test_[38] and the _one-cycle policy_[39], to determine the ideal LR value and train the model for accurate fine-tuning. The LR range test is a method for determining what are considered to be the acceptable minimum and maximum boundary values for LR. It entails running the model over several epochs while letting the LR rise linearly between low and high LR values after each mini-batch until the loss value increases enormously. Plotting accuracy trend variating LR values, we can choose the LR one order lower than the point where loss is minimum. Figure 10a shows the output for ReFuse model learning range test: we chose  
Figure 9: Flow execution process for the imperviousness map generation. A Docker image stores library dependencies. The model, weights and flow code are stored in object-based storage. During flow execution, the orchestrator pulls the image from the Docker registry and using it starts a Dask cluster. Finally, the prediction flow is executed on the Dask cluster.  
a maximum LR of \(1^{-3}\). On the other hand, one-cycle policy is a technique similar to the simulated annealing algorithm [40] designed for varying the LR during the training. The method needs an initial interval of values: we choose the maximum value using the LR range test and the lower one as \(1/10\)th of the maximum LR. The algorithm moves from the lower to the higher value during the first half of the cycle and from the higher back to the lower during the latter half of the cycle. Finally, in the last few iterations, this method anneals the learning rate way below the lower learning rate value for a final fine-tuning. Conventionally, the learning rate decreases as the learning start converges, but the idea behind this approach is that a higher learning rate may help overcome saddle points. In [39] the author shows that when the learning rate is higher, during the middle of learning, the learning rate works as a regularisation method and keeps the network from overfitting because it avoids steep areas of loss and finds a better flatter minimum. Figure 10b shows LR values used during training across the considered iterations. As soon as the model performance on the validation dataset ceased improving, we employed an early-stopping criterion to stop the training.  
The GIS web application and the inference pipeline were executed on cloud infrastructure, using a Kubernetes cluster. In particular, the inference pipeline was released on a serverless infrastructure to reduce the solution's running costs.