# Incorporating Aleatoric Uncertainties in Lake Ice Mapping Using RADARSAT-2 SAR Images and CNNs
## 4 Methodology
### CNN Architecture and Hyperparameter Selection  
The CNN architectures used were based on [PERSON] et al. [21], with some modifications. The first architecture tested, to provide a baseline, was identical to the one suggested by [PERSON] et al. [21]. To incorporate aleatoric uncertainty, this architecture was modified to incorporate two outputs after the final dense layer instead of one (Figure 5a). The sigmoid function that previously followed the dense layer was removed as this was incorporated in the custom loss function. Therefore, the base CNN architecture predicted two quantities, a logit and a variance. Python was used as the programming language. Key utilized libraries include Tensorflow, Tensorflow probability for developing CNNs and NetCDF4 and scipy for handling data processing and post processing steps. To run the codes, computational resources of two v100 GPUs, 18 CPUs with a total of 90 G of memory has been utilized.  
To evaluate the impact of including bathymetry, the baseline CNN was first trained and evaluated (without incorporating aleatoric uncertainty). In this case the architecture proposed by [PERSON] et al. [21] was used, except having an additional input, which consisted of zonal bathymetry. Bathymetry is not at the SAR image pixel scale. A zonal bathymetry class is assigned to each lat/lon value on the label grid. Therefore, the classes are incorporated in the CNN at a much lower level when the CNN output is of the same dimension as the label (Figure 5b).  
Finally, to incorporate both bathymetry and aleatoric uncertainty the same CNN architecture was used, but with two outputs after the final dense layer, and with the removal of the sigmoid and the use of custom loss function. In this setting however, the way the bathymetry data are interpreted by the CNN is different than without aleatoric uncertainty. These data are part of the Monte Carlo sampling that is used to find the optimal value of the logit and variance, and the weights linking the bathymetry to the network output (mean and variance) are updated in the backpropagation. In this way, the aleatoric uncertainty can be a function of not only the input image patch, but also the bathymetry.  
To obtain a smooth reduction in the loss function when both bathymetry and aleatoric uncertainty are incorporated, transfer learning was used. Instead of initializing by random weights, we used weights of trained network with aleatoric loss when no bathymetry was incorporated. The part of the network with transferred weights is delineated by a red box in Figure 5. This method was found to lead to a smooth decrease in the loss function. Before applying this technique, other popular methods for tuning hyperparameters and optimizers for weight initialization and updates were tested but showed to have a minimal impact on the loss trend. We also tested the orthogonality of the weight matrices while training because this can contribute to an uneven decrease in the loss function, we monitored  
Figure 4: Zonal Bathymetry of Lake Erie (**a**) and Ontario (**b**) adapted from [39].  
the Forbenius norm of weights (W) calculated by W\({}^{\text{T}}\)W, and found no sign of losing the orthogonality of the weight matrices while training.  
For all CNN architectures, hyperparameter tuning was carried out to ensure an effective learning strategy. We tested different optimizers such as Adam and stochastic gradient descent (SGD) and tuned respective parameters. We found SGD to perform the best for our problem. After selecting SGD, the learning rate was tuned. A cyclic learning rate schedule was found to be optimal in helping the model converge compared to step decay [40].