## 3 Materials and Methods
### Architecture Description  
The nnU-Net framework [22] is a standardized and flexible tool for medical image segmentation, proven to outperform other DL approaches in various tasks. To use nnU-Net for any specific segmentation task, the only requirement is to format the raw data according to specific guidelines, including file naming conventions, folder structure, and data con  
Figure 4: The original image and three annotations by different radiologists from the Heart dataset.  
figuration files. Once prepared, nnU-Net ensures consistent preprocessing for all models, such as resampling, normalization, and cropping.  
By default, nnU-Net employs five-fold cross-validation and trains five models. Since data generation in nnU-Net is separated from training, this ensures that, after finalizing the data generators (and thus fixing the data split into cross-validation folds), all models are trained on the exact same data, with identical augmentations. This guarantees a fair comparison of model performance.  
As shown in Figure 5, we extended nnU-Net framework to train the following deep models for predicting segmentation (un)certainty:  
* Ensemble models,
* MC dropout models,
* Regression models.  
The ensembles used in this study consisted of the five models trained by default within the nnU-Net framework. For each input image, these five models generated five separate predictions, and the average of these predictions was used to estimate segmentation (un)certainty, which was then compared with the ground truth (GT) certainty maps, as described in the previous section. The ensembles were trained using two versions of the training sets: with and without multiplication. These experiments aimed to determine whether multiple annotations for a single image are necessary for reliable segmentation (un)certainty estimation.  
To train the MC dropout models, dropout rates for U-Net were set to nonzero values of 20%, 40%, 60%, and 80%, with the network remaining in training mode during inference. For each dropout rate, 30 MC samples were drawn, and the average of these 30 predictions was used to estimate segmentation accuracy as a function of dropout rate. Specifically, the average of the MC dropout samples was compared with the GT certainty maps to assess segmentation (un)certainty.  
Converting the nnU-Net segmentation framework into a regression framework requires some modifications. The segmentation task must be reformulated as a multi-channel input task, where the number of input channels equals one plus the number of segmentation classes for foreground objects. For all datasets except the Heart dataset, this results in two input channels, as we are solving a binary segmentation task. In the case of the Heart  
Figure 5: Show the model architecture based on the nnU-Net framework. The model has one input channel, and the number of output channels equals the number of segmented classes.  
dataset, which contains two foreground classes (lungs and heart), there are three input channels. The first channel always contains the original image, while the remaining channels hold the ground truth (GT) certainty maps for the foreground objects. Although manual segmentation masks are processed by nnU-Net data generators, they are not used during training. Instead, GT certainty maps are extracted from the input data channels and serve as reference data during training. The original nnU-Net loss function is replaced by the mean absolute error (MAE) loss to train the regression models. These models directly predict segmentation certainty maps, which are then compared with the GT certainty maps during testing. Our re-implementation of nnU-Net is available publicly on GitHub.