## 5 Discussion  
Prior studies often focused on specific aspects of the uncertainty segmentation, such as using a particular dataset or model architecture. However, these results can have limited generalizability, as a model might perform well on a specific dataset but poorly on unseen data or produce unrealistic certainty values, hindering its clinical application. This work aimed to shed light on multiple aspects of using deep learning with uncertainty quantification for medical image segmentation. Most prior studies relied on single-annotator datasets. A drawback of this approach is the inherent variability in expert opinions. To address this limitation, at least three experts segmented each dataset in this study. Subsequently, models were trained on both single-annotation and multiple-annotation versions to evaluate the impact of multiple annotations on uncertainty measurement. Another limitation in prior research was its narrow focus on specific datasets or image types, potentially hindering model generalization. In contrast, we trained our models on diverse datasets covering various body parts and imaging modalities, including CT scans, MRIs, and X-rays. Our study employed three established architectures--ensembles, regressions, and MC dropout--implemented in a customized nnU-Net framework. Each architecture was trained separately on all datasets to facilitate a fair comparison and to understand how performance varies with dataset complexity. Furthermore, we investigated the impact of MC dropout rates (20%, 40%, 60%, 80%) on uncertainty models. The present study uses four datasets: Knee, SJJ, Lung, and Heart, each showing varied model performance, largely influenced by their distinct characteristics detailed in Table 2. Dataset size is a crucial factor: Knee, with the smallest size (47 images), results in modest performance across models (median GDC below 85%). Conversely, the Lung dataset, the largest (1338 images), consistently performs well across all models (median NRMSE below 0.1, except for MA at 0.25). The ample data in the Lung dataset likely enhances model learning. However, larger sample sizes do not always lead to better model performance. The Heart and SJJ datasets illustrate this point. Despite having only 65 images, the Heart dataset yields superior performance across all models. In contrast,  
Figure 8: Shows correlation plots for the GDC metric. the SJJ dataset, with 345 images, shows significantly lower performance in both GDC and NMI metrics, with all models scoring median GDC below 80%. These contrasting results suggest that factors beyond sample size, such as annotation diversity and data complexity, significantly influence model efficiency. We further explored how multiple annotations affect model performance. Intuitively, more annotations should improve performance due to the combined expertise of multiple observers. However, our findings show no consistent advantage for models trained with multiple annotations over those trained with single annotations. In some cases, single-annotation models even outperform multiple-annotation models. Notably, the MA model scored 5% higher GDC median for the HHS dataset and 0.3 less NRMSE median for the SIJ dataset. Conversely, the SA model scored 7% higher GDC median for the Knee dataset and 0.15 lower median NRMSE for the Lung dataset. Generally, both models demonstrate similar performance in other cases. Our results show that multiple annotations lead to a clear improvement in uncertainty estimation in only one case--heart segmentation in X-ray images, with a roughly 5% increase in GDC. However, we believe this improvement may be due to chance, as no significant enhancements were observed for other datasets when multiple annotations were used for training. We think the limited impact of multiple annotations on uncertainty prediction accuracy is related to the local properties of the images being segmented. Specifically, when the boundaries of the segmented objects are clear, annotators can delineate them with high certainty.