# Exploring transformer reliability in clinically significant prostate cancer segmentation: A comprehensive in-depth investigation

Gustavo Andrade-Miranda, Pedro Juan Soto Vega, Kamilia Taguelmimt, Hong-Phuong Dang, Dimitris Visvikis, Julien Bert

<figure><img src="image_1.png" /></figure>

## To cite this version:

Gustavo Andrade-Miranda, Pedro Juan Soto Vega, Kamilia Taguelmimt, Hong-Phuong Dang, Dimitris Visvikis, et al.. Exploring transformer reliability in clinically significant prostate cancer segmentation: A comprehensive in-depth investigation. Computerized Medical Imaging and Graphics, 2024, 118, pp.102459. 10.1016/j.compmedimag.2024.102459. hal-04792689

HAL Id: hal-04792689

https://hal.science/hal-04792689v1

Submitted on 4 Feb 2025

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Graphical Abstract

Exploring Transformer Reliability in Clinically Significant Prostate
Cancer Segmentation: A Comprehensive In-Depth Investigation

Gustavo Andrade-Miranda, Pedro Soto Vega, Kamilia Taguelmimt, Hong-Phuong Dang, Dimitris Visvikis, Julien Bert

<figure><img src="image_3.png" /></figure>
Highlights

Exploring Transformer Reliability in Clinically Significant Prostate
Cancer Segmentation: A Comprehensive In-Depth Investigation

Gustavo Andrade-Miranda, Pedro Soto Vega, Kamilia Taguelmimt, Hong-Phuong Dang, Dimitris Visvikis, Julien Bert

* The reliability of transformer-based models for clinically significant prostate cancer is evaluated at voxel, lesion, and image levels.

* A complete assessment framework comprising robustness, calibration, out-of-distribution, and misclassification detection is presented.

* Variations in performance between in-domain and out-of-domain data are explored.

* No model remains unsusceptible to challenges posed by domain shifts.

* Hierarchical, hybrid, or emulating attention using convolutional components—prove to be the most reliable models.
Exploring Transformer Reliability in Clinically
Significant Prostate Cancer Segmentation: A
Comprehensive In-Depth Investigation

Gustavo Andrade-Mirandaª, Pedro Soto Vegaª,ᵇ, Kamilia Taguelmimtª,
Hong-Phuong Dangª,ᵈ, Dimitris Visvikisª,*, Julien Bertª,ᶜ

<sup>a</sup>LaTIM UMR1101, INSERM, University of Brest, Brest, France

<sup>b</sup>Vision-AD and AutoRob Team, L@bisen,ISEN Yncréa Ouest, Brest, France

<sup>c</sup>University Hospital of Brest, Brest, France

<sup>d</sup>CentraleSupélec, IETR UMR CNRS 6164, Cesson-Sévigné, France

**Abstract**

Despite the growing prominence of transformers in medical image segmen-
tation, their application to clinically significant prostate cancer (csPCa) has
been overlooked. Minimal attention has been paid to domain shift analysis and
uncertainty assessment, critical for safely implementing computer-aided diag-
nosis (CAD) systems. Domain shift in medical imagery refers to differences
between the data used to train a model and the data evaluated later, arising
from variations in imaging equipment, protocols, patient populations, and ac-
quisition noise. While recent models enhance in-domain performance, areas
such as robustness and uncertainty estimation in out-of-domain distributions
have received limited investigation, creating indecisiveness about model relia-
bility. In contrast, our study addresses csPCa at voxel, lesion, and image levels,
investigating models from traditional U-Net to cutting-edge transformers. We
focus on four key points: robustness, calibration, out-of-distribution (OOD), and
misclassification detection (MD). Findings show that transformer-based models
exhibit enhanced robustness at image and lesion levels, both in and out of do-
main. However, this improvement is not fully translated to the voxel level, where
Convolutional Neural Networks (CNNs) outperform in most robustness metrics.
Regarding uncertainty, hybrid transformers and transformer encoders performed
better, but this trend depends on misclassification or out-of-distribution tasks.

Keywords: Prostate cancer, vision transformers, calibration,
out-of-distribution, misclassification detection, robustness

Manuscript received January 25, 2024.

*Corresponding author

Email addresses: andradema@univ-brest.fr (Gustavo Andrade-Miranda),
pedro-juan.soto-vega@isen-ouest.yncrea.fr (Pedro Soto Vega),
kamilia.taguelmimt@univ-brest.fr (Kamilia Taguelmimt),
hong-phuong.dang@ecam-rennes.fr (Hong-Phuong Dang), visvikis@univ-brest.fr
(Dimitris Visvikis), julien.bert@univ-brest.fr (Julien Bert)
# 1. Introduction

Prostate cancer is the most common cancer affecting men in 112 countries, accounting for 15% of all male cancer cases Sung et al. (2021). It is the second leading cause of cancer-related deaths in men, following lung cancer Bray et al. (2023). Projections indicate that new cases will rise from 1.4 million in 2020 to 2.9 million by 2040 James et al. (2024). Screening for prostate cancer (PCa) using multi-parametric magnetic resonance imaging (mpMRI) has been shown to reduce advanced-stage cases and disease-specific mortality. mpMRI-based diagnosis has a high negative predictive value and sensitivity for determining clinically significant prostate cancers (csPCa), enabling reliable exclusion of non-significant cases and guiding patient selection for prostate biopsy Elkhoury et al. (2019).

However, interpreting mpMRI can be time-consuming, and the lack of experienced radiologists may lead to overlooking factors such as multifocality and morphological diversity Morote et al. (2023). Additionally, the similarity of PCa to various non-malignant conditions reduces inter-reader agreement and leads to sub-optimal interpretations Rosenkrantz et al. (2016); Westphalen et al. (2020). This highlights the need for developing automated, reliable Computer-Aided Diagnosis (CAD) systems for detecting and segmenting csPCa. These systems could address the shortages in healthcare professionals and expertise, accurately diagnose cancers, and subdivide the disease into subgroups to aid treatment selection James et al. (2024). In resource-limited settings, these advancements could be transformative Gueye et al. (2003).

AI-supported diagnosis can streamline mpMRI processing, increase throughput, reduce costs, and improve diagnostic consistency. It can also enhance mpMRI assessment by reducing scan times, minimizing interpretation variance, automating prostate gland outlining, and identifying intraprostatic targets before biopsy and treatment Mehralivand et al. (2020); Raman et al. (2021). The advent of deep learning (DL) models has significantly enhanced CAD systems, making them valuable tools in clinical radiology Hosseinzadeh et al. (2022); Gue-tari et al. (2023); Sun et al. (2023). Particularly, automatic csPCa segmentation aims to identify and quantify malignant lesions within the prostate gland. This segmentation process is critical in diagnosis, risk stratification, and the planning of MRI-targeted biopsies Bashkanov et al. (2023); Bosma et al. (2023).

In the context of csPCa, like many other vision-based tasks, Convolutional Neural Networks (CNNs) have traditionally been employed due to their proven effectiveness in most image analysis tasks Yoo et al. (2019); Saha et al. (2023); Conze et al. (2023). Notably, a significant advancement in this domain involves incorporating attention mechanisms within CNN architectures Pellicer-Valero et al. (2022); Li et al. (2022a); Duran et al. (2022); Song et al. (2023); Li et al. (2023); Wang et al. (2023). These attention layers prioritize specific spatial or channel-related information, enhancing the models' performance. Furthermore,
Vision Transformers (ViT) Dosovitskiy et al. (2021) models have gained promi-
nence in computer vision tasks and are gradually finding their place in medical
image analysis, particularly in medical image segmentation (MIS) Xiao et al.
(2023); Andrade-Miranda et al. (2023).

Despite the demonstrated success of transformer-based models in various
medical image analysis applications Li et al. (2022b); Shamshad et al. (2022),
their potential in the context of csPCa segmentation and diagnosis has not
been properly addressed. Most studies employing transformers have primarily
focused on prostate gland segmentation, specifically the peripheral and transi-
tional zones Hung et al. (2023); Ren et al. (2023). In addition, minimal atten-
tion has been given to domain shift phenomenons and uncertainty assessment
Ricci Lara et al. (2023); Chaves et al. (2023). In short, domain shift is a term
that describes the dissimilarities between the marginal distributions of the data
used for training classifiers and the data that will be evaluated later. This dis-
parity occurs due to fluctuations in imaging equipment, protocols, patient pop-
ulations, and noise during acquisition, which hinder the accuracy of DL-based
methods. Thus, guaranteeing consistent performance and preventing overconfi-
dence when dealing with domain shifts is essential for the secure implementation
of CAD systems for csPCa Jiang et al. (2023). Motivated by the scenario de-
scribed so far, the central objective of our study is to present a comprehensive
and reliable evaluation of contemporary transformer-based models for segmen-
tation and diagnosis of csPCa using bi-parametric MRI (bpMRI) Brancato et al.
(2020), with a specific emphasis on evaluating their robustness and ability to
estimate uncertainty. To shed light on these aspects, we thoroughly assess
five state-of-the-art transformers-based models, a CNN transformers-inspired
model, and the well-established U-Net model, appraising their performance in
in-domain (ID) versus out-of-domain (OOD) conditions to gauge their robust-
ness. Additionally, we evaluate their calibration, misclassification detection,
and OOD detection capabilities to better understand their ability to estimate
uncertainty.

In summary, our contributions in this paper are four-fold. Firstly, we con-
duct a series of controlled experiments to analyze the in-domain versus out-of-
domain robustness of transformers-based models at voxel (segmentation), lesion
(detection), and image levels (classification). Secondly, we evaluate the models'
capabilities in detecting misclassifications in both in-domain (ID) and out-of-
domain (OOD) instances at the voxel level. Thirdly, we explore their domain
of expertise by studying out-of-domain detection using AUROC values. Lastly,
we assess their calibration performance in both in-domain and out-of-domain
scenarios. The remainder of the paper is organized as follows: Section 2 intro-
duces the datasets and the transformer-based models. Section 3 describes the
experimental data partition, implementation details, and evaluation strategy.
The presentation and in-depth analysis of the results and discussions are pro-
vided in Section 4. Finally, Section 5 concludes our study, offering suggestions
for potential avenues of further investigation in the near future.
## 2. Materials and methods

### 2.1. *Datasets*

To explore the reliability of transformer-based models, we conducted an extensive search of publicly available datasets related to csPCa, carefully reviewing the existing dedicated datasets Sunoqrot et al. (2022). We found that most publicly available bpMRI prostate datasets focus on prostate anatomy segmentation. However, only two datasets explicitly address csPCa detection and diagnosis. As these represent the only publicly accessible datasets suited for this specific purpose, they were selected for our evaluation.

**PICAI** Saha et al. (2022, 2024) dataset, sourced from the corresponding grand challenge, comprises over 10,000 meticulously curated prostate MRI exams. Its purpose is to validate Artificial Intelligence (AI) algorithms and estimate radiologists' performance in detecting and diagnosing clinically significant prostate cancer. Our analysis focuses on the publicly available subset of this dataset, consisting of 1500 cases. Each case includes three modalities: T2-weighted imaging (T2W), diffusion-weighted imaging (DWI), and apparent diffusion coefficient maps (ADC). Patient cases within this subset were annotated with histologically-confirmed positive cases ($ISUP \ge 2$) and histologically-negative cases ($ISUP \le 1$). From the Public Training and Development Dataset's 1500 cases, we specifically consider the 1295 cases with expert annotations, encompassing 220 csPCa lesions. Lesions are labeled with values 2, 3, 4, or 5, corresponding to histologically-confirmed ISUP scores. The remaining cases (1075) indicate the absence of lesions or $ISUP \le 1$. The original csPCa annotations are resampled to the spatial resolution of the associated axial T2W scan. Additionally, automated AI-derived delineations of the entire prostate gland are available for all 1295 scans. Alongside the images, the dataset includes patient clinical attributes such as age, PSA, PSA density, Gleason score, and prostate volume.

**Prostate 158** Adams et al. (2022) dataset is a carefully labeled collection of 158 bpMRI images designed to automate the segmentation of anatomical zones and cancerous lesions. The dataset includes T2W, DWI, and ADC scans, ensuring comprehensive imaging coverage. All 158 MRI examinations were performed on Siemens VIDA and Skyra clinical 3.0-T scanners, following a standardized acquisition protocol. To maintain consistency across studies, all images within each study were resampled to have the same orientation, direction, and spacing. PCa lesions were meticulously annotated by two board-certified radiologists during independent reading sessions. Pixel-wise segmentations were also performed for the central gland (central zone and transitional zone) and the peripheral zone of the prostate. PCa lesions were defined as suspicious areas with a PI-RADS score of $\ge 4$. Segmentation was executed at the voxel level, utilizing axial T2W sequences for zonal segmentations. Areas suspicious for PCa were
delineated on the ADC map and correlated with T2W sequences and DWI images by two experienced annotators. For each MRI, two segmentation volumes were created, one containing only the anatomical segmentations and the other solely the tumor.

## 2.2. Network architectures

Overall, we conducted a comprehensive comparison of seven architectures. These include transformer-based models such as two hybrid CNN-ViT encoders (MCNN+ViT<sub>s</sub>-B/1 and CNN+ViT<sub>v</sub>-B/1 Andrade-Miranda et al. (2023)), two pure Transformer encoders (Swin UNETR Hatamizadeh et al. (2022a) and UNETR Hatamizadeh et al. (2022b)), and a fully Transformer-based model (nnFormer Zhou et al. (2021)). Additionally, we evaluated a CNN architecture designed to emulate transformer attention, specifically the adaptation of ConvNeXt Liu et al. (2022) for medical imaging segmentation (MedNeXt Roy et al. (2023)). Lastly, we included a traditional state-of-the-art U-Net constructed iteratively using the nn-UNet framework Isensee et al. (2019).

**U-Net** (●) is constructed using the self-configuring nn-UNet framework, which dynamically determines the number of downsampling and upsampling layers based on the input volume size. This information is obtained from the planning and pre-processing steps executed by the nn-UNet framework beforehand. The 3D CNN encoder consists of ResNet blocks to capture compact features at multiple scales. Each resolution stage comprises a sequence of {conv3D, instance norm, leaky ReLU}, where all the convolution kernels and stride values are set based on the nn-UNet planning. The initial number of feature maps is set to 16 and doubled with each down-sampling operation. After encoding the image into a high-dimensional feature representation, the decoder aims to reduce the number of feature maps while increasing their dimensions using successive upsampling operations. Then, the feature maps from the downsampling are concatenated with the upsampling part, aiding in the propagation of spatial information. Finally, a 1 × 1 × 1 convolution processes the final feature maps to generate a dense prediction with per-pixel class scores.

**MedNeXt** (■) is designed based on 3D ConvNeXt Liu et al. (2022) blocks. It aims to combine the scalability and long-range spatial representation learning capabilities of Vision and Swin Transformers with the inherent inductive bias of ConvNets. MedNeXt blocks are constructed using three layers: a depthwise convolution layer, an expansion layer, and a compression layer, which mimic the behavior of Transformer blocks. The depthwise convolution layer features a $k \times k \times k$ kernel size, followed by channel-wise GroupNorm, replacing ConvNeXt's original LayerNorm to enhance stability with small batches. This convolutional approach allows the layer's large kernels to simulate a wide-range attention mechanism while minimizing computational load. The expansion layer employs a 1 × 1 × 1 convolutional kernel to limit computation and enable width-wise
scaling. This is achieved by multiplying the number of input channel features with a ratio factor, followed by a GELU activation function. Lastly, the compression layer uses a 1 × 1 × 1 kernel to restore the number of feature channels to their original count, performing channel-wise compression of the feature maps. In this work, all experiments are conducted using the default settings of the MedNeXt-B version.

**Swin UNETR(♦)** incorporates Transformer-based mechanisms from the swin block Liu et al. (2021) as the network's encoder and utilizes CNN upsampling blocks for the decoder, following a structure similar to 3D-UNet-like architecture. The swin blocks introduce a sliding window strategy to compute attention within local windows only, thereby reducing the computational cost of global multi-head self-attention (MHSA). They gradually decrease the initial number of tokens by implementing patch merging layers as the network deepens. This characteristic delivers a hierarchical representation similar to the behavior of CNNs. In practice, it consists of four stages and performs 2 × 2 spatial downsampling in the patch merging layer of each stage. The merging layer concatenates the features of each group of 2 × 2 neighboring patches and applies a linear transformation to reduce the number of concatenated features to half of their dimension.

**MCNN+ViT$_{s}$-B/1(+)** is a multi-encoder network designed to capture both inter- and intra-modality interactions. This network stands out as a versatile architecture, capable of learning modality-specific feature representations while also preventing false adaptation among modalities. Initially, each modality undergoes tokenization using a 3D CNN encoder to obtain high-dimensional feature representations. These tokens from each modality are then concatenated and processed by a single-stream set of stacked Transformer blocks. Cross-modal interaction among modalities is achieved by assigning a learnable modal-type embedding to each modal input embedding, indicating its belonging to a particular modality. The primary advantage of this early cross-modal interaction formulation is that all multi-modal token positions can be attended to as a single sequence, allowing for each modality's positions to be well encoded by considering the context of other modalities. However, longer input sequences after concatenation lead to increased computational complexity.

**nnFormer(※)** is a full 3D Transformer architecture designed for volumetric medical image segmentation. Initially, it incorporates a lightweight convolutional embedding layer before the transformer blocks, enabling precise spatial information and extraction of low-level yet high-resolution 3D features. Following the embedding block, transformer and convolutional downsampling blocks are alternated to effectively capture long-term dependencies and hierarchical object concepts at various scales. nnFormer employs local volume-based MHSA (LV-MSA) and global volume-based MHSA (GV-MSA) to construct feature pyramids and ensure a sufficient receptive field for learning representations across both local and global
3D volumes. Lastly, traditional concatenation/summation operations in skip connections, as found in U-Net-like architectures, are replaced by skip connections that leverage attention mechanisms.

**CNN+ViTv-B/1(▲)** integrates a hybrid CNN-Transformer architecture as the encoder, combined with a cascaded upsampler to achieve precise structure localization. It captures local 3D context information using 3D CNN feature maps while leveraging the vanilla Transformer module with an effective patch size of 1 × 1 × 1 to focus on capturing global features. In contrast to the MCNN+ViTs-B/1 architecture, all modalities are fused at the input of the 3D CNN encoder tokenization step. Additionally, no cross-modal interaction is performed at the Transformer level.

**UNETR(★)** employs a contracting-expanding pattern, consisting of a stack of transformer blocks as the encoder, which is connected to a CNN decoder via skip connections. Since Transformer-based architectures operate over 1D sequences, the input volumes must be tokenized and represented in an embedding space before being fed into the Transformer layers. Tokenization is achieved by splitting the 3D input volume into a set of non-overlapping 3D patches. After tokenization, the set of tokens is embedded into a *d*-dimensional space through a trainable linear projection and a positional learnable embedding is added to retain positional information. Subsequently, the tokens pass through a vanilla transformer block comprising alternating multi-head self-attention (MHSA) and multi-layer perceptron (MLP) blocks. Additionally, layer normalization (LN) is applied before each block, and a residual connection is added after each block.

# 3. Experiments

## 3.1. Data management

To evaluate the effects of domain shift on transformer-based models, we constructed in-domain (ID) and out-of-domain (OOD) datasets as follows. Given the larger size of the PICAI dataset, we used it as the ID dataset. We split it into a training set and a corresponding test set. Consequently, the training dataset comprises 1,035 samples, while the test set consists of 260 cases. Importantly, we ensured no patient overlap between the training and test sets and maintained an equivalent percentage of ISUP cases in both datasets. The distribution of clinically significant prostate cancer (csPCa) and non-csPCa cases is as follows: in the training dataset, there are 859 non-csPCa cases and 176 csPCa cases, whereas in the test set, there are 216 non-csPCa cases and 44 csPCa cases. For simplicity, hereafter the test set will be referred to only as PICAI. Additionally, it is worth noting that only one case in the ID dataset presents more than one lesion in a patient.

On the other hand, Prostate 158 is used entirely as OOD dataset. Considering the integral aspect of multi-annotation in the Prostate 158 dataset, we conduct the OOD reliability evaluation using both annotators, denoting the
OOD dataset as Prostate 158 Rater A and Prostate 158 Rater B, encompassing 158 samples with different labels obtained from both experts. In this case, Prostate 158 Rater A and Prostate 158 Rater B present 102 csPCa and 56 non-csPCa cases. It is also worth noting that, unlike PICAI, Prostate 158 is more prone to having multiple lesions for patients. Figure 1 depicts the boxplot with the volume distribution per lesion of the training, PICAI, Prostate 158 Rater A and Prostate 158 Rater B datasets.

<figure><img src="image_4.png" /><figcaption>Figure 1: Boxplot with the volume distribution per lesion of the training, ID, and OOD datasets.</figcaption></figure>

## 3.2. Pre-processing and augmentation strategies

To make a comprehensive and equitable comparison among all the models under consideration, we followed the pre-processing and planning strategy outlined in nn-UNet Isensee et al. (2020). The pre-processing phase began by segmenting the entire prostate gland using an automated AI-based delineation model built on the nn-UNet framework Saha et al. (2022). From this segmentation mask, we derived a region of interest (ROI). The size of the ROI varied based on the prostate's dimensions, extending by 10 voxels along the x and y axes and 2 voxels along the z-axis to account for potential segmentation errors. This cropping was uniformly applied to all training cases, reducing the image sizes of datasets and enhancing computational efficiency. Subsequently, a dataset fingerprint was crafted to encapsulate dataset-specific properties, including image sizes, voxel spacings, and intensity information (mean, standard deviation, 0.5 and 99.5 percentiles of the foreground regions).

Throughout both training and inference, each image underwent independent normalization using z-scoring. All images were resampled to a common target spacing, employing either third-order linear or nearest interpolation methods. The target spacing was determined based on the compilation of all spacings within the training data, with the median value along each axis selected as the
default target spacing. In line with the nn-UNet default strategy, data augmentation techniques were applied on the fly during training. These techniques encompassed random rotation, scaling, flipping, Gaussian noising, Gaussian blurring, brightness and contrast adjustments, simulation of low resolution, and gamma transformation Isensee et al. (2019). This comprehensive pre-processing pipeline ensures a standardized foundation for fair model comparison, promoting consistency and robustness across diverse datasets.

### 3.3. Training details

The models were trained for a total of 300 epochs with a batch size of 4. We endeavored to preserve most of the original training recipes for each model to facilitate a meaningful comparison of their optimal performance. However, certain hyper-parameters, including the optimizer, learning rate, and training loss, were tailored based on recommendations from the PICAI challenges Saha et al. (2021); Bosma et al. (2023). These adjustments were implemented to ensure successful convergence for all architectures. The models were trained employing the Adam optimizer and the focal loss (FL) function Lin et al. (2020). The FL's gamma was set to the default value of 1.0, while alpha was automatically determined as the inverse of the case-level class balance for the training split per fold. The learning rate was set to $1 \times 10^{-4}$, with a weight decay of 0.05, and a cosine learning rate schedule with a warm-up period of 30 epochs. During the segmentation tasks, input images were randomly sampled based on the 3D full-resolution planning of the nn-UNet framework. Half of the samples were extracted randomly, while the others contained at least one foreground class. The size of the random samples for transformer-based models such as UNET(☆), Swin UNET(◆), nnFormer(※), and MedNeXt(■) are set to (96, 96, 96) according to their original recipes. For fully convolutional and hybrid models such as U-Net(●), MCNN+ViTs-B/1(◆), and CNN+ViTv-B/1(▲), we adopt an image size of (160, 160, 20), following the guidelines outlined in the planning phase of the nn-UNet framework.

In the testing stage, we implemented the sliding window inference strategy, where the window size matched the training sample size, with an overlap portion of 0.5 between contiguous regions. Results on test datasets were derived through an averaged ensemble of five models trained and validated on different splits of the training set. It is noteworthy that in this study, we deliberately refrained from incorporating pre-trained weights or implementing deep supervision for either CNNs or transformer-based models.

### 3.4. Reliability metrics

We assess the reliability of the model based on four aspects: robustness, misclassification detection, OOD detection, and calibration De Jorge et al. (2023).

#### Robustness

We assess robustness through the evaluation of various metrics at different levels: voxel level for segmentation, image level for classification, and lesion
level for detection. The employed metrics encompass volume agreement measures such as volumetric similarity (VS) and average volume distance (AVD), as well as overlap agreement metrics like Dice, precision, and recall. Additionally, distance metrics such as Average Symmetric Surface Distance (ASSD), Hausdorff Distance, $q = 95^{th}$ percentile (HD95), and Surface Dice at Tolerance (SDT) are considered.

330

For image-level evaluation, we include sensitivity, specificity, precision, correct classification ratio (CCR), and the area under the curve (AUC). The lesion-level analysis is conducted through free receiver operating characteristic (FROC) analysis. A single score is extracted from the FROC curve by measuring sensitivity at six predefined false positive rates: 1/8, 1/4, 1/2, 1, 2, and 4 false positives per scan Van Ginneken et al. (2010). These values are averaged to derive the overall system score.

335

Additionally, a pairwise statistical comparison of the seven architectures, based on VS, is conducted using a significance matrix. This matrix is derived from a one-sided Wilcoxon signed-rank test at a 5% significance level, with adjustments for multiple comparisons made using the Holm method Wiesenfarth et al. (2021). The p-values are presented in an upper triangular matrix, facilitating the identification of statistically significant differences between each pair of architectures. VS was chosen as the primary metric for statistical analysis due to its independence from the reference annotation volume, its ability to account for uncertainty in the reference annotations, and its capacity to evaluate empty agreement. These characteristics make VS particularly well-suited for robust and unbiased comparisons across architectures.

340

345

350

All assessments are performed using the USE-Evaluator Ostmeier et al. (2023). For each study, we compute a 95% confidence interval using bootstrapping with 1,000 bootstraps. Each iteration employs a new set of candidates sampled at the scan level with replacement.

355

### Misclassification detection

A crucial requirement for a reliable model in medical image segmentation is its ability to exhibit higher confidence levels for correct outputs compared to incorrect ones. Ideally, if we arranged all predictions in ascending order of confidence, incorrect predictions would precede correct ones. Misclassification detection measures how far away are we from such an ideal case.

360

We employ the Prediction Rejection Ratio (PRR) to assess misclassification detection Malinin et al. (2020), which yields a positive score when the model's confidence surpasses a random baseline, as detailed in De Jorge et al. (2023). Notably, PRR achieves its maximum score of 100% when the model aligns perfectly with an oracle. In the context of segmentation, where predictions are made at a per-voxel level, the sheer number of samples becomes overwhelming if we consider the entire volume. To address this, we strategically sub-sample the number of pixels per volume by restricting our analysis to voxels within the prostate. This approach helps manage the computational load while maintaining the integrity of the evaluation process.

365

370
### Out-of-domain detection

Another crucial aspect of reliability is that models should be aware of their domain of expertise. When a sample significantly deviates from the training samples, we anticipate the model exhibiting higher uncertainty in its predictions. For example, a model trained to find tumors using specific machines and methods might struggle when it faces images with different conditions or recorded with different tools in real clinical situations. Therefore, it is insufficient for models to merely make accurate predictions on the content they were trained on; they should also possess the capability to convey whether a new input is familiar enough to yield a reliable prediction.

375

380

We consider datasets in a manner that presents a notable domain shift; specifically, we designate **PICAI** as in-domain and **Prostate 158** as out-of-domain. Consequently, for each patient case, we employ the predicted csPCa lesion detection map to compute a singular floating-point value, ranging from 0 to 1, representing the patient's overall likelihood of harboring csPCa. This value corresponds to the maximum value obtained from the csPCa lesion detection map. To quantify the model's performance, we utilize the Area Under the Receiver Operating Characteristic curve (AUROC), a metric ranging from 0 to 1, with 1 signifying optimal performance.

385

390

### Calibration

A model is considered calibrated when its predictive probabilities closely match the true probabilities. In practical terms, if we group all samples where the predicted probability is 90%, we would expect that 90% of those predictions are correct. To measure this, we use the most widely adopted metric, Expected Calibration Error (ECE), which quantifies the difference between predicted and actual probabilities.

395

400

In segmentation tasks, where predictions are made on a per-voxel basis and the number of voxels is significantly high, calculating ECE can become computationally intensive. To mitigate this, we subsampled the number of voxels per scan, focusing exclusively on those within the prostate. This approach reduces the computational load of ECE estimation while maintaining a focus on the relevant region for prostate cancer detection and segmentation.

Our analysis does not involve performing any model calibration. Instead, our primary objective is to evaluate how well a model's confidence (its prediction) corresponds to the actual likelihood of being correct (its accuracy).

405

## 4. Results and Discussions

In the following, we present the main findings of our reliability analysis of transformed-based models, considering four-fold; robustness, misclassification detection out-of-domain detection, and calibration.

410

### Robustness

Robustness in medical image segmentation heavily depends on the choice of metrics and the level of performance evaluation. We examine in-domain
(PICAI) and out-of-distribution scenarios across three levels: segmentation, classification, and lesion. In in-domain cases, focusing on segmentation performance within PICAI, different models excel depending on the metric used. MCNN+ViTs-B/1(+) outperforms others in volume agreement, closely followed by U-Net(●). Generally, architectures based on local representations, such as pure CNNs like MedNeXt(■) or hybrid architectures like MCNN+ViTs-B/1 (+) and CNN+ViT<sub>v</sub>-B/1(▲), perform better. In contrast, models based solely on transformers, such as UNETR(★), Swin UNETR(◆), and nnFormer(※), tend to underperform in this category. Notably, the VS metric equals 1 when the models correctly detect the absence of csPCa.

The trend continues in the overlap category, with MCNN+ViT<sub>s</sub>-B/1(+) and U-Net(●) outperforming other models. However, a larger gap is observed when comparing these models with UNETR(★) and nnFormer(※). This discrepancy arises because the overlap metrics are computed only for cases with csPCa, unlike the VS metric. In distance-based metrics, MedNeXt(■) generally scores the highest, closely followed by U-Net(●). When evaluating the robustness of classification performance, UNETR(★) takes the lead with the best AUC score and notably superior sensitivity in the in-domain scenario. This highlights the effectiveness of transformer-based encoder models for classification tasks. At the lesion level, UNETR(★) again shows the best results, reflecting its ability to represent long-range dependencies. Notably, MCNN+ViT<sub>s</sub>-B/1(+) closely follows in average CPM. Table 1 summarizes the robustness findings for the in-domain analysis.

Moving out of the domain, the hierarchical transformer architecture, Swin UNETR (◆), performs better under volume metrics followed by MedNeXt(■) model. However, there's a significant performance drop when moving out-of-distribution, indicating sensitivity to domain shifts. In overlap and distance metrics, when only csPCa cases are considered, U-Net(●) continues to outperform other models, now followed by MedNeXt(■). Notably, MCNN+ViT<sub>s</sub>-B/1 (+) experiences a considerable drop in performance, indicating higher sensitivity to OOD datasets at the voxel level. In image-level classification, the transformer-based models, particularly Swin UNETR(◆) and UNETR(★), appear to perform better in out-of-distribution scenarios, maintaining the highest sensitivity and positive prediction ratio. It seems that Swin UNETR(◆) slightly outperforms UNETR(★) and appears to be less affected by out-of-distribution scenarios, presenting the best CCR and AUC results. At the lesion level, the trend remains consistent, with UNETR(★) being the preferred option closed followed by Swin UNETR(◆) in CPM metrics with values of 0.41 and 0.40, respectively. The findings for the robustness assessment in out-of-distribution scenarios are summarized in detail in Table 2 and Table 3, for Rater A and Rater B, respectively. Additionally, the complete distribution of the VS, Dice, and ASSD scores for each dataset is represented in the boxplots shown in Figures 2, 3, and 4, respectively. These figures also include the number of correctly classified csPCa predictions.

The significance maps of VS in Fig. 5 demonstrate that, within the in-domain setting, the uncertainty is too high to identify statistically meaningful differences
<table><caption>Table 1: Results of Robustness performance for PICAI dataset.</caption><thead><tr><th colspan="8">Segmentation performance</th></tr><tr><th colspan="2">Categories</th><th colspan="7">PICAI - (CsPCA=44, Non-CsPCA=216)</th></tr><tr><th></th><th>Metrics</th><th>(●)</th><th>(■)</th><th>(◆)</th><th>(●)</th><th>(※)</th><th>(▲)</th><th>(★)</th></tr></thead><tbody><tr><td rowspan="10">Voxel-level</td><td>Volume</td><td>VS (↑)<br>AVD (↓)</td><td><b>0.84 ±0.02</b><br>1.38 ±0.27</td><td>0.83 ±0.02<br>1.09 ±0.19</td><td>0.80 ±0.03<br>1.05 ±0.15</td><td><b>0.84 ±0.02</b><br><b>0.98 ±0.15</b></td><td>0.82 ±0.02<br>1.68 ±0.32</td><td>0.83 ±0.02<br>1.25 ±0.21</td><td>0.81 ±0.03<br>1.04 ±0.17</td></tr><tr><td rowspan="3">Overlap</td><td>Dice (↑)</td><td>0.52 ±0.07</td><td>0.53 ±0.05</td><td>0.52 ±0.05</td><td><b>0.54 ±0.04</b></td><td>0.39 ±0.05</td><td>0.51 ±0.04</td><td>0.45 ±0.04</td></tr><tr><td>Precision (↑)</td><td><b>0.84 ±0.03</b></td><td>0.77 ±0.04</td><td>0.66 ±0.04</td><td>0.73 ±0.04</td><td>0.68 ±0.05</td><td>0.77 ±0.04</td><td>0.63 ±0.05</td></tr><tr><td>Recall (↑)</td><td>0.49 ±0.09</td><td>0.53 ±0.06</td><td><b>0.56 ±0.06</b></td><td>0.53 ±0.05</td><td>0.35 ±0.06</td><td>0.48 ±0.06</td><td>0.44 ±0.05</td></tr><tr><td rowspan="3">Distance</td><td>ASSD (↓)</td><td>2.05 ±0.48</td><td><b>1.84 ±0.32</b></td><td>2.86 ±0.76</td><td>2.25 ±0.3</td><td>2.88 ±0.4</td><td>2.13 ±0.28</td><td>5.25 ±1.37</td></tr><tr><td>HD95 (↓)</td><td>7.56 ±1.6</td><td><b>6.62 ±1.05</b></td><td>10.53 ±1.53</td><td>8.77 ±1.4</td><td>8.56 ±1.24</td><td>8.10 ±1.22</td><td>13.18 ±1.9</td></tr><tr><td>SDT (small) (↑)</td><td>0.64 ±0.06</td><td><b>0.66 ±0.05</b></td><td><b>0.66 ±0.04</b></td><td>0.65 ±0.04</td><td>0.51 ±0.05</td><td>0.63 ±0.04</td><td>0.57 ±0.05</td></tr><tr><td rowspan="2"></td><td>SDT (large) (↑)</td><td>0.84 ±0.06</td><td>0.86 ±0.04</td><td>0.84 ±0.04</td><td><b>0.88 ±0.02</b></td><td>0.79 ±0.04</td><td>0.86 ±0.03</td><td>0.75 ±0.05</td></tr><tr><th colspan="8">Classification performance</th></tr><tr><td rowspan="6">Image-level</td><td>Np/Ntotal</td><td>16/44</td><td>29/44</td><td>33/44</td><td>32/44</td><td>26/44</td><td>29/44</td><td><b>39/44</b></td></tr><tr><td>Sensitivity</td><td>0.36 ±0.0</td><td>0.66 ±0.07</td><td>0.75 ±0.07</td><td>0.73 ±0.07</td><td>0.59 ±0.07</td><td>0.66 ±0.07</td><td><b>0.89 ±0.05</b></td></tr><tr><td>Specificity</td><td><b>0.83 ±0.0</b></td><td>0.77 ±0.0</td><td>0.60 ±0.0</td><td>0.63 ±0.0</td><td><b>0.83 ±0.0</b></td><td>0.73 ±0.0</td><td>0.56 ±0.0</td></tr><tr><td>Precision</td><td>0.31 ±0.0</td><td>0.37 ±0.0</td><td>0.28 ±0.0</td><td>0.29 ±0.0</td><td><b>0.41 ±0.0</b></td><td>0.33 ±0.0</td><td>0.29 ±0.0</td></tr><tr><td>CCR</td><td>0.75 ±0.03</td><td>0.75 ±0.03</td><td>0.63 ±0.03</td><td>0.65 ±0.03</td><td><b>0.79 ±0.03</b></td><td>0.72 ±0.03</td><td>0.62 ±0.03</td></tr><tr><td>AUC</td><td>0.63 ±0.04</td><td>0.74 ±0.04</td><td>0.73 ±0.04</td><td>0.74 ±0.04</td><td>0.72 ±0.04</td><td>0.73 ±0.04</td><td><b>0.77 ±0.04</b></td></tr><tr><th colspan="8">Lesion detection performance</th></tr><tr><td rowspan="6">Lesion-level</td><td>fps-1/8</td><td>0.29 ±0.0</td><td>0.46 ±0.0</td><td>0.43 ±0.0</td><td><b>0.51 ±0.0</b></td><td>0.21 ±0.0</td><td>0.34 ±0.0</td><td>0.40 ±0.0</td></tr><tr><td>fps-1/4</td><td>0.31 ±0.0</td><td>0.53 ±0.0</td><td>0.58 ±0.0</td><td><b>0.64 ±0.0</b></td><td>0.21 ±0.0</td><td>0.34 ±0.0</td><td>0.58 ±0.0</td></tr><tr><td>fps-1/2</td><td>0.31 ±0.0</td><td>0.53 ±0.0</td><td>0.67 ±0.0</td><td>0.65 ±0.0</td><td>0.21 ±0.0</td><td>0.34 ±0.0</td><td><b>0.69 ±0.0</b></td></tr><tr><td>fps-1</td><td>0.31 ±0.0</td><td>0.53 ±0.0</td><td>0.67 ±0.0</td><td>0.65 ±0.0</td><td>0.21 ±0.0</td><td>0.34 ±0.0</td><td><b>0.71 ±0.0</b></td></tr><tr><td>fps-2</td><td>0.31 ±0.0</td><td>0.53 ±0.0</td><td>0.67 ±0.0</td><td>0.65 ±0.0</td><td>0.21 ±0.0</td><td>0.34 ±0.0</td><td><b>0.71 ±0.0</b></td></tr><tr><td>fps-4</td><td>0.31 ±0.0</td><td>0.53 ±0.0</td><td>0.67 ±0.0</td><td>0.65 ±0.0</td><td>0.21 ±0.0</td><td>0.34 ±0.0</td><td><b>0.71 ±0.0</b></td></tr><tr><td colspan="2">CPM</td><td>0.31 ±0.0</td><td>0.52 ±0.0</td><td>0.61 ±0.0</td><td>0.62 ±0.0</td><td>0.21 ±0.0</td><td>0.34 ±0.0</td><td><b>0.63 ±0.0</b></td></tr></tbody></table>

<figure><img src="image_5.png" /><figcaption>Figure 2: Boxplot illustration of VS metric from left to right: ID PICAI, OOD Prostate 158 - Rater A, and Prostate 158 - Rater A, respectively.</figcaption></figure>
<table><caption>Table 2: Results of Robustness performance for Prostate 158 - Rater A dataset.</caption><thead><tr><th colspan="8">Segmentation performance</th></tr><tr><th colspan="2">Categories</th><th colspan="7">Prostate 158 - Rater A - (CsPCA=102, Non-CsPCA=56)</th></tr><tr><th></th><th>Metrics</th><th>(●)</th><th>(■)</th><th>(♦)</th><th>(★)</th><th>(*)</th><th>(▲)</th><th>(★)</th></tr></thead><tbody><tr><td rowspan="10">Voxel-level</td><td>Volume</td><td>VS (↑)<br>AVD (↓)</td><td>0.44 ±0.04<br>2.37 ±0.33</td><td>0.54 ±0.04<br>1.98 ±0.33</td><td><strong>0.55 ±0.03</strong><br><strong>1.94 ±0.31</strong></td><td>0.52 ±0.04<br>2.28 ±0.39</td><td>0.45 ±0.04<br>2.53 ±0.4</td><td>0.53 ±0.03<br>1.99 ±0.32</td><td>0.50 ±0.04<br>2.03 ±0.34</td></tr><tr><td rowspan="3">Overlap</td><td>Dice (↑)</td><td>0.45 ±0.05</td><td><strong>0.46 ±0.04</strong></td><td>0.42 ±0.04</td><td>0.38 ±0.04</td><td>0.36 ±0.05</td><td>0.44 ±0.04</td><td>0.38 ±0.04</td></tr><tr><td>Precision (↑)</td><td><strong>0.81 ±0.04</strong></td><td>0.75 ±0.04</td><td>0.74 ±0.04</td><td>0.75 ±0.04</td><td>0.79 ±0.04</td><td>0.80 ±0.04</td><td>0.72 ±0.04</td></tr><tr><td>Recall (↑)</td><td><strong>0.41 ±0.06</strong></td><td>0.40 ±0.04</td><td>0.35 ±0.04</td><td>0.33 ±0.04</td><td>0.31 ±0.05</td><td>0.37 ±0.04</td><td>0.33 ±0.04</td></tr><tr><td rowspan="3">Distance</td><td>ASSD (↓)</td><td><strong>2.41 ±0.36</strong></td><td>3.47 ±0.66</td><td>4.95 ±1.17</td><td>5.28 ±1.08</td><td>3.83 ±0.66</td><td>4.67 ±1.01</td><td>4.10 ±0.77</td></tr><tr><td>HD95 (↓)</td><td><strong>9.19 ±1.42</strong></td><td>11.40 ±1.29</td><td>12.38 ±1.43</td><td>13.94 ±1.47</td><td>12.51 ±1.53</td><td>12.76 ±1.5</td><td>11.14 ±1.19</td></tr><tr><td>SDT (small) (↑)</td><td>0.54 ±0.05</td><td><strong>0.55 ±0.04</strong></td><td>0.51 ±0.04</td><td>0.47 ±0.04</td><td>0.43 ±0.05</td><td>0.52 ±0.04</td><td>0.50 ±0.04</td></tr><tr><td rowspan="2">Image-level</td><td>SDT (large) (↑)</td><td><strong>0.79 ±0.04</strong></td><td>0.76 ±0.04</td><td>0.72 ±0.04</td><td>0.66 ±0.04</td><td>0.67 ±0.05</td><td>0.71 ±0.04</td><td>0.74 ±0.03</td></tr><tr><th colspan="8">Classification performance</th></tr><tr><td rowspan="7">Lesion-level</td><td>Np/Ntotal</td><td>27/102</td><td>51/102</td><td>62/102</td><td>59/102</td><td>39/102</td><td>53/102</td><td><strong>64/102</strong></td></tr><tr><td>Sensitivity</td><td>0.26 ±0.04</td><td>0.50 ±0.05</td><td>0.61 ±0.05</td><td>0.58 ±0.05</td><td>0.38 ±0.05</td><td>0.52 ±0.05</td><td><strong>0.63 ±0.05</strong></td></tr><tr><td>Specificity</td><td><strong>0.98 ±0.0</strong></td><td>0.96 ±0.0</td><td>0.86 ±0.0</td><td>0.93 ±0.0</td><td><strong>0.98 ±0.0</strong></td><td>0.95 ±0.0</td><td>0.84 ±0.0</td></tr><tr><td>Precision</td><td>0.96 ±0.0</td><td>0.96 ±0.0</td><td>0.89 ±0.0</td><td>0.94 ±0.0</td><td><strong>0.97 ±0.0</strong></td><td>0.95 ±0.0</td><td>0.88 ±0.0</td></tr><tr><td>CCR</td><td>0.52 ±0.04</td><td>0.66 ±0.04</td><td><strong>0.70 ±0.04</strong></td><td><strong>0.70 ±0.04</strong></td><td>0.59 ±0.04</td><td>0.67 ±0.04</td><td><strong>0.70 ±0.04</strong></td></tr><tr><td>AUC</td><td>0.63 ±0.02</td><td>0.75 ±0.03</td><td><strong>0.78 ±0.03</strong></td><td>0.77 ±0.03</td><td>0.70 ±0.03</td><td>0.76 ±0.03</td><td>0.76 ±0.03</td></tr><tr><th colspan="8">Lesion detection performance</th></tr><tr><td rowspan="7">Lesion-level</td><td>fps-1/8</td><td>0.19 ±0.0</td><td>0.32 ±0.0</td><td><strong>0.39 ±0.0</strong></td><td>0.35 ±0.0</td><td>0.21 ±0.0</td><td>0.34 ±0.0</td><td>0.38 ±0.0</td></tr><tr><td>fps-1/4</td><td>0.19 ±0.0</td><td>0.32 ±0.0</td><td>0.40 ±0.0</td><td>0.35 ±0.0</td><td>0.21 ±0.0</td><td>0.34 ±0.0</td><td><strong>0.41 ±0.0</strong></td></tr><tr><td>fps-1/2</td><td>0.19 ±0.0</td><td>0.32 ±0.0</td><td>0.40 ±0.0</td><td>0.35 ±0.0</td><td>0.21 ±0.0</td><td>0.34 ±0.0</td><td><strong>0.41 ±0.0</strong></td></tr><tr><td>fps-1</td><td>0.19 ±0.0</td><td>0.32 ±0.0</td><td>0.40 ±0.0</td><td>0.35 ±0.0</td><td>0.21 ±0.0</td><td>0.34 ±0.0</td><td><strong>0.41 ±0.0</strong></td></tr><tr><td>fps-2</td><td>0.19 ±0.0</td><td>0.32 ±0.0</td><td>0.40 ±0.0</td><td>0.35 ±0.0</td><td>0.21 ±0.0</td><td>0.34 ±0.0</td><td><strong>0.41 ±0.0</strong></td></tr><tr><td>fps-4</td><td>0.19 ±0.0</td><td>0.32 ±0.0</td><td>0.40 ±0.0</td><td>0.35 ±0.0</td><td>0.21 ±0.0</td><td>0.34 ±0.0</td><td><strong>0.41 ±0.0</strong></td></tr><tr><td>CPM</td><td>0.19 ±0.0</td><td>0.32 ±0.0</td><td>0.40 ±0.0</td><td>0.35 ±0.0</td><td>0.21 ±0.0</td><td>0.34 ±0.0</td><td><strong>0.41 ±0.0</strong></td></tr></tbody></table>

<figure><img src="image_6.png" /><figcaption>Figure 3: Boxplot illustration of Dice metric from left to right: ID PICAI, OOD Prostate 158 - Rater A, and Prostate 158 - Rater B, respectively.</figcaption></figure>
<table><caption>Table 3: Results of Robustness performance for Prostate 158 - Rater B dataset.</caption><thead><tr><th colspan="8">Segmentation performance</th></tr><tr><th colspan="2">Categories</th><th>Metrics</th><th colspan="6">Prostate 158 - Rater B - (CsPCA=102, Non-CsPCA=56)</th></tr><tr><th></th><th></th><th></th><th>(●)</th><th>(■)</th><th>(▼)</th><th>(▼)</th><th>(※)</th><th>(▲)</th><th>(★)</th></tr></thead><tbody><tr><td rowspan="10">Voxel-level</td><td rowspan="2">Volume</td><td>VS (↑)</td><td>0.45 ±0.04</td><td>0.54 ±0.04</td><td><strong>0.57 ±0.03</strong></td><td>0.53 ±0.04</td><td>0.46 ±0.04</td><td>0.55 ±0.04</td><td>0.52 ±0.04</td></tr><tr><td>AVD (↓)</td><td>2.08 ±0.35</td><td>1.82 ±0.33</td><td><strong>1.72 ±0.32</strong></td><td>2.15 ±0.43</td><td>2.30 ±0.44</td><td>1.86 ±0.34</td><td>1.88 ±0.34</td></tr><tr><td rowspan="3">Overlap</td><td>Dice (↑)</td><td><strong>0.47 ±0.05</strong></td><td>0.45 ±0.04</td><td>0.42 ±0.04</td><td>0.39 ±0.04</td><td>0.36 ±0.05</td><td>0.45 ±0.04</td><td>0.39 ±0.04</td></tr><tr><td>Precision (↑)</td><td><strong>0.82 ±0.04</strong></td><td>0.70 ±0.04</td><td>0.70 ±0.04</td><td>0.70 ±0.04</td><td>0.75 ±0.05</td><td>0.75 ±0.04</td><td>0.67 ±0.04</td></tr><tr><td>Recall (↑)</td><td><strong>0.43 ±0.06</strong></td><td>0.41 ±0.04</td><td>0.38 ±0.04</td><td>0.36 ±0.04</td><td>0.33 ±0.05</td><td>0.39 ±0.04</td><td>0.36 ±0.04</td></tr><tr><td rowspan="4">Distance</td><td>ASSD (↓)</td><td><strong>2.55 ±0.37</strong></td><td>3.73 ±0.7</td><td>5.06 ±1.12</td><td>5.40 ±1.07</td><td>4.59 ±0.88</td><td>4.63 ±1.03</td><td>4.34 ±0.8</td></tr><tr><td>HD95 (↓)</td><td><strong>10.36 ±1.72</strong></td><td>11.41 ±1.36</td><td>11.98 ±1.44</td><td>13.97 ±1.49</td><td>12.37 ±1.48</td><td>12.51 ±1.57</td><td>11.11 ±1.24</td></tr><tr><td>SDT (small) (↑)</td><td><strong>0.56 ±0.05</strong></td><td>0.55 ±0.04</td><td>0.52 ±0.04</td><td>0.48 ±0.04</td><td>0.44 ±0.05</td><td>0.75 ±0.04</td><td>0.51 ±0.04</td></tr><tr><td>SDT (large) (↑)</td><td><strong>0.80 ±0.04</strong></td><td>0.75 ±0.04</td><td>0.72 ±0.04</td><td><strong>0.67 ±0.04</strong></td><td>0.67 ±0.05</td><td>0.39 ±0.04</td><td>0.74 ±0.04</td></tr><tr><th colspan="8">Classification performance</th></tr><tr><td rowspan="6">Image-level</td><td>Np/Ntotal</td><td>27/102</td><td>51/102</td><td>62/102</td><td>59/102</td><td>39/102</td><td>53/102</td><td><strong>64/102</strong></td></tr><tr><td>Sensitivity</td><td>0.27 ±0.04</td><td>0.50 ±0.05</td><td>0.61 ±0.05</td><td>0.58 ±0.05</td><td>0.39 ±0.05</td><td>0.52 ±0.05</td><td><strong>0.63 ±0.05</strong></td></tr><tr><td>Specificity</td><td><strong>0.98 ±0.0</strong></td><td>0.96 ±0.0</td><td>0.86 ±0.0</td><td>0.93 ±0.0</td><td><strong>0.98 ±0.0</strong></td><td>0.95 ±0.0</td><td>0.84 ±0.0</td></tr><tr><td>Precision</td><td>0.96 ±0.0</td><td>0.96 ±0.0</td><td>0.89 ±0.0</td><td>0.94 ±0.0</td><td><strong>0.97 ±0.0</strong></td><td>0.95 ±0.0</td><td>0.88 ±0.0</td></tr><tr><td>CCR</td><td>0.53 ±0.04</td><td>0.67 ±0.04</td><td>0.70 ±0.04</td><td><strong>0.71 ±0.04</strong></td><td>0.60 ±0.04</td><td>0.68 ±0.04</td><td><strong>0.71 ±0.04</strong></td></tr><tr><td>AUC</td><td>0.63 ±0.02</td><td>0.75 ±0.03</td><td><strong>0.79 ±0.03</strong></td><td>0.77 ±0.03</td><td>0.71 ±0.03</td><td>0.77 ±0.03</td><td>0.77 ±0.03</td></tr><tr><th colspan="8">Lesion detection performance</th></tr><tr><td rowspan="6">Lesion-level</td><td>fps-1/8</td><td>0.20 ±0.0</td><td>0.35 ±0.0</td><td><strong>0.39 ±0.0</strong></td><td>0.37 ±0.0</td><td>0.21 ±0.0</td><td>0.36 ±0.0</td><td><strong>0.39 ±0.0</strong></td></tr><tr><td>fps-1/4</td><td>0.20 ±0.0</td><td>0.35 ±0.0</td><td><strong>0.41 ±0.0</strong></td><td>0.37 ±0.0</td><td>0.21 ±0.0</td><td>0.37 ±0.0</td><td><strong>0.41 ±0.0</strong></td></tr><tr><td>fps-1/2</td><td>0.20 ±0.0</td><td>0.35 ±0.0</td><td>0.41 ±0.0</td><td>0.37 ±0.0</td><td>0.21 ±0.0</td><td>0.37 ±0.0</td><td><strong>0.43 ±0.0</strong></td></tr><tr><td>fps-1</td><td>0.20 ±0.0</td><td>0.35 ±0.0</td><td>0.41 ±0.0</td><td>0.37 ±0.0</td><td>0.21 ±0.0</td><td>0.37 ±0.0</td><td><strong>0.43 ±0.0</strong></td></tr><tr><td>fps-2</td><td>0.20 ±0.0</td><td>0.35 ±0.0</td><td>0.41 ±0.0</td><td>0.37 ±0.0</td><td>0.21 ±0.0</td><td>0.37 ±0.0</td><td><strong>0.43 ±0.0</strong></td></tr><tr><td>fps-4</td><td>0.20 ±0.0</td><td>0.35 ±0.0</td><td>0.41 ±0.0</td><td>0.37 ±0.0</td><td>0.21 ±0.0</td><td>0.37 ±0.0</td><td><strong>0.43 ±0.0</strong></td></tr><tr><td colspan="2">CPM</td><td>0.20 ±0.0</td><td>0.35 ±0.0</td><td>0.41 ±0.0</td><td>0.37 ±0.0</td><td>0.21 ±0.0</td><td>0.37 ±0.0</td><td><strong>0.42 ±0.0</strong></td></tr></tbody></table>

<figure><img src="image_7.png" /><figcaption>Figure 4: Boxplot illustration of ASSD metric from left to right: ID PICAI, OOD Prostate 158 - Rater A, and Prostate 158 - Rater A, respectively.</figcaption></figure>

between the architectures. Although U-Net(●) achieves the highest VS score in this setting, this should not be interpreted as substantial superiority over the other models, as the significance map reveals no strong statistical support for this difference. In contrast, the out-of-domain analysis shows that nnFormer(※) and U-Net(●) exhibit statistically significant differences from the other models,

460
indicating that these two architectures perform worse and may generalize less effectively to unseen data. It is also important to highlight that the VS metric offers a more comprehensive evaluation by considering the correct classification of lesion presence, which is often overlooked by simpler metrics like Dice.

<figure><img src="image_8.png" /><figcaption>Figure 5: Visualization of statistical analysis in VS, computed using a one-sided Wilcoxon signed-rank test at a 5% significance level with adjustment for multiple testing according to Holm Wiesenfarth et al. (2021). Yellow shading indicates that the VS of the architectures on the x-axis are significantly superior to those on the y-axis, while blue shading indicates no significant superiority. From left to right: ID PICAI, OOD Prostate 158 - Rater A, and Prostate 158 - Rater B, respectively.</figcaption></figure>

These results generally indicate that models incorporating the CNN inductive bias perform better at voxel-level metrics but do not show the same level of performance at lesion and image levels. This suggests that while CNN-based models excel in detailed, pixel-level tasks, they may lack the critical global context understanding for lesion and image-level evaluations. In contrast, transformer-based models, which naturally capture broader contextual information, may be better suited for these higher-level tasks. However, recent advancements in medical image segmentation have shown that transformer-based networks, especially those pre-trained on large-scale datasets, such as foundational models Kirillov et al. (2023); Ravi et al. (2024), hold significant potential. The extensive pre-training on diverse and expansive datasets allows these models to generalize better across various medical imaging tasks, including those involving complex structures and variable imaging conditions Mazurowski et al. (2023).

In the context of image-level analysis, Figures 6, 7, and 8 provide a detailed representation of the confusion matrix for both in-domain and out-of-distribution datasets, presented as scatter plots. The volumes are expressed in milliliters (ml) and visualized on a log scale, with a threshold of $10^{-2}$ ml ensuring the inclusion of all relevant samples. To aid interpretation, each model is uniquely differentiated by color and symbol, as outlined in the Materials and Methods section.

Analyzing the PICAI dataset (Figure 6), a notable pattern emerges: the majority of incorrectly classified false negative cases involve small lesions, ranging
<figure><img src="image_9.png" /><figcaption>Figure 6: Scatter plot with a log-scale and a confusion matrix with a volume threshold of 10⁻² ml for ID PICAI dataset. FP, TP, TN, and FN per model are displayed at the top corner of each quadrant.</figcaption></figure>

from approximately 0.1 to 1 ml. U-Net(●) stands out with the highest number of false negatives (28 cases), contrasting with the more reliable UNETR(★) which exhibits only 5 cases erroneously classified as negative. Interestingly, transformers-based encoder networks such as Swin UNETR and UNETR display a distinct issue of over-segmentation, contributing to a noteworthy increase in false positives, 94 and 86 cases, respectively. On the other hand, models like U-Net and nnFormer(※) only have 36 and 37 cases wrongly classified as false positives.

Expanding our observations to out-of-distribution scenarios (Figure 7, 8), a consistent trend emerges where also larger lesions are misclassified as false negative. U-Net(●) still stands out as the model with the highest number of false negatives cases (75), meanwhile UNETR(★) shows up as the model with the fewer false negative cases (38). This underscores the dual challenge of accurately segmenting and detecting lesions, particularly in unfamiliar distributions. Notably, inter-rater variability in volume delineation introduces a layer of complexity; the choice between Rater A and Rater B for evaluation results in varying ranges of false negatives. For instance, Rater B encompasses a broader spectrum of false negatives, ranging from 0.1 to 10 ml, while Rater A tends to misclassify tumors when their size is around 1 ml. Concerning the true
<figure><img src="image_10.png" /><figcaption>Figure 7: Scatter plot with a log-scale and a confusion matrix with a volume threshold of 10⁻² ml for ID Prostate 158 Rater A dataset. FP, TP, TN, and FN per model are displayed at the top corner of each quadrant.</figcaption></figure>

positive values, Swin UNETR and UNETR have the best csPCa detection rate. These insightful findings underscore the fact that transformers-based models are good at detecting csPCa lesions but also highlight that they tend to over-detect benign regions.

It is worth noting that the current results are highly dependent on accurate whole prostate segmentation. While this segmentation process is generally robust, with reported Dice scores around 92% Aldoj et al. (2020); Bhandary et al. (2023), it is not entirely free from errors. To mitigate potential issues, we implemented specific measures: we extended the AI-based segmentation mask obtained by the nn-UNet framework in the *x*, *y*, and *z* dimensions to ensure more comprehensive coverage of the prostate. During the training phase, we carefully reviewed and manually corrected a few instances of incorrect segmentation, primarily related to over-segmentation. Notably, in our evaluation of both ID and OOD datasets, no significant segmentation errors of this nature were observed. Despite this, the risk of losing critical information in the tumor region cannot be entirely dismissed, especially in complex cases with ambiguous tumor boundaries or substantial variability in imaging quality.
<figure><img src="image_11.png" /><figcaption>Figure 8: Scatter plot with a log-scale and a confusion matrix with a volume threshold of 10⁻² ml for ID Prostate 158 Rater B dataset. FP, TP, TN, and FN per model are displayed at the top corner of each quadrant.</figcaption></figure>

## Misclassification detection

In Figure 9, we conduct a comprehensive comparison of all models based on misclassification detection versus robustness at the voxel level, represented by PRR score (↑) versus Volume Similarity (VS) (↑). Within the same domain, considering all models, higher VS looks like implying higher PRR. Notably, the hybrid architecture MCNN+ViT<sub>s</sub>-B/1 outperforms more recent transformer-based architectures, indicating its effectiveness in in-domain setting.

As we extend our analysis to different domains, specifically for Prostate158 Rater A and Prostate158 Rater B, recent models like MedNeXt(■) and MCNN +ViT<sub>s</sub> -B/1(♦) excel in both VS and PRR. However, Swin UNETR(◆) unexpectedly shows a low PRR in the out-of-distribution setting. This observation challenges the general trend observed with recent models.

The PRR results highlight the trade-off between residual error and the proportion of samples rejected by a model based on confidence thresholds. In the in-domain (ID) setting, the PRR demonstrates a strong potential for reducing network errors by rejecting low-confidence samples. However, in out-of-domain (OOD) scenarios, this error reduction is less pronounced, with models like Swin UNETR (◆) being notably impacted, while others continue to perform robustly.
To understand these findings, it is essential to consider the complexities inherent in each model’s architecture and training strategies. The interaction between robustness and misclassification detection under varying domain shifts may be heavily influenced by specific design decisions, highlighting the importance of a detailed examination of model behavior across different contexts. These results underscore the necessity of evaluating multiple metrics and considering domain-specific factors when assessing the performance of deep learning models in medical image segmentation.

<figure><img src="image_12.png" /><figcaption>Figure 9: PRR (↑) vs. VS (↑): Labels A and B represent Prostate 158 - Rater A and Prostate 158 - Rater B, respectively. MCNN+ViT$_{S}$-B/1(+) outperform more recent models (other markers) in ID misclassification detection (zoomed plot), and the trend continues over domain shift</figcaption></figure>

### Out-of-domain detection

In Figure 10, we present a comparative analysis of OOD detection versus robustness at the voxel level, represented by AUROC score and VS, respectively, across all models. Our OOD detection methodology aligns with the approach introduced in De Jorge et al. (2023), wherein ID volumes (PICAI) are distinguished from OOD volumes (Prostate 158). Consequently, the y-axis remains consistent across all three plots, with only the VS varying. In the ID scenario, as is typical, MCNN+ViT$_{S}$-B/1(+) exhibits the most favorable trade-off between AUC and VS, closely followed by MedNeXt(■). This suggests that hybrid and CNN-based models perform better in OOD detection within the ID distribution. However, a noteworthy shift occurs in the OD distribution, where
Swin UNETR(♦) distinctly demonstrates the best trade-off between OOD detection and robustness. A closer examination of Prostate 158 - Rater A and Prostate 158 - Rater B reveals a negative trend in U-Net(●), transitioning from superior performance in ID to one of the least effective in OOD. As expected, nnFormer(※) consistently exhibits the poorest performance, both in ID and OOD scenarios.

<figure><img src="image_13.png" /><figcaption>Figure 10: Comparison of AUROC (↑) and VS (↑) reveals that MCNN+ViTs-B/1(♦) and MedNeXt(■) stand out as the best models for OOD detection in the domain. However, a shift in the trend occurs with Swin UNETR(♦), while MedNeXt(■) remains consistently effective in OOD detection.</figcaption></figure>

## Calibration

Figure 11 presents a visual representation of each model’s calibration, specifically the Expected Calibration Error (ECE), under different domain shifts. In the in-domain (ID) setup, the hybrid models MCNN+ViTs-B/1(♦) and U-Net(●) showcase an optimal balance between ECE and error. However, more recent transformer-based architectures like Swin UNETR(♦), nnFormer(※), UNETR(★), and its counterpart MedNeXt(■) exhibit challenges in calibration. Despite improvements in robustness, these newer models show only slight enhancements in calibration within the ID setting. Notably, MedNeXt(■), Swin UNETR(♦), and nnFormer(※) display elevated ECE values, while MCNN+ViTs-B/1(♦) stands out with the most favorable trade-off between ECE and robustness.

In the out-of-distribution (OOD) scenario, all models witness an approximately 20-point increase in ECE, emphasizing the need for further advancements.
in model calibration. Examining the calibration versus robustness trade-off, models with hierarchical attention mechanisms, such as Swin UNETR(♦) and MedNeXt(■), exhibit the best values, closely followed by UNETR, albeit with a higher error rate. Notably, the use of multiple encoders in MCNN+ViTs-B/1(+) significantly impacts calibration, revealing a notable gap with its counterpart CNN+ViTv-B/1(▲). Therefore, while MCNN+ViTs-B/1(+) generally performs well in terms of robustness, careful consideration is warranted regarding its confidence in predictions. Lastly, it is essential to note that pure CNN-based models like U-Net(●) exhibit greater calibration challenges under domain shift.

<figure><img src="image_14.png" /><figcaption>Figure 11: Expected VS error (↓) versus calibration error (↓). Labels A and B represent Prostate 158 - Rater A and Prostate 158 - Rater B, respectively. All models undergo evaluation through ensembling (5-fold). No calibration methods were employed.</figcaption></figure>

## Visual Segmentation and Uncertainty Mapping

To qualitatively evaluate the segmentation and uncertainty results, Figure 12 depicts visual segmentation and uncertainty maps outcomes. Each row corresponds to a different architecture type, with the first two columns representing the ID dataset and the remaining columns representing the OOD datasets. In terms of segmentation, all models exhibit under-segmentation errors compared to the ground truth mask (green line). For the PICAI dataset (first column), nnFormer performs the worst, as highlighted in Table 1. The visual evaluation of the other models reveals that it is difficult to distinguish a clear best performer by visual inspection alone, as their segmentation boundaries are closely aligned. This observation is consistent with the Dice similarity coefficients reported in
Table 1, which indicate marginal differences in segmentation performance among these models.

The evaluation of segmentation performance on the OOD dataset presents a different scenario. Although nnFormer (※) continues to perform the worst overall, its decline in performance is less pronounced compared to other models. This suggests that nnFormer (※) may retain some level of robustness when generalizing to OOD data. Regarding overlap with the ground truth, the hierarchical transformer-based model, Swin UNETR(◆) exhibits a closer delineation, demonstrating better alignment with the true tumor boundaries. Similar patterns are observed for MedNeXt(■) and U-Net(●), which also show reasonable performance in their overlap with the ground truth, although slight variations in the segmentation boundaries remain. Interestingly, annotations provided by raters A and B (third and fourth columns) differ for the same patients, highlighting inter-observer variability. Despite this, all models consistently localized only one of the annotated regions correctly, failing to identify the second region.

Given our focus on assessing epistemic uncertainty, we adopt a Bayesian approximation through Monte Carlo dropout, applying a dropout rate of 0.2 over 100 stochastic forward passes Taguelmimt et al. (2025). This approach allows us to quantify uncertainty by approximating the posterior distribution of the model’s weights, providing insight into the model’s confidence in its predictions for both ID and OOD datasets. The uncertainty results for the ID dataset are depicted in the second column of Figure 12. Higher uncertainty are observed in the tumor regions for the UNETR(☆) and CNN+ViTv-B/1(▲) models. In contrast, models such as MCNN+ViTs-B/1(◆) and MedNeXt(■) exhibit lower uncertainty, with MCNN+ViTs-B/1(◆) being the most confident. The U-Net(●) and nnFormer(※) models perform reasonably well, showing only a few pixels with high uncertainty in the tumor core. Lastly, Swin UNETR(◆) demonstrates confidence in its predictions within the tumor region, though less certainty is observed in the surrounding areas.

In the case of the OOD dataset (last column), we observe increased uncertainty across all models, with non-tumor regions also appearing in the uncertainty maps. For example, MCNN+ViTs-B/1(◆) shows reduced confidence in its prediction within the tumor core, and a new uncertain region appears in the upper part of the transitional zone. Similarly, CNN+ViTv-B/1(▲) displays decreased confidence, with additional uncertain regions surrounding the tumor. A similar pattern is observed in MedNeXt(■) and nnFormer(※), where the uncertainty regions extend beyond the tumor area. In contrast, UNETR(☆) is more confident in its tumor predictions, though new uncertainty regions emerge at the boundary between the transitional and peripheral zones of the prostate. Notably, Swin UNETR(◆) shows the highest overall uncertainty, with regions extending throughout the transitional and peripheral zones, although it remains confident within the tumor region. Lastly, we observe that, except for U-Net(●), all models exhibit uncertainty in the second tumor annotated by rater B. Although these models did not segment this region as a tumor, they demonstrate uncertainty regarding its classification. These results support and complement the findings from the PRR score versus Volume Similarity analysis (Figure 9).
<figure><img src="image_15.png" /><figcaption>Figure 12: Visual segmentation and uncertainty mapping results of transformer-based models for the ID and OOD datasets. Green contours indicate ground-truth tumors, while blue represents the prostate gland. Each row corresponds to a different transformer-based model.</figcaption></figure>

## 5. Conclusion

While numerous studies have explored novel model architectures, only a few have comprehensively examined their reliability. Many works have primarily focused on assessing robustness using volume, overlapping, or distance metrics. However, depending on the clinical application, a more thorough evaluation is crucial. Understanding error sources and model confidence is essential to enhance our understanding and ensure secure integration into clinical settings.
Our investigation delves into the reliability of recent segmentation models, focusing on robustness and uncertainty estimation during common domain shifts in medical imaging. Although no single model excels in all scenarios, recent models such as Swin UNETR(♦) and MedNeXt(■) demonstrate notably greater robustness to domain shifts compared to U-Net(●) baseline. This robustness also extends to enhanced calibration when contrasted with baselines experiencing significant degradation outside the domain. Notably, pure transformer models like nnFormer(※) perform sub-optimally in all scenarios, while models employing attention mechanisms—whether hierarchical, hybrid, or emulating attention using convolutional components—prove to be the most reliable.

However, no model remains unsusceptible to challenges posed by domain shifts. Thus, finding methods to enhance model calibration in out-of-distribution (OOD) settings is crucial. Our results indicate that recent and robust models tend to outperform U-Net(●) baseline (except for Swin UNETR(♦)) in misclassification under intense domain shifts. On the contrary, OOD detection Swin UNETR(♦) distinctly demonstrates the best trade-off under domain shifts between robustness and uncertainty. Despite positive strides in robustness, our findings underscore the necessity to improve the reliability of segmentation models for the safe deployment of medical computer-aided diagnosis (CAD) applications. In this context, we identify several promising directions that we hope will inspire future research on this critical topic.

It is important to clarify that our focus is not on achieving state-of-the-art performance through different loss, preprocessing, or post-processing strategies. Instead, we aim to understand the behavior of emerging network trends in both ID and OOD scenarios. Consequently, our results may not be directly comparable to those presented in the PICAI challenge, where diverse multi-tasking, multiple training, or other strategies, including self-supervised approaches, were employed.

Moreover, it is important to recognize that some of the current results may exhibit minor variability due to potential inaccuracies in the initial whole prostate segmentation step. While these segmentation errors likely affect all models uniformly, they emphasize the need for careful consideration of such factors when interpreting results. This attention is critical for the accurate evaluation and reliable deployment of medical computer-aided diagnosis (CAD) systems. Ensuring robustness and minimizing errors during both pre- and post-processing stages is essential for maintaining high standards of safety and efficacy in clinical practice.

Additionally, the generalizability of this work must be rigorously evaluated across a wider range of datasets, particularly in out-of-domain (OOD) scenarios, to fully assess its applicability and effectiveness. Since the current results are based on a single OOD dataset, and no public datasets were available for broader testing, the conclusions regarding the model's robustness are inherently limited. Future studies should include testing on a broader spectrum of OOD datasets, ideally incorporating publicly available datasets, to provide a more comprehensive understanding of the model's performance in varied clinical environments and its ability to generalize to unseen data.
References

Adams, L. C., Makowski, M. R., Engel, G., Rattunde, M., Busch, F., Asbach, P., Niehues, S. M., Vinayahalingam, S., van Ginneken, B., Litjens, G., & Bressem, K. K. (2022). Prostate158 - an expert-annotated 3t mri dataset and algorithm for prostate cancer detection. *Computers in Biology and Medicine*, 148, 105817. doi:https://doi.org/10.1016/j.compbiomed.2022.105817.

705

Aldoj, N., Biavati, F., Michallek, F. et al. (2020). Automatic prostate and prostate zones segmentation of magnetic resonance images using densenet-like u-net. *Scientific Reports*, 10, 14315. URL: https://doi.org/10.1038/s41598-020-71080-0. doi:10.1038/s41598-020-71080-0.

710

Andrade-Miranda, G., Jaouen, V., Tankyevych, O., Rest, C. C. L., Visvikis, D., & Conze, P.-H. (2023). Multi-modal medical transformers: A meta-analysis for medical image segmentation in oncology. *Computerized Medical Imaging and Graphics*, (p. 102308). doi:https://doi.org/10.1016/j.compmedimag.2023.102308.

715

Bashkanov, O., Rak, M., Meyer, A., Engelage, L., Lumiani, A., Muschter, R., & Hansen, C. (2023). Automatic detection of prostate cancer grades and chronic prostatitis in bi-parametric mri. *Computer Methods and Programs in Biomedicine*, 239, 107624. doi:https://doi.org/10.1016/j.cmpb.2023.107624.

720

Bhandary, S., Kuhn, D., Babaiee, Z., Fechter, T., Benndorf, M., Zamboglou, C., Grosu, A.-L., & Grosu, R. (2023). Investigation and benchmarking of u-nets on prostate segmentation tasks. *Computerized Medical Imaging and Graphics*, 107, 102241. URL: https://www.sciencedirect.com/science/article/pii/S0895611123000599. doi:https://doi.org/10.1016/j.compmedimag.2023.102241.

725

Bosma, J. S., Saha, A., Hosseinzadeh, M., Slootweg, I., de Rooij, M., & Huisman, H. (2023). Semisupervised learning with report-guided pseudo labels for deep learning-based prostate cancer detection using biparametric MRI. *Radiology: Artificial Intelligence*, 5.

730

Brancato, V., Di Costanzo, G., Basso, L., Tramontano, L., Puglia, M., Ragozzino, A., & Cavaliere, C. A. (2020). Assessment of DCE Utility for PCa Diagnosis Using PI-RADS v2.1: Effects on Diagnostic Accuracy and Reproducibility. *Diagnostics (Basel)*, 10, 164. doi:10.3390/diagnostics10030164.

735

Bray, F., Colombet, M., Aitken, J. F., Bardot, A., Eser, S., Galceran, J., Hagenimana, M., Matsuda, T., Mery, L., Piñeros, M., Soerjomataram, I., de Vries, E., Wiggins, C., Won, Y.-J., Znaor, A., & Ferlay, J. (Eds.) (2023). *Cancer Incidence in Five Continents, Vol. XII (IARC Cancer-Base No. 19)*. Lyon: International Agency for Research on Cancer. URL: https://ci5.iarc.who.int accessed [5/01/2024].

740
Chaves, H., Serra, M. M., Shalom, D. E., & et al. (2023). Assessing robustness and generalization of a deep neural network for brain MS lesion segmentation on real-world data. *European Radiology*, doi:10.1007/s00330-023-10093-5.

745

Conze, P.-H., Andrade-Miranda, G., Singh, V. K., Jaouen, V., & Visvikis, D. (2023). Current and emerging trends in medical image segmentation with deep learning. *IEEE Transactions on Radiation and Plasma Medical Sciences*, 7, 545-569. doi:10.1109/TRPMS.2023.3265863.

750

De Jorge, P., Volpi, R., Torr, P., & Gregory, R. (2023). Reliability in semantic segmentation: Are we on the right track? In *The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)*.

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., Gelly, S., Uszkoreit, J., & Houlsby, N. (2021). An image is worth 16x16 words: Transformers for image recognition at scale. In *International Conference on Learning Representations (ICLR)*.

755

Duran, A., Dussert, G., Rouvière, O., Jaouen, T., Jodoin, P.-M., & Lartizien, C. (2022). Prostattention-net: A deep attention model for prostate cancer segmentation by aggressiveness in MRI scans. *Medical Image Analysis*, 77, 102347. doi:https://doi.org/10.1016/j.media.2021.102347.

760

Elkhoury, F. F., Felker, E. R., Kwan, L., Sisk, A. E., Delfin, M., Natarajan, S., & Marks, L. S. (2019). Comparison of targeted vs systematic prostate biopsy in men who are biopsy naive: the prospective assessment of image registration in the diagnosis of prostate cancer (pairedcap) study. *JAMA Surgery*, 154, 811-818.

765

Guetari, R., Ayari, H., & Sakly, H. (2023). Computer-aided diagnosis systems: a comparative study of classical machine learning versus deep learning-based approaches. *Knowl Inf Syst*, 65, 3881-3921. doi:10.1007/s10115-023-01894-7.

770

Gueye, S., Zeigler-Johnson, C., Friebel, T., Spangler, E., Jalloh, M., MacBride, S., Malkowicz, S., & Rebbeck, T. (2003). Clinical characteristics of prostate cancer in african americans, american whites, and senegalese men. *Urology*, 61, 987-992.

775

Hatamizadeh, A., Nath, V., Tang, Y., Yang, D., Roth, H., & Xu, D. (2022a). Swin UNETR: Swin Transformers for semantic segmentation of brain tumors in MRI images. In *Brainlesion: Glioma, Multiple Sclerosis, Stroke and Traumatic Brain Injuries*.

Hatamizadeh, A., Tang, Y., Nath, V., Yang, D., Myronenko, A., Landman, B., Roth, H. R., & Xu, D. (2022b). UNETR: Transformers for 3D medical image segmentation. In *IEEE/CVF Winter Conference on Applications of Computer Vision* (pp. 272-284).

780
Hosseinzadeh, M., Saha, A., & Brand, P. e. a. (2022). Deep learning-assisted prostate cancer detection on bi-parametric mri: minimum training data size requirements and effect of prior knowledge. *Eur Radiol*, 32, 2224-2234. URL: https://doi.org/10.1007/s00330-021-08320-y. doi:10.1007/s00330-021-08320-y.

785

Hung, A. L. Y., Zheng, H., Miao, Q., Raman, S. S., Terzopoulos, D., & Sung, K. (2023). Cat-net: A cross-slice attention transformer model for prostate zonal segmentation in mri. *IEEE Transactions on Medical Imaging*, 42, 291-303. URL: https://doi.org/10.1109/TMI.2022.3211764. doi:10.1109/TMI.2022.3211764. arXiv:36194719.

790

Isensee, F., Jaeger, P. F., Kohl, S. A. A., Petersen, J., & Maier-Hein, K. H. (2020). nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. *Nature Methods*, 18, 203-211.

795

Isensee, F., Jäger, P. F., Kohl, S. A., Petersen, J., & Maier-Hein, K. H. (2019). Automated design of deep learning methods for biomedical image segmentation. arXiv:1904.08128.

800

James, N. D., Tannock, I., N'Dow, J., Feng, F., Gillessen, S., Ali, S. A., Trujillo, B., Al-Lazikani, B., Attard, G., Bray, F., Compérat, E., Eeles, R., Fatiregun, O., Grist, E., Halabi, S., Haran, A., Herchenhorn, D., Hofman, M. S., Jalloh, M., Loeb, S., MacNair, A., Mahal, B., Mendes, L., Moghul, M., Moore, C., Morgans, A., Morris, M., Murphy, D., Murthy, V., Nguyen, P. L., Padhani, A., Parker, C., Rush, H., Sculpher, M., Soule, H., Sydes, M. R., Tilki, D., Tunariu, N., Villanti, P., & Xie, L.-P. (2024). The lancet commission on prostate cancer: planning for the surge in cases. *Lancet*, 403, 1683-1722. doi:10.1016/S0140-6736(24)00651-2. Epub 2024 Apr 4.

805

Jiang, X., Hu, Z., Wang, S., & Zhang, Y. (2023). Deep learning for medical image-based cancer diagnosis. *Cancers (Basel)*, 15, 3608. doi:10.3390/cancers15143608.

810

Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., Dollár, P., & Girshick, R. (2023). Segment anything. arXiv:2304.02643.

815

Li, B., Oka, R., Xuan, P., Yoshimura, Y., & Nakaguchi, T. (2022a). Robust multi-modal prostate cancer classification via feature autoencoder and dual attention. *Informatics in Medicine Unlocked*, 30, 100923. doi:https://doi.org/10.1016/j.imu.2022.100923.

820

Li, J., Chen, J., Tang, Y., Landman, B. A., & Zhou, S. K. (2022b). Transforming medical imaging with transformers? a comparative review of key properties, current progresses, and future perspectives. arXiv:2206.01136.

Li, Y., Huang, M., Zhang, Y., Feng, S., Chen, J., & Bai, Z. (2023). A dual attention-guided 3d convolution network for automatic segmentation of
825

prostate and tumor. *Biomedical Signal Processing and Control*, 85, 104755.
doi:https://doi.org/10.1016/j.bspc.2023.104755.

Lin, T.-Y., Goyal, P., Girshick, R., He, K., & Dollár, P. (2020). Focal loss for dense object detection. *IEEE Transactions on Pattern Analysis and Machine Intelligence*, 42, 318-327.

830

Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021). Swin Transformer: Hierarchical vision Transformer using shifted windows. In *International Conference on Computer Vision (ICCV)* (pp. 10012-10022).

Liu, Z., Mao, H., Wu, C.-Y., Feichtenhofer, C., Darrell, T., & Xie, S. (2022). A ConvNet for the 2020s. In *Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)*. doi:10.48550/arxiv.2201.03545.

835

Malinin, A., Mlodozieiec, B., & Gales, M. (2020). Ensemble distribution distil-
lation. In *International Conference on Learning Representations*.

840

Mazurowski, M. A., Dong, H., Gu, H., Yang, J., Konz, N., & Zhang, Y. (2023). Segment anything model for medical image analysis: An experimental study. *Medical Image Analysis*, 89, 102918.

Mehralivand, S., Harmon, S. A., Shih, J. H., Smith, C. P., Lay, N., Argun, B., Bednarova, S., Baroni, R. H., Canda, A. E., Ercan, K. et al. (2020). Multicenter multireader evaluation of an artificial intelligence-based attention mapping system for the detection of prostate cancer with multiparametric mri. *American Journal of Roentgenology*, 215, 903-912.

845

Morote, J., Borque-Fernando, A., Triquell, M., Abascal, J. M., Servian, P., Planas, J., Mendez, O., Esteban, L. M., & Tilla, E. (2023). A risk-organised model for clinically significant prostate cancer early detection. *BJUI Com-pass*, 4, 420-422. doi:10.1002/bco2.230.

850

Ostmeier, S., Axelrod, B., Isensee, F., Bertels, J., Mlynash, M., Christensen, S., Lansberg, M. G., Albers, G. W., Sheth, R., Verhaaren, B. F., Mahammedi, A., Li, L.-J., Zaharchuk, G., & Heit, J. J. (2023). Use-evaluator: Performance metrics for medical image segmentation models supervised by uncertain, small or empty reference annotations in neuroimaging. *Medical Image Analysis*, 90, 102927. doi:https://doi.org/10.1016/j.media.2023.102927.

855

Pellicer-Valero, O., Marenco Jiménez, J., & Gonzalez-Perez, V. e. a. (2022). Deep learning for fully automatic detection, segmentation, and gleason grade estimation of prostate cancer in multiparametric magnetic resonance images. *Sci Rep*, 12, 2975. doi:10.1038/s41598-022-06730-6.

860

Raman, A. G., Sarma, K. V., Raman, S. S., Priester, A. M., Mirak, S. A., Riskin-
Jones, H. H., Dhinagar, N., Speier, W., Felker, E., Sisk, A. E. et al. (2021).
Optimizing spatial biopsy sampling for the detection of prostate cancer. *The
Journal of urology*, 206, 595-603.
Ravi, N., Gabeur, V., Hu, Y.-T., Hu, R., Ryali, C., Ma, T., Khedr, H., Rädle, R., Rolland, C., Gustafson, L., Mintun, E., Pan, J., Alwala, K. V., Carion, N., Wu, C.-Y., Girshick, R., Dollár, P., & Feichtenhofer, C. (2024). Sam 2: Segment anything in images and videos. URL: https://arxiv.org/abs/2408.00714. arXiv:2408.00714.

865

Ren, C., Guo, Z., Ren, H., Jeong, D., Kim, D., Zhang, S., Wang, J., & Zhang, G. (2023). Prostate segmentation in MRI using transformer encoder and decoder framework. *IEEE Access, 11*, 101630–101643. doi:10.1109/ACCESS. 2023.3313420.

870

Ricci Lara, M. A., Mosquera, C., Ferrante, E., & Echeveste, R. (2023). Towards unraveling calibration biases in medical image analysis. In S. Wesarg, E. Puyol Antón, J. S. H. Baxter, M. Erdt, K. Drechsler, C. Oyarzun Laura, M. Freiman, Y. Chen, I. Rekik, R. Eagleson, A. Feragen, A. P. King, V. Cheplygina, M. Ganz-Benjaminsen, E. Ferrante, B. Glocker, D. Moyer, & E. Petersen (Eds.), *Clinical Image-Based Procedures, Fairness of AI in Medical Imaging, and Ethical and Philosophical Issues in Medical Imaging* (pp. 132–141).

875

880

Rosenkrantz, A. B., Ginocchio, L. A., Cornfeld, D., Froemming, A. T., Gupta, R. T., Turkbey, B., Westphalen, A. C., Babb, J. S., & Margolis, D. J. (2016). Interobserver reproducibility of the pi-rads version 2 lexicon: A multicenter study of six experienced prostate radiologists. *Radiology, 280*, 793–804. doi:10.1148/radiol.2016152542. PMID: 27035179.

885

Roy, S., Koehler, G., Ulrich, C., Baumgartner, M., Petersen, J., Isensee, F., Jaeger, P. F., & Maier-Hein, K. H. (2023). Mednext: Transformer-driven scaling of convnets for medical image segmentation. In *International Conference on Medical Image Computing and Computer-Assisted Intervention*. URL: https://api.semanticscholar.org/CorpusID:257623026.

890

Saha, A., Bosma, J., Linmans, J., Hosseinzadeh, M., & Huisman, H. J. (2021). Anatomical and diagnostic bayesian segmentation in prostate MRI - should different clinical objectives mandate different loss functions? *CoRR*, abs/2110.12889.

895

Saha, A., Bosma, J., Twilt, J., Van Ginneken, B., Yakar, D., Elschot, M., Veltman, J., Fütterer, J., de Rooij, M., & henkjan huisman (2023). Artificial intelligence and radiologists at prostate cancer detection in MRI — the PI-CAI challenge. In *Medical Imaging with Deep Learning, short paper track*.

900

Saha, A., Bosma, J. S., Twilt, J. J., van Ginneken, B., Bjartell, A., Padhani, A. R., Bonekamp, D., Villeirs, G., Salomon, G., Giannarini, G., Kalpathy-Cramer, J., Barentsz, J., Maier-Hein, K. H., Rusu, M., Rouvière, O., van den Bergh, R., Panebianco, V., Kasivisvanathan, V., Obuchowski, N. A., Yakar, D., Elschot, M., Veltman, J., Fütterer, J. J., de Rooij, M., Huisman, H., Saha, A., Bosma, J. S., Twilt, J. J., van Ginneken, B., Noordman, C. R.,
Slootweg, I., Roest, C., Fransen, S. J., Sunoqrot, M. R., Bathen, T. F., Rouw, D., Immerzeel, J., Geerdink, J., van Run, C., Groeneveld, M., Meakin, J., Karagöz, A., Bône, A., Routier, A., Marcoux, A., Abi-Nader, C., Li, C. X., Feng, D., Alis, D., Karaarslan, E., Ahn, E., Nicolas, F., Sonn, G. A., Bhat-tacharya, I., Kim, J., Shi, J., Jahanandish, H., An, H., Kan, H., Oksuz, I., Qiao, L., Rohé, M.-M., Yergin, M., Khadra, M., Şeker, M. E., Kartal, M. S., Debs, N., Fan, R. E., Saunders, S., Soerensen, S. J., Moroianu, S., Vesal, S., Yuan, Y., Malakoti-Fard, A., Mačiūnien, A., Kawashima, A., de M.G. de Sousa Machadov, A. M., Moreira, A. S. L., Ponsiglione, A., Rappaport, A., Stanzione, A., Ciuvasovas, A., Turkbey, B., de Keyzer, B., Pedersen, B. G., Eijlers, B., Chen, C., Riccardo, C., Alis, D., Courrech Staal, E. F., Jäderling, F., Langkilde, F., Aringhieri, G., Brembilla, G., Son, H., Vanderlelij, H., Raat, H. P., Pikūnienė, I., Macova, I. et al. (2024). Artificial intelligence and radiologists in prostate cancer detection on mri (pi-cai): an international, paired, non-inferiority, confirmatory study. *The Lancet Oncology*, 25, 879–887. doi:https://doi.org/10.1016/S1470-2045(24)00220-1.

910

915

920

Saha, A., Twilt, J. J., Bosma, J. S., van Ginneken, B., Yakar, D., Elschot, M., Veltman, J., Fütterer, J., & Maarten de Rooij, H. H. (2022). Artificial Intelligence and Radiologists at Prostate Cancer Detection in MRI: The PI-CAI Challenge (Study Protocol). doi:10.5281/zenodo.6667655.

925

Shamshad, F., Khan, S., Zamir, S. W., Khan, M. H., Hayat, M., Khan, F. S., & Fu, H. (2022). Transformers in medical imaging: A survey. arXiv:2201.09873.

930

Song, E., Long, J., Ma, G., Liu, H., Hung, C.-C., Jin, R., Wang, P., & Wang, W. (2023). Prostate lesion segmentation based on a 3D end-to-end convolution neural network with deep multi-scale attention. *Magnetic Resonance Imaging*, 99, 98–109. doi:https://doi.org/10.1016/j.mri.2023.01.015.

935

Sun, Z., Wang, K., & Kong, Z. e. a. (2023). A multicenter study of artificial intelligence-aided software for detecting visible clinically significant prostate cancer on mpmri. *Insights Imaging*, 14, 72. doi:10.1186/s13244-023-01421-w.

940

Sung, H., Ferlay, J., Siegel, R. L., Laversanne, M., Soerjomataram, I., Jemal, A., & Bray, F. (2021). Global cancer statistics 2020: Globocan estimates of incidence and mortality worldwide for 36 cancers in 185 countries. *CA: A Cancer Journal for Clinicians*, 71, 209–249.

Sunoqrot, M. R., Saha, A., Hosseinzadeh, M., Elschot, M., & Huisman, H. (2022). Artificial intelligence for prostate mri: open datasets, available applications, and grand challenges. *European Radiology Experimental*, 6, 35. doi:10.1186/s41747-022-00288-8.

945

Taguelmimt, K., Dang, H.-P., Andrade-Miranda, G., Visvkis, D., Malavaud, B., & Bert, J. (2025). Uncertainty-aware deep learning classification for mri-based prostate cancer detection. In S. Ali, F. van der Sommen, B. W. Papież,
N. Ghatwary, Y. Jin, & I. Kolenbrander (Eds.), MICCAI 2024 3rd International Workshop on Cancer Prevention, detection, and intervention. Accepted on July 15 (pp. 114–123). Cham: Springer Nature Switzerland.

Van Ginneken, B., Armato, S. G., de Hoop, B., Van Amelsvoort-van de Vorst, S., Duindam, T., Niemeijer, M., Murphy, K., Schilham, A., Retico, A., Fantacci, M. E., Camarlinghi, N., Bagagli, F., Gori, I., Hara, T., Fujita, H., Gargano, G., Bellotti, R., Tangaro, S., Bolaños, L., Carlo, F. D., Cerello, P., Cristian Cheran, S., Lopez Torres, E., & Prokop, M. (2010). Comparing and combining algorithms for computer-aided detection of pulmonary nodules in computed tomography scans: The anode09 study. *Medical Image Analysis*, 14, 707–722. doi:https://doi.org/10.1016/j.media.2010.05.005.

Wang, Z., Wu, R., Xu, Y., Liu, Y., Chai, R., & Ma, H. (2023). A two-stage CNN method for MRI image segmentation of prostate with lesion. *Biomedical Signal Processing and Control*, 82, 104610. doi:https://doi.org/10.1016/j.bspc.2023.104610.

Westphalen, A. C., McCulloch, C. E., Anaokar, J. M., Arora, S., Barashi, N. S., Barentsz, J. O., Bathala, T. K., Bittencourt, L. K., Booker, M. T., Braxton, V. G., Carroll, P. R., Casalino, D. D., Chang, S. D., Coakley, F. V., Dhatt, R., Eberhardt, S. C., Foster, B. R., Froemming, A. T., Fütterer, J. J., Ganeshan, D. M., Gertner, M. R., Mankowski Gettle, L., Ghai, S., Gupta, R. T., Hahn, M. E., Houshyar, R., Kim, C., Kim, C. K., Lall, C., Margolis, D. J. A., McRae, S. E., Oto, A., Parsons, R. B., Patel, N. U., Pinto, P. A., Polascik, T. J., Spilseth, B., Starcevich, J. B., Tammisetti, V. S., Taneja, S. S., Turkbey, B., Verma, S., Ward, J. F., Warlick, C. A., Weinberger, A. R., Yu, J., Zagoria, R. J., & Rosenkrantz, A. B. (2020). Variability of the positive predictive value of pi-rads for prostate mri across 26 centers: Experience of the society of abdominal radiology prostate cancer disease-focused panel. *Radiology*, 296, 76–84. doi:10.1148/radiol.2020190646. PMID: 32315265.

Wiesenfarth, M., Reinke, A., Landman, B. A., Eisenmann, M., Saiz, L. A., Cardoso, M. J., Maier-Hein, L., & Kopp-Schneider, A. (2021). Methods and open-source toolkit for analyzing and visualizing challenge results. *Scientific reports*, 11, 1–15.

Xiao, H., Li, L., Liu, Q., Zhu, X., & Zhang, Q. (2023). Transformers in medical image segmentation: A review. *Biomedical Signal Processing and Control*, 84, 104791. doi:https://doi.org/10.1016/j.bspc.2023.104791.

Yoo, S., Gujrathi, I., & Haider, M. e. a. (2019). Prostate cancer detection using deep convolutional neural networks. *Sci Rep*, 9, 19518. doi:10.1038/s41598-019-55972-4.

Zhou, H., Guo, J., Zhang, Y., Yu, L., Wang, L., & Yu, Y. (2021). nnFormer: Interleaved transformer for volumetric segmentation. *CoRR*, abs/2109.03201.
CRediT author statement

**Gustavo Andrade-Miranda:** conceptualization, methodology, software, validation, formal analysis, writing - original draft, visualization. **Pedro Soto Vega:** methodology, writing - original draft, formal analysis, reviewing, and editing. **Kamilia Taguelmimt:** data curation, reviewing, and editing. **Hong-Phuong Dang:** reviewing and editing. **Dimitris Visvikis:** Supervision, project administration. **Julien Bert:** conceptualization, methodology, writing - reviewing and editing, supervision, funding acquisition.

990

995 Acknowledgements

This work was partly supported by the French State under the Future Investments Program through the DIANA project (PSPC-DIANA-2021 BPI) and by France Life Imaging (grant ANR-11-INBS-0006).