# PI-RADS inspired architecture: A new deep learning approach to detect clinically significant prostate cancer on MRI

Kamilia Taguelmimt, Hong-Phuong Dang, Gustavo Andrade-Miranda, Dimitris Visvikis, Bernard Malavaud, Julien Bert

<figure><img src="image_1.png" /></figure>

## To cite this version:

Kamilia Taguelmimt, Hong-Phuong Dang, Gustavo Andrade-Miranda, Dimitris Visvikis, Bernard Malavaud, et al.. PI-RADS inspired architecture: A new deep learning approach to detect clinically significant prostate cancer on MRI. IEEE International Symposium on Biomedical Imaging, Apr 2025, Houston, United States. 10.1109/ISBI60581.2025.10980749. hal-04934805

HAL Id: hal-04934805

https://imt-atlantique.hal.science/hal-04934805v1

Submitted on 2 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# PI-RADS INSPIRED ARCHITECTURE: A NEW DEEP LEARNING APPROACH TO DETECT CLINICALLY SIGNIFICANT PROSTATE CANCER ON MRI

Kamilia Taguelmimt¹

Hong-Phuong Dang¹,²

Gustavo Andrade-Miranda¹

Dimitris Visvikis¹

Bernard Malavaud³

Julien Bert¹

¹ LaTIM, UMR1101, INSERM, University of Brest, France

² CentraleSupélec, IETR UMR CNRS 6164, Cesson-Sévigné, France

³ Toulouse Oncopole University Cancer Institute, France

## ABSTRACT

This work presents PI-RADS-Net, a deep-learning model inspired by the PI-RADS protocol to detect clinically significant prostate cancers (csPCa) on MRI by emulating radiologists' decision-making. The model employs a two-branch architecture, using T2-weighted images for the transition zone and a combination of diffusion-weighted imaging (DWI) and apparent diffusion coefficient (ADC) sequences for the peripheral zone. An initial extension, PI-RADS-BimodalNet, which incorporates clinical parameters, was also developed. Experimental results demonstrate that PI-RADS-Net significantly improves classification accuracy, while PI-RADS-BimodalNet achieves an area under the curve (AUC) of 0.83 in testing, surpassing other models. Our approach holds promise for improving the diagnosis of clinically significant prostate cancer, reducing unnecessary biopsies, and enhancing patient outcomes.

**Index Terms**— Prostate Cancer, Deep Learning, MRI Classification, PI-RADS, Bimodal.

## 1 Introduction

Prostate cancer is the most common cancer in 112 countries, accounting for 15% of all cancer cases in men. The number of diagnoses is expected to increase from 1.4 million in 2020 to 2.9 million by 2040 [1]. While systematic screening for prostate cancer is not recommended, it may be considered for individuals at high risk or upon patient request. Prostate-specific antigen (PSA) testing and magnetic resonance imaging (MRI) are the main diagnostic tools, providing better accuracy when used together.

PSA is a prostate-specific biomarker, but elevated levels (above 3 ng/ml) can also indicate benign conditions, reducing its reliability for cancer screening. This highlights the growing role of MRI in detection. Multiparametric MRI (mpMRI) is currently considered the imaging modality of choice for the diagnosis and evaluation of localized prostate cancer. This examination utilizes several sequences, including T2-weighted (T2W), apparent diffusion coefficient (ADC), diffusion-weighted imaging (DWI) and sequences following Gadolinium injection, to assess the prostate gland and detect potentially suspicious lesions, known as target lesions [2]. These abnormalities are then classified using the Prostate Imaging Reporting and Data System (PI-RADS) [3],

which assigns a score from 1 (low probability) to 5 (high probability) to indicate the likelihood of a lesion being cancerous.

The determination of a PI-RADS category for each lesion is based on evaluating T2W and ADC/DWI sequences according to the prostate's zonal anatomy. Accurately distinguishing between the prostate zones is essential for proper interpretation: the peripheral zone (PZ), where roughly 75% of prostate cancers are found, is mainly assessed using diffusion sequences (ADC/DWI), whereas the transition zone (TZ), which encircles the prostatic urethra, is primarily evaluated with T2W imaging [4]. While MRI is an effective screening tool, manual analysis is time-consuming and prone to errors, highlighting the need for machine-learning-based CAD tools [5]. Additionally, many methods neglect the clinical reasoning process, focusing only on input-output transformations without reflecting how clinicians make decisions.

This study presents an innovative approach using a 3D convolutional neural network (CNN) to detect csPCa, relying on the PI-RADS system as a reference. We analyze the MRI data in two distinct branches: one for the TZ, using T2W sequences, and the other for the PZ, using ADC and DWI sequences. Additionally, we integrate clinical data such as the PSA level, patient age, and prostate volume using the Tab-Net model [6], thereby refining the final classification. This approach aims to improve diagnostic accuracy while minimizing risks associated with unnecessary invasive treatments.

## 2 Related work

Machine learning techniques have proven to be promising in the diagnosis of prostate cancer. Methods such as support vector machines, logistic regression, and random forests have shown to be good predictors of prostate cancer using MRIs, as well as clinical parameters such as PSA, despite its low specificity. It is important to emphasize that these parameters should not be neglected in the detection process. A retrospective study has shown that these techniques can predict prostate cancer with an area under the curve (AUC) of 0.918 [7].

Notably, artificial neural networks (ANN) have improved the interpretation of PSA levels and the accuracy of risk predictions [8]. By combining several factors, including PSA, age, prostate volume, and results from digital rectal exams, ANN have achieved good predictive performance in biopsy
<figure><img src="image_3.png" /><figcaption>**Fig. 1.** PI-RADS-Net and PI-RADS architectures feature a two-branch design: one branch processes T2W sequences focusing on the TZ, while the other handles ADC/DWI sequences focusing on the PZ. Both branches employ convolutional blocks with SE modules and residual connections. Clinical data are integrated with the combined features and passed to a binary classifier for csPCa detection.</figcaption></figure>

outcomes [9]. Moreover, it has been demonstrated that ANN can reduce unnecessary biopsies by up to 17%, which illustrates their effectiveness in the assessment of prostate cancer [10].

While ANNs are effective for certain tasks, deep learning excels in modeling complex, large datasets and has shown promising results in this domain. Several architectures using 3D MRI data and CNNs have reported AUCs between 0.81 and 0.87 [11, 12]. Zone-based segmentation and direct classification of prostate cancer aggressiveness have also emerged, with fully automated systems achieving AUCs above 0.85 [13]. The integration of zonal information has been shown to significantly improve detection performance [14].

# 3 Methodology

Inspired by the PI-RADS protocol, our approach emphasizes the integration of different MRI sequences as well as clinical data to enhance diagnostic accuracy. We introduce a novel model, **PI-RADS-Net**, that leverages the features of MRI images, and its extension **PI-RADS-BimodalNet**, which incorporates clinical data. Fig. 1 illustrates the architecture of these two models. We evaluate their performance against benchmark deep architectures, assessing their effectiveness in diagnostic tasks. The details of these architectures are provided below.

## 3.1 PI-RADS-Net

This two-branch model consists of two distinct branches: one processing T2W sequences focused on TZ, and the other processing ADC and DWI sequences targeting the PZ. In each branch, the corresponding segmentation mask is combined with the imaging sequences into a single multichannel input. Each branch utilizes four-stage convolutional blocks with

Squeeze and Excitation (SE) [15] modules to enhance feature extraction. Finally, both features were combined and passed to a fully connected layer to output probability.

## 3.2 PI-RADS-BimodalNet

An extension of PI-RADS-Net that integrates clinical parameters, including PSA levels, prostate volume, and patient age, using the TabNet framework. PI-RADS-BimodalNet concatenates features from the MRI branches with the clinical data, enhancing its ability to leverage patient-specific information and improve the final classification process.

## 3.3 Baselines

### 3.3.1 Unified-BranchNet and Bimodal-Unified-BranchNet

Unified-BranchNet uses a single processing stream that combines T2W, DWI, and ADC sequences along with a complete prostate mask as input. This model also applies four-stage convolutional blocks with SE modules while merging the data at the input stage to simplify the architecture while maintaining high accuracy. And Bimodal-Unified-BranchNet extends this model by incorporating clinical parameters.

### 3.3.2 Deep Architecture Models

Here, we also use the DenseNet121 and ResNet50 models to compare lesion classification using all MRI sequences with prostate masks as input. These deep architectures are chosen for their ability to extract complex features.

## 3.4 Data Description

For this study, we utilized the PICAI (Prostate Imaging Cancer AI) database [16]. The dataset comprises 1500 cases from 1476 patients across 11 centers, utilizing seven different MRI
<table><caption>**Table 1.** Performance comparison of various models on training and test sets for the best fold, with average test set metrics across folds. Metrics include AUC, sensitivity, specificity, and accuracy with 95% confidence intervals (CI).</caption><thead><tr><th>Model</th><th>Metric</th><th>Train (CI)</th><th>Test (CI)</th><th>Average (Test)</th></tr></thead><tbody><tr><td rowspan="4">Bimodal-Unified-BranchNet</td><td>AUC</td><td>0.82 (0.72, 0.91)</td><td>0.80 (0.72, 0.88)</td><td>0.79</td></tr><tr><td>Sensitivity</td><td>0.81 (0.64, 0.91)</td><td>0.84 (0.71, 0.92)</td><td>0.77</td></tr><tr><td>Specificity</td><td>0.74 (0.67, 0.80)</td><td>0.67 (0.60, 0.72)</td><td>0.66</td></tr><tr><td>Accuracy</td><td>0.75 (0.69, 0.80)</td><td>0.69 (0.64, 0.75)</td><td>0.68</td></tr><tr><td rowspan="4">PI-RADS-BimodalNet</td><td>AUC</td><td>0.90 (0.82, 0.98)</td><td>0.83 (0.75, 0.91)</td><td><strong>0.83</strong></td></tr><tr><td>Sensitivity</td><td>0.93 (0.77, 0.98)</td><td>0.73 (0.58, 0.84)</td><td>0.74</td></tr><tr><td>Specificity</td><td>0.78 (0.72, 0.84)</td><td>0.72 (0.66, 0.78)</td><td>0.73</td></tr><tr><td>Accuracy</td><td>0.80 (0.74, 0.85)</td><td>0.72 (0.66, 0.77)</td><td>0.73</td></tr><tr><td rowspan="4">Unified-BranchNet</td><td>AUC</td><td>0.84 (0.76, 0.92)</td><td>0.80 (0.72, 0.88)</td><td>0.77</td></tr><tr><td>Sensitivity</td><td>0.85 (0.72, 0.93)</td><td>0.75 (0.61, 0.85)</td><td>0.68</td></tr><tr><td>Specificity</td><td>0.73 (0.66, 0.80)</td><td>0.73 (0.67, 0.79)</td><td>0.71</td></tr><tr><td>Accuracy</td><td>0.76 (0.70, 0.81)</td><td>0.73 (0.68, 0.78)</td><td>0.71</td></tr><tr><td rowspan="4">PI-RADS-Net</td><td>AUC</td><td>0.82 (0.72, 0.91)</td><td>0.82 (0.75, 0.90)</td><td><strong>0.80</strong></td></tr><tr><td>Sensitivity</td><td>0.65 (0.47, 0.79)</td><td>0.68 (0.53, 0.80)</td><td>0.67</td></tr><tr><td>Specificity</td><td>0.81 (0.75, 0.86)</td><td>0.84 (0.78, 0.88)</td><td>0.75</td></tr><tr><td>Accuracy</td><td>0.79 (0.73, 0.84)</td><td>0.81 (0.76, 0.85)</td><td>0.74</td></tr><tr><td rowspan="4">DenseNet121</td><td>AUC</td><td>0.86 (0.78, 0.93)</td><td>0.80 (0.71, 0.88)</td><td>0.77</td></tr><tr><td>Sensitivity</td><td>0.76 (0.61, 0.87)</td><td>0.61 (0.47, 0.74)</td><td>0.68</td></tr><tr><td>Specificity</td><td>0.82 (0.76, 0.87)</td><td>0.77 (0.71, 0.82)</td><td>0.72</td></tr><tr><td>Accuracy</td><td>0.81 (0.75, 0.86)</td><td>0.74 (0.68, 0.79)</td><td>0.72</td></tr><tr><td rowspan="4">ResNet50</td><td>AUC</td><td>0.81 (0.71, 0.90)</td><td>0.80 (0.72, 0.88)</td><td>0.78</td></tr><tr><td>Sensitivity</td><td>0.81 (0.64, 0.91)</td><td>0.77 (0.63, 0.87)</td><td>0.67</td></tr><tr><td>Specificity</td><td>0.69 (0.62, 0.76)</td><td>0.71 (0.64, 0.76)</td><td>0.74</td></tr><tr><td>Accuracy</td><td>0.71 (0.64, 0.77)</td><td>0.72 (0.66, 0.77)</td><td>0.73</td></tr></tbody></table>

scanner models. Of these cases, 1075 are benign or represent indolent cancers, while 425 cases feature clinically significant prostate cancer (csPCa) with an ISUP score ≥ 2.

The median patient age is 66 years (IQR: 61–70 years), with a median PSA level of 8.5 ng/mL (IQR: 6–13 ng/mL) and a median prostate volume of 57 mL (IQR: 40–80 mL). Out of 1295 cases annotated by human experts, 220 contain csPCa lesions, accounting for approximately 17% of the annotations. These annotations were meticulously performed using ITK-SNAP v3.80 by a team supervised by expert radiologists.

The dataset includes segmentation masks of the prostate gland, generated using a deep learning approach combining five nnU-Net models [17] with 5-fold cross-validation, achieving a Dice similarity coefficient of 0.8968 ± 0.0547. MRI images were preprocessed with steps such as alignment, cropping, resampling to 144 × 144 × 32 voxels with voxel spacing of (0.5, 0.5, 0.3) mm, and Min-Max normalization. T2W, ADC, and DWI sequences, along with segmentation masks, were integrated into a single multichannel image.

## 3.5 Training and Evaluation

The data was divided into 80% for training and validation and 20% for testing, employing 5-fold cross-validation to ensure robust model evaluation. Focal loss was used as the loss function during training to address class imbalance and enhance the model's ability to detect clinically significant cancers. Performance metrics included AUC, sensitivity, specificity, and accuracy.

<figure><img src="image_4.png" /><figcaption>**Fig. 2.** Distribution of AUC values for all models across folds.</figcaption></figure>

# 4 Results and discussion

Fig.2 shows the distributions of AUC values across all folds for the evaluated models: Bimodal-Unified-BranchNet, PI-RADS-BimodalNet, Unified-BranchNet, PI-RADS-Net, DenseNet121, and ResNet50. The performance trends observed in this figure are reinforced by the results presented in Table 1, highlighting the effectiveness of models leveraging the PI-RADS protocol for prostate cancer classification. The PI-RADS-Net model achieves an average AUC of 0.80, surpassing the single-branch Unified-BranchNet model, which processes all MRI sequences simultaneously, with an AUC of
0.77 an improvement of 3.9%. Other deep learning models such as DenseNet121 and ResNet50, which take all MRI sequences as input, show average AUC values below 0.80. This shows that the optimal use of specific information from T2W, ADC and DWI MRI sequences, as directed by the PI-RADS protocol, enhances the efficacy of feature extraction for diagnosis.

In addition, the incorporation of clinical data into the deep learning model improved its performance. We observe that the PI-RADS-BimodalNet model obtained an AUC value of 0.83, i.e. an increase in AUC of 3.75% compared with PI-RADS-Net. Similarly, the Bimodal-Unified-BranchNet model, which also incorporates clinical data, shows relatively stable performance with an AUC value of 0.79, outperforming the image-only Unified-BranchNet model with an AUC value of 0.77.

Table 1 also reveals that the bimodal approach yields a notable 10.4 increase in average sensitivity (the ability to correctly identify positive cases), from 0.67 to 0.74, in the two PI-RADS models. This improvement in sensitivity is accompanied by a slight decrease in specificity (the ability to correctly identify negative cases), from 0.75 to 0.73. However, this trade-off remains acceptable in a clinical context where comprehensive detection of potentially malignant lesions is crucial.

# 5 Conclusion

This study compared several deep learning models for the classification of prostate lesions using MRI images. The results show that the PI-RADS-Net model, which integrates different PI-RADS MRI sequences (T2W, ADC, and DWI), already improves classification performance. Building on this, the PI-RADS-BimodalNet model extends PI-RADS-Net by incorporating clinical data, achieving further gains in both performance and stability.

These findings highlight the importance of integrating multi-sequence and clinical data to improve the diagnostic accuracy of prostate cancer. Bimodal models, in particular, demonstrated superior ability to distinguish malignant from benign lesions, making them especially promising for clinical use. Future work, building upon the PI-RADS-BimodalNet model, could explore further improvements by incorporating additional clinical parameters and optimizing attention mechanisms to enhance the robustness and effectiveness of these models and testing on external datasets. Additionally, it will be crucial to account for uncertainty in diagnostic decisions, an aspect that warrants further investigation in future studies.

# 6 Acknowledgments

This work was partly supported by the French State under the Future Investments Program through the DIANA project (PSPC-DIANA-2021 BPI) and by France Life Imaging (grant ANR-11-INBS-0006).

# 7 References

[1] N.D. James et al., “The lancet commission on prostate cancer: planning for the surge in cases,” *The Lancet*, vol. 403, no. 10437, pp. 1683–1722, 2024.

[2] J.J. Fütterer et al., “Can clinically significant prostate cancer be detected with multiparametric magnetic resonance imaging? A systematic review of the literature,” *Eur. Urol*, vol. 68, no. 6, pp. 1045–1053, 2015.

[3] Baris Turkbey et al., “Prostate imaging reporting and data system version 2.1: 2019 update of prostate imaging reporting and data system version 2,” *Eur. Urol.*, vol. 76, no. 3, pp. 340–2019.

[4] Anna Żurowska, Rafal Peksa, Michał Bieńkowski, Katarzyna Skrobisz, Marek Sowa, Marcin Matuszewski, Wojciech Biernat, and Edyta Szurowska, “Prostate cancer and its mimics—a pictorial review,” *Cancers*, vol. 15, pp. 3682, 07 2023.

[5] R. R. Wildeboer, R. J. van Sloun, H. Wijkstra, and M. Mischi, “Artificial intelligence in multiparametric prostate cancer imaging with focus on deep-learning methods,” *Comput. Methods Programs Biomed.*, vol. 189, pp. 105316, 2020.

[6] Sercan Ö Arik and Tomas Pfister, “Tabnet: Attentive interpretable tabular learning,” in *Proc. AAAI Conf Artif. Inte.*, 2021.

[7] S. Chen, T. Jian, C. Chi, et al., “Machine learning-based models enhance the prediction of prostate cancer,” *Front. Oncol.*, vol. 12, pp. 941349, 2022.

[8] B. Djavan, M. Remzi, A. Zlotta, C. Seitz, P. Snow, and M. Marberger, “Novel artificial neural network for early detection of prostate cancer,” *J. Clin. Oncol.*, vol. 20, no. 4, pp. 921–929, 2002.

[9] Jason C. Cai et al., “Fully automated deep learning model to detect clinically significant prostate cancer at mri,” *Radiology*, vol. 312, no. 2, 2024.

[10] Carsten Stephan et al., “Artificial neural network (ann) velocity better identifies benign prostatic hyperplasia but not prostate cancer compared with psa velocity,” *BMC Urology*, vol. 8, pp. 10, 2008.

[11] S. Yoo, D. Han, H. Chung, et al., “Automated detection of clinically significant prostate cancer in axial multiparametric mri using cnn,” *J. Magn. Reson. Imaging*, vol. 50, no. 4, pp. 1012–1021, 2019.

[12] R. Cao, J. Cai, J. Zhang, et al., “Focalnet: A multi-class cnn for prostate cancer detection and gleason score prediction,” *IEEE Trans. Med. Imaging*, vol. 38, no. 3, pp. 828–838, 2019.

[13] T. Pellicer, M. J. Larriba, V. Beltrán, et al., “Automated prostate cancer detection and gleason score classification using deep learning: Zonal and patient-level approaches,” *Eur. Radiol.*, vol. 32, no. 1, pp. 85–95, 2022.

[14] Kamilia Taguelmimt, Hong-Phuong Dang, Gustavo Andrade Miranda, Dimitris Visvikis, Bernard Malavaud, and Julien Bert, “Uncertainty-aware deep learning classification for mri-based prostate cancer detection,” in *MICCAI Workshop on Cancer Prevention through Early Detection*. Springer, 2024, pp. 114–123.

[15] Jie Hu, Li Shen, and Gang Sun, “Squeeze-and-excitation networks,” in *Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit.*, 2018.

[16] Anindo Saha et al., “Artificial intelligence and radiologists in prostate cancer detection on mri (pi-cai): an international, paired, non-inferiority, confirmatory study,” *The Lancet Oncology*, 2024.

[17] Fabian Isensee, Paul F Jaeger, Simon AA Kohl, Jens Petersen, and Klaus H Maier-Hein, “nnu-net: a self-configuring method for deep learning-based biomedical image segmentation,” *Nature methods*, vol. 18, no. 2, pp. 203–211, 2021.