# ClinicaDL: an open-source deep learning software for reproducible neuroimaging processing

Elina Thibeau-Sutre, Mauricio Diaz, Ravi Hassanaly, Olivier Colliot, Ninon Burgos

<figure><img src="image_1.png" /></figure>

## To cite this version:

Elina Thibeau-Sutre, Mauricio Diaz, Ravi Hassanaly, Olivier Colliot, Ninon Burgos. ClinicaDL: an open-source deep learning software for reproducible neuroimaging processing. OHBM 2022 - Annual meeting of the Organization for Human Brain Mapping, Jun 2022, Glasgow, United Kingdom. hal-04279014

HAL Id: hal-04279014

https://inria.hal.science/hal-04279014v1

Submitted on 10 Nov 2023

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
# ClinicaDL: an open-source deep learning software for reproducible neuroimaging processing

Elina Thibeau-Sutre†,¹, Mauricio Díaz†,¹,², Ravi Hassanaly¹, Olivier Colliot¹, Ninon Burgos¹

†Joint co-authorship

¹ Sorbonne Université, Institut du Cerveau - Paris Brain Institute, Inserm, CNRS, AP-HP, Hôpital Pitié Salpê-trière, Inria, Aramis project-team, Paris, France

²Inria, Service d'Expérimentation et de Développement, Paris, France

## Introduction

In addition to facing a reproducibility crisis (Hutson, 2018), deep learning studies often include methodological flaws, especially in the field of neuroimaging in which deep learning contributors may not be aware of the specificities of the domain. This situation leads to the production of biased and overestimated results. To overcome this issue, we developed ClinicaDL: an end-to-end deep learning framework for deep learning users working on neuroimaging data that aims to prevent common pitfalls that we identified and described in our previous study (Wen et al., 2020).

## Methods

ClinicaDL allows its users to work with a great diversity of neuroimaging data sets as it interacts with a neuroimaging standard, the Brain Imaging Data Structure (Gorgolewski et al., 2016) (BIDS). This structure can be easily preprocessed by Clinica (Routier et al., 2021), the companion project of ClinicaDL, to prepare images (structural MRI or PET) in a standard way (bias field correction, registration...) and store them in a folder hierarchy called CAPS (ClinicaA Processed Structure). ClinicaDL takes this folder as input and outputs a Model Analysis and Processing Structure (MAPS) which contains all information necessary for the reproduction of deep learning experiments. In addition, ClinicaDL implements a set of technical solutions to avoid the main methodological issues causing data leakage found in the literature. One of them consists in providing feature extractors (Fig. 1) ensuring that the data partitioning between train and test is at the subject level and not at the patch or slice level as it was already witnessed in several studies (Wen et al., 2020).

<figure><img src="image_3.png" /><figcaption>**Figure 1.** ClinicaDL users can handle their 3D images in different ways. They can use the whole 3D image, extract 3D patches according to a given size and stride, extract 2D slices according to a given axis or extract 3D regions of interest (ROI) according to a binary mask (which is not necessarily a cuboid) manually designed by users.</figcaption></figure>

## Results

ClinicaDL has adopted standard practices for software development and distribution of the software with the aim to facilitate the reproduction of experiments. The open-source code is available on Github (https://github.com/aramis-lab/clinicadl) and the software is distributed as a python package uploaded to the PyPI index. ClinicaDL ensures its usability by proposing a well-maintained documentation
(https://clinicadl.readthedocs.io/en/stable/) and tutorials (https://aramislab.paris.inria.fr/clinicadl/tuto/intro.html) that can be run online to help newcomers to familiarize with the software, but also to raise awareness on methodological flaws often found in our field. Continuous integration runs a series of tests on the main functionalities of the code and it prevents the introduction of new bugs and the detection of potential failures.

ClinicaDL includes a set of tools to prepare data for deep learning tasks (such as quality check, label definition, generation of synthetic data), architecture search, network training, as well as result inference, model evaluation and interpretation (Fig. 2). Different tasks can be learnt by the network trained by ClinicaDL, such as classification, regression and image reconstruction. However, the framework is meant to be easily enriched with new features of external contributors: this way new tasks can be easily added by creating a new class following a template. This process is described in the documentation to encourage new contributions to the software on several levels: tasks learnt by networks, architectures, validation settings, metrics and feature extractors.

<figure><img src="image_4.png" /><figcaption>**Figure 2.** ClinicaDL main functionalities. extract, tsvtool and generate functionalities read and write in the ClinicA Processed Structure (CAPS) which contains neuroimaging data preprocessed by Clinica pipelines. ClinicaDL writes its own outputs in the Model Analysis and Processing Structure (MAPS), which contains the results of the training phase as well as inference on new data or the results of interpretability methods.</figcaption></figure>

## Conclusion

ClinicaDL helps deep learning researchers to produce reliable studies in neuroimaging. ClinicaDL has been developed to answer three issues we identified in our field: (1) the difficult use of neuroimaging data sets by users with little expertise, (2) data leakage in validation and evaluation methods and (3) insufficient reproducibility. Some issues remain open for further contributions, such as exploiting optimally multi-gpu workspaces. To answer them, we plan to integrate in the future standard tools of the deep learning or neuroimaging communities, such as pytorch-lightning (Falcon & The PyTorch Lightning team, 2019), MLflow (MLflow, 2018/2021) or TorchIO (Pérez-García et al., 2021).

## References:

* Falcon, W. & The PyTorch Lightning team. (2019). *PyTorch Lightning* (1.4) [Python]. https://doi.org/10.5281/zenodo.3828935

* Gorgolewski, K. J., Auer, T., Calhoun, V. D., Craddock, R. C., Das, S., Duff, E. P., Flandin, G., Ghosh, S. S., Glatard, T., Halchenko, Y. O., Handwerker, D. A., Hanke, M., Keator, D., Li, X., Michael, Z., Maumet, C., Nichols, B. N., Nichols, T. E., Pellman, J., ... Poldrack, R. A. (2016). The brain imaging data structure, a format for organizing and describing outputs of neuroimaging experiments. *Scientific Data*, 3(1), 160044. https://doi.org/10.1038/sdata.2016.44

* Hutson, M. (2018). Artificial intelligence faces reproducibility crisis. *Science*, 359(6377), 725-726. https://doi.org/10.1126/science.359.6377.725
MLflow: A Machine Learning Lifecycle Platform. (2021). [Python]. MLflow.
https://github.com/mlflow/mlflow (Original work published 2018)

Pérez-García, F., Sparks, R., & Ourselin, S. (2021). TorchIO: A Python library for efficient loading, preprocessing, augmentation and patch-based sampling of medical images in deep learning. *Computer Methods and Programs in Biomedicine*, 208, 106236. https://doi.org/10.1016/j.cmpb.2021.106236

Routier, A., Burgos, N., Díaz, M., Bacci, M., Bottani, S., El-Rifai, O., Fontanella, S., Gori, P., Guillon, J., Guyot, A., Hassanaly, R., Jacquemont, T., Lu, P., Marcoux, A., Moreau, T., Samper-González, J., Teichmann, M., Thibeau-Sutre, E., Vaillant, G., ... Colliot, O. (2021). Clinica: An Open-Source Software Platform for Reproducible Clinical Neuroscience Studies. *Frontiers in Neuroinformatics*, 15, 689675. https://doi.org/10.3389/fninf.2021.689675

Wen, J., Thibeau-Sutre, E., Diaz-Melo, M., Samper-González, J., Routier, A., Bottani, S., Dormont, D., Durrleman, S., Burgos, N., Colliot, O., & for, the A. D. N. I. (2020). Convolutional neural networks for classification of Alzheimer's disease: Overview and reproducible evaluation. *Medical Image Analysis*, 63. Scopus. https://doi.org/10.1016/j.media.2020.101694