Package: picai-prep
Preprocessing Utilities for 3D Medical Image ArchivesThis repository contains standardized functions to process 3D medical images and image archives —with its processing strategy being geared towards clinically significant prostate cancer (csPCa) detection in MRI. It is used for the official preprocessing pipeline of thePI-CAI challenge.Supported ConversionsDICOM Archive→MHA ArchiveMHA Archive→nnU-Net Raw Data ArchivennU-Net Raw Data Archive→nnDetection Raw Data ArchiveNote: theMHA Archive→nnU-Net Raw Data Archiveconversion includes resampling sequences to a shared voxel spacing (per sample). Optionally, this step can resample all samples to a uniform voxel spacing and/or take a centre crop.Installationpicai_prepis pip-installable:pip install picai_prepUsageOur preprocessing pipeline consists of four independent stages:DICOM Archive→MHA Archive→nnU-Net Raw Data Archive→nnDetection Raw Data Archive. All three conversion steps between these four stages can be performed independently. See below for documentation on each step.DICOM Archive → MHA ArchiveConversion fromDICOM Archive→MHA Archiveis controlled through a configuration file, which lists all DICOM sequences. This configuration file specifies how different sequences should be selected from the available DICOM sequences. An excerpt of the format is given below:"mappings":{"t2w":{"SeriesDescription":["t2_tse_tra"]},},"options":{"num_threads":4,"verify_dicom_filenames":True,"allow_duplicates":False},"archive":[{"patient_id":"ProstateX-0000","study_id":"07-07-2011-NA-MR prostaat kanker detectie WDSmc MCAPRODETW-05711","path":"ProstateX-0000/07-07-2011-NA-MR prostaat kanker detectie WDSmc MCAPRODETW-05711/3.000000-t2tsesag-87368"},]This excerpt includes all available options:num_threads: int, default: 4
	number of threads to use for multiprocessing
verify_dicom_filenames: bool, default: True
	explicitly verify dicom filenames as a sanity check
allow_duplicates: bool, default: False
	when multiple series apply to a mapping, convert allFull configuration file for this excerpt, can be foundhere. It can also be generated, as follows:frompicai_prep.examples.dcm2mha.sample_archiveimportgenerate_dcm2mha_settingsgenerate_dcm2mha_settings(archive_dir="/path/to/picai_public_images/",output_path="/path/to/picai_public_images/dcm2mha_settings.json")Using this configuration file, theDICOM Archive→MHA Archiveconversion can be performed using Python:frompicai_prepimportDicom2MHAConverterarchive=Dicom2MHAConverter(input_dir="/input/path/to/dicom/archive",output_dir="/output/path/to/mha/archive",dcm2mha_settings="/path/to/dcm2mha_settings.json",)archive.convert()Or from the command line:python-mpicai_prepdcm2mha--input/input/path/to/dicom/archive--output/output/path/to/mha/archive--json/path/to/dcm2mha_settings.jsonFor more examples ofDICOM Archivestructures, seeexamples/dcm2mha/.MHA Archive → nnU-Net Raw Data ArchiveConversion from theMHA Archiveformat to thennU-Net Raw Data Archiveformat is controlled through a configuration file, which lists all input sequences (and optionally, annotations). This configuration file specifies which sequences should be selected from the available (MHA) sequences. An excerpt of the format is given below:"dataset_json":{"task":"Task100_test",...},"preprocessing":{"matrix_size":[20,160,160],"spacing":[3.0,0.5,0.5]},"archive":[{"patient_id":"ProstateX-0000","study_id":"07-07-2011","scan_paths":["ProstateX-0000/ProstateX-0000_07-07-2011_t2w.mha","ProstateX-0000/ProstateX-0000_07-07-2011_adc.mha","ProstateX-0000/ProstateX-0000_07-07-20111_hbv.mha"],"annotation_path":"ProstateX-0000_07-07-2011.nii.gz"},]Full configuration file for this except, can be foundhere. It can also be generated, as follows:python-mpicai_prepmha2nnunet_settings--structurepicai_archive--input/input/images/--annotations/input/labels/csPCa_lesion_delineations/human_expert/resampled--json/workdir/mha2nnunet_settings.jsonOr from Python:frompicai_prep.examples.mha2nnunet.picai_archiveimportgenerate_mha2nnunet_settingsgenerate_mha2nnunet_settings(archive_dir="/input/images/",annotations_dir="/input/labels/csPCa_lesion_delineations/human_expert/resampledoutput_path="/workdir/mha2nnunet_settings.json",)The--annotations(command line) orannotations_dir(Python) parameter will check if the annotation is present in the specified folder. If not, the item will be skipped.Using this configuration file, theMHA Archive→nnU-Net Raw Data Archiveconversion can be performed using Python:frompicai_prepimportMHA2nnUNetConverterarchive=MHA2nnUNetConverter(scans_dir="/input/path/to/mha/archive",annotations_dir="/input/path/to/annotations",# defaults to input_pathoutput_dir="/output/path/to/nnUNet_raw_data",mha2nnunet_settings="/path/to/mha2nnunet_settings.json",)archive.convert()Or from the command line:python-mpicai_prepmha2nnunet--input/input/path/to/mha/archive--annotations/input/path/to/annotations--output/output/path/to/nnUNet_raw_data--json/path/to/mha2nnunet_settings.jsonOr using a Docker container:dockerrun-v/path/to/picai_data:/input\-v/path/to/nnUNet_raw_data:/output/\picai_nnunetpython-mpicai_prepmha2nnunet\--input/input/images\--annotations/input/labels/csPCa_lesion_delineations/human_expert/resampled\--output/output/nnUNet_raw_data\--json/input/mha2nnunet_settings.jsonFor more examples ofMHA Archivestructures, seeexamples/mha2nnunet/.nnU-Net Raw Data Archive → nnDetection Raw Data ArchiveFor certain applications, the nnU-Net and nnDetection raw data archive formats can be converted to each other. Conversion from the nnDetection to nnU-Net structure can always be performed (seennDetection's documentation). However, conversion from nnU-Net to nnDetection structure requires object instances to be non-connected and non-overlapping, such that they can be correctly identified as separate, individual objects. If this assumption holds true,nnU-Net Raw Data Archive→nnDetection Raw Data Archiveconversion can be performed using Python:frompicai_prepimportnnunet2nndetnnunet2nndet(nnunet_raw_data_path="/input/path/to/nnUNet_raw_data/Task100_test",nndet_raw_data_path="/output/path/to/nnDet_raw_data/Task100_test",)Or from the command line:python-mpicai_prepnnunet2nndet--input/input/path/to/nnUNet_raw_data/Task100_test--output/output/path/to/nnDet_raw_data/Task100_testWhat is a 'DICOM Archive'?With a DICOM archive we mean a dataset that comprises the scans as DICOM (.dcm) files, such as theProstateX dataset. Typically, such an archive is structured in the following way:/path/to/archive/
├── [patient UID]/
    ├── [study UID]/
        ├── [series UID]/
            ├── slice-1.dcm
            ...
            ├── slice-n.dcmIn a DICOM archive multiple sequences (such as axial T2-weighted scans) can exist, and each patient can have multiple studies. A single study can even have multiple instances of the same sequence, for example a repeated transversal T2-weighted scan when the first scan experienced motion blur artefacts.What is an 'MHA Archive'?With an MHA archive we mean a dataset that comprises the scans as MHA (.mha) files, such as thePI-CAI dataset. In case of the PI-CAI Challenge: Public Training and Development Dataset, the archive is structured in the following way (after extracting the zips):/path/to/archive/
├── [patient UID]/
    ├── [patient UID]_[study UID]_[modality].mha
    ...For the PI-CAI dataset, the available modalities aret2w(axial T2-weighted scan),adc(apparent diffusion coefficient map),hbv(calculated high b-value scan),sag(sagittal T2-weighted scan) andcor(coronal T2-weighted scan).ReferenceIf you are using this codebase or some part of it, please cite the following article:●A. Saha, J. J. Twilt, J. S. Bosma, B. van Ginneken, D. Yakar, M. Elschot, J. Veltman, J. J. Fütterer, M. de Rooij, H. Huisman, "Artificial Intelligence and Radiologists at Prostate Cancer Detection in MRI: The PI-CAI Challenge (Study Protocol)", DOI: 10.5281/zenodo.6667655BibTeX:@ARTICLE{PICAI_BIAS,
    author = {Anindo Saha, Jasper J. Twilt, Joeran S. Bosma, Bram van Ginneken, Derya Yakar, Mattijs Elschot, Jeroen Veltman, Jurgen Fütterer, Maarten de Rooij, Henkjan Huisman},
    title  = {{Artificial Intelligence and Radiologists at Prostate Cancer Detection in MRI: The PI-CAI Challenge (Study Protocol)}}, 
    year   = {2022},
    doi    = {10.5281/zenodo.6667655}
}Managed ByDiagnostic Image Analysis Group,
Radboud University Medical Center,
Nijmegen, The NetherlandsContact InformationJoeran Bosma:Joeran.Bosma@radboudumc.nlStan Noordman:Stan.Noordman@radboudumc.nlAnindo Saha:Anindya.Shaha@radboudumc.nlHenkjan Huisman:Henkjan.Huisman@radboudumc.nl