Package: nisupply
nisupplyA python module for dealing with unstructured or semi-structured neuroimaging datasets which do not conform to theBrain Imaging Data Structure (BIDS). Although this packages was originall developed for neuroimaging datasets it can work on any file type!InstallationInstall via pip with:pip install nisupplyAimsThough more and more datasets become available in the standardized BIDS-format, researchers will still often find themelves in situations, where:The dataset is not BIDS-formatted at all (this is often the case with old 'in-house' datasets that were aquired during a time were BIDS didn't exist yet).The dataset is wrongly BIDS-formatted because it sticks to an outdated BIDS version or because the maintainers made errors (as a consequence, verification tools like theBIDS-validatorwill throw errors)It is not possible to convert the datasets to BIDS, because youdon't have access to the original DICOM-filesdon't have time and ressources to do sodon't have all the information, but the contact to the original maintainer got lostAs a consequence, one cannot use tools from theBIDS Apps universewhich by default require the files to be BIDS-conform in order to work. The idea behind the nisupply module is to provide helper functions that facilitate the often tedious data-wrangling work that can happen with unstructured data sets.DocumentationThe nisupply package provides three main modules:nisupply.io: for input-output-operations (finding files, copying them over to a different directory)nisupply.structure: helper functions to bring the dataset into a new formatnisupply.utils: helper functions that work on the files themselves (e.g. uncompressing.nii.gzto.niias needed for SPM12)Consider this semistructured dataset as an example:src
├── fmri_nback_subject_3_session_2.nii.gz
├── subject_1
│   ├── fmri_gambling.nii.gz
│   ├── fmri_nback.nii.gz
│   └── session_2
│       └── fmri_nback.nii.gz
├── subject_2_fmri_nback.nii.gz
└── subject_4.txtWe can runnisupply.io.get_filepath_dfon this directory to find all the files that we need and gather the filepaths in a pandas dataframe (that is, search the directory./srcfor files that end with.nii.gzand start withfmri_nback. Add two new columnssubject_idandtaskto it that extract these entities using regular expressions).df = get_filepath_df(src_dir='./src',
                     regex_dict={'subject_id':'subject_(\d)',
                                 'task': 'fmri_(nback|gambling)'},
                     file_suffix='.nii.gz',
                     file_prefix='fmri_nback')This gives you a pandas dataframe that looks like this:filepath subject_id   task
0  src/fmri_nback_subject_3_session_2.nii.gz          3  nback
1            src/subject_1/fmri_nback.nii.gz          1  nback
2  src/subject_1/session_2/fmri_nback.nii.gz          1  nbackWe can now usenisupply.structure.get_file_extensionandnisupply.structure.get_new_filepathto create a new filepath
for each file:df = get_file_extension(df)
df['dst'] = './dst'
df = get_new_filepath(df,template="{dst}/sub-{subject_id}/sub-{subject_id}_task-{task}{file_extension}")Which outputs:filepath  ...                         filepath_new
0  src/fmri_nback_subject_3_session_2.nii.gz  ...  ./dst/sub-3/sub-3_task-nback.nii.gz
1            src/subject_1/fmri_nback.nii.gz  ...  ./dst/sub-1/sub-1_task-nback.nii.gz
2  src/subject_1/session_2/fmri_nback.nii.gz  ...  ./dst/sub-1/sub-1_task-nback.nii.gzFinally, we can copy over the files usingnisupply.io.copy_files(df,src_col='filepath',tgt_col='filepath_new')to a new location to obtain a new
tidy dataset:dst/
├── sub-1
│   └── sub-1_task-nback.nii.gz
└── sub-3
    └── sub-3_task-nback.nii.gzNoteThe nisupply module doesnotprovide any functions to convert DICOM files to NIFTI files. If you are looking for tools to do that, check out tools likeheudiconvorbidscointhat can do that for you.Similar ProjectsThere are similar projects out there following the same idea:Have a look at Stephen Larroque'spathmatcherpackage which works primarily with regex.Theinterfaces.iomodule fromnipype(especially theDataFinderclass)The focus of nisupply is to avoid quite unreadable regex-matches as much as possible. It therefore is best suited for semi-structured datsets that are neither completely unordered but also neither 100% standardized.