Package: antspymm
ANTsPyMMmappingprocessing utilities for timeseries/multichannel images - mostly neuroimagingthe outputs of these processes can be used for data inspection/cleaning/triage
as well for interrogating hypotheses.this package also keeps track of the latest preferred algorithm variations for
production environments.install thedevversion by calling (within the source directory):python setup.py installor install the latest release viapip install antspymmwhat this will doANTsPyMM will process several types of brain MRI into tabular form as well as normalized (standard template) space.  The processing includes:T1wHier uses hierarchical processing from ANTsPyT1w organized around these measurementsCIT168 template 10.1101/211201Desikan Killiany Tourville (DKT) 10.3389/fnins.2012.00171basal forebrain (Avants et al HBM 2022 abstract)other regions (egMTL) 10.1101/2023.01.17.23284693also produces jacobian datarsfMRI: resting state functional MRIuses 10.1016/j.conb.2012.12.009 to estimate network specific correlationsf/ALFF 10.1016/j.jneumeth.2008.04.012NM2DMT: neuromelanin mid-brain imagesCIT168 template 10.1101/211201DTI: DWI diffusion weighted images organized via:CIT168 template 10.1101/211201JHU atlases 10.1016/j.neuroimage.2008.07.009  10.1016/j.neuroimage.2007.07.053DKT for cortical to cortical tractography estimates based on DiPyT2Flair: flair for white matter hyperintensityhttps://pubmed.ncbi.nlm.nih.gov/30908194/https://pubmed.ncbi.nlm.nih.gov/30125711/https://pubmed.ncbi.nlm.nih.gov/35088930/T1w: voxel-based cortical thickness (DiReCT) 10.1016/j.neuroimage.2008.12.016Results of these processes are plentiful; processing for a single subject
will all modalities will take around 2 hours on an average laptop.documentation of functionshere.data dictionaryhere.notes on QChereachieved through four steps (recommended approach):organize data in NRG formatperform blind QCcompute outlierness per modality and select optimally matched modalities ( steps 3.1 and 3.2 )run the main antspymm functionfirst time setupimportantspyt1wimportantspymmantspyt1w.get_data(force_download=True)antspymm.get_data(force_download=True)NOTE:get_datahas aforce_downloadoption to make sure the latest
package data is installed.NOTE: some functions inantspynetwill download deep network model weights on the fly.  if one is containerizing, then it would be worth running a test case through in the container to make sure all the relevant weights are pre-downloaded.example processingsee the latest help but this snippet gives an idea of how one might use the package:importosos.environ["TF_NUM_INTEROP_THREADS"]="8"os.environ["TF_NUM_INTRAOP_THREADS"]="8"os.environ["ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS"]="8"importantspymmimportantspyt1wimportantspynetimportants...i/ocodehere...tabPro,normPro=antspymm.mm(t1,hier,nm_image_list=mynm,rsf_image=rsf,dw_image=dwi,bvals=bval_fname,bvecs=bvec_fname,flair_image=flair,do_tractography=False,do_kk=False,do_normalization=True,verbose=True)antspymm.write_mm('/tmp/test_output',t1wide,tabPro,normPro)blind quality controlthis package also provides tools to identify thebestmulti-modality image set at a given visit.the code below provides guidance on how to automatically qc, filter and match multiple modality images at each time point.  these tools are based on standard unsupervised approaches and are not perfect so we recommend using the associated plotting/visualization techniques to check the quality characterizations for each modality.## run the qc on all images - requires a relatively large sample per modality to be effective## then aggregateqcdf=pd.DataFrame()forfninfns:qcdf=pd.concat([qcdf,antspymm.blind_image_assessment(fn)],axis=0)qcdfa=antspymm.average_blind_qc_by_modality(qcdf,verbose=True)## reduce the time series qcqcdfaol=antspymm.outlierness_by_modality(qcdfa)# estimate outlier scoresprint(qcdfaol.shape)print(qcdfaol.keys)matched_mm_data=antspymm.match_modalities(qcdfaol)or just get modality-specific outlierness "by hand" then matchmm:importantspymmimportpandasaspdmymods=antspymm.get_valid_modalities()alldf=pd.DataFrame()forninrange(len(mymods)):m=mymods[n]jj=antspymm.collect_blind_qc_by_modality("qc/*"+m+"*csv")jjj=antspymm.average_blind_qc_by_modality(jj,verbose=False).dropna(axis=1)## reduce the time series qcjjj=antspymm.outlierness_by_modality(jjj,verbose=False)alldf=pd.concat([alldf,jjj],axis=0)jjj.to_csv("mm_outlierness_"+m+".csv")print(m+" done")# write the joined data outalldf.to_csv("mm_outlierness.csv",index=False)# find the best mm collectionmatched_mm_data=antspymm.match_modalities(alldf,verbose=True)matched_mm_data.to_csv("matched_mm_data.csv",index=False)matched_mm_data['negative_outlier_factor']=1.0-matched_mm_data['ol_loop'].astype("float")matched_mm_data2=antspymm.highest_quality_repeat(matched_mm_data,'subjectID','date',qualityvar='negative_outlier_factor')matched_mm_data2.to_csv("matched_mm_data2.csv",index=False)an example on open neuro (BIDS) datafrom :ANT PDimagesBIDS/
└── ANTPD
    └── sub-RC4125
        └── ses-1
            ├── anat
            │   ├── sub-RC4125_ses-1_T1w.json
            │   └── sub-RC4125_ses-1_T1w.nii.gz
            ├── dwi
            │   ├── sub-RC4125_ses-1_dwi.bval
            │   ├── sub-RC4125_ses-1_dwi.bvec
            │   ├── sub-RC4125_ses-1_dwi.json
            │   └── sub-RC4125_ses-1_dwi.nii.gz
            └── func
                ├── sub-RC4125_ses-1_task-ANT_run-1_bold.json
                ├── sub-RC4125_ses-1_task-ANT_run-1_bold.nii.gz
                └── sub-RC4125_ses-1_task-ANT_run-1_events.tsvimportantspymmimportpandasaspdimportglobasglobfns=glob.glob("imagesBIDS/ANTPD/sub-RC4125/ses-*/*/*gz")fns.sort()randid='000'# BIDS does not have unique image ids - so we assign onestudycsv=antspymm.generate_mm_dataframe('ANTPD','sub-RC4125','ses-1',randid,'T1w','/Users/stnava/data/openneuro/imagesBIDS/','/Users/stnava/data/openneuro/processed/',t1_filename=fns[0],dti_filenames=[fns[1]],rsf_filenames=[fns[2]])studycsv2=studycsv.dropna(axis=1)mmrun=antspymm.mm_csv(studycsv2,mysep='_')# aggregate the data after you've run on many subjects# studycsv_all would be the vstacked studycsv2 data frameszz=antspymm.aggregate_antspymm_results_sdf(studycsv_all,subject_col='subjectID',date_col='date',image_col='imageID',base_path=bd,splitsep='_',idsep='-',wild_card_modality_id=True,verbose=True)NRG exampleNRG format detailshereimagesNRG/
└── ANTPD
    └── sub-RC4125
        └── ses-1
            ├── DTI
            │   └── 000
            │       ├── ANTPD_sub-RC4125_ses-1_DTI_000.bval
            │       ├── ANTPD_sub-RC4125_ses-1_DTI_000.bvec
            │       ├── ANTPD_sub-RC4125_ses-1_DTI_000.json
            │       └── ANTPD_sub-RC4125_ses-1_DTI_000.nii.gz
            ├── T1w
            │   └── 000
            │       └── ANTPD_sub-RC4125_ses-1_T1w_000.nii.gz
            └── rsfMRI
                └── 000
                    └── ANTPD_sub-RC4125_ses-1_rsfMRI_000.nii.gzimportantspymmimportpandasaspdimportglobasglobt1fn=glob.glob("imagesNRG/ANTPD/sub-RC4125/ses-*/*/*/*T1w*gz")[0]# flair also takes a single imagedtfn=glob.glob("imagesNRG/ANTPD/sub-RC4125/ses-*/*/*/*DTI*gz")rsfn=glob.glob("imagesNRG/ANTPD/sub-RC4125/ses-*/*/*/*rsfMRI*gz")studycsv=antspymm.generate_mm_dataframe('ANTPD','sub-RC4125','ses-1','000','T1w','/Users/stnava/data/openneuro/imagesNRG/','/Users/stnava/data/openneuro/processed/',t1fn,rsf_filenames=rsfn,dti_filenames=dtfn)studycsv2=studycsv.dropna(axis=1)mmrun=antspymm.mm_csv(studycsv2,mysep='_')Population studiesLarge population studies may need more care to ensure everything is reproducibly organized and processed.  In this case, we recommend:1. blind qcfirst run the blind qc function that would look liketests/blind_qc.py.
this gives a quick view of the relevant data to be processed. it provides
both figures and summary data for each 3D and 4D (potential) input image.2. collect outlierness measurementsthe outlierness function gives one an idea of how each image relates to
the others in terms of similarity.  it may or may not succeed in detecting
true outliers but does a reasonable job of providing some rank ordering
of quality when there is repeated data.  seetests/outlierness.py.3. match the modalities for each subject and timepointthis occurs at the end oftests/outlierness.py.  the output of the
function will select the best quality time point multiple modality
collection and will define the antspymm cohort in a reproducible manner.4. run the antspymm processingfor each subject/timepoint, one would run:# ... imports above ...studyfn="matched_mm_data2.csv"df=pd.read_csv(studyfn)index=20# 20th subject/timepointcsvfns=df['filename']csvrow=df[df['filename']==csvfns[index]]csvrow['projectID']='MyStudy'############################################################################################template=ants.image_read("~/.antspymm/PPMI_template0.nii.gz")bxt=ants.image_read("~/.antspymm/PPMI_template0_brainmask.nii.gz")template=template*bxttemplate=ants.crop_image(template,ants.iMath(bxt,"MD",12))studycsv2=antspymm.study_dataframe_from_matched_dataframe(csvrow,rootdir+"nrgdata/data/",rootdir+"processed/",verbose=True)mmrun=antspymm.mm_csv(studycsv2,dti_motion_correct='SyN',dti_denoise=True,normalization_template=template,normalization_template_output='ppmi',normalization_template_transform_type='antsRegistrationSyNQuickRepro[s]',normalization_template_spacing=[1,1,1])5. aggregate resultsif you have a large population study then the last step would look like this:importantspymmimportglobasglobimportreimportpandasaspdimportosdf=pd.read_csv("matched_mm_data2.csv")pdir='./processed/'df['projectID']='MYSTUDY'merged=antspymm.merge_wides_to_study_dataframe(df,pdir,verbose=False,report_missing=False,progress=100)print(merged.shape)merged.to_csv("mystudy_results_antspymm.csv")useful tools for converting dicom to niftidcm2niixdicom2niftiimportdicom2niftidicom2nifti.convert_directory(dicom_directory,output_folder,compression=True,reorient=True)simpleitkimportSimpleITKassitkimportsysimportosimportglobasglobimportantsdd='dicom'oo='dicom2nifti'folders=glob.glob('dicom/*')k=0forfinfolders:print(f)reader=sitk.ImageSeriesReader()ff=glob.glob(f+"/*")dicom_names=reader.GetGDCMSeriesFileNames(ff[0])iflen(ff)>0:fnout='dicom2nifti/image_'+str(k).zfill(4)+'.nii.gz'ifnotexists(fnout):failed=Falsereader.SetFileNames(dicom_names)try:image=reader.Execute()except:failed=Truepassifnotfailed:size=image.GetSpacing()print(image.GetMetaDataKeys())print(size)sitk.WriteImage(image,fnout)img=ants.image_read(fnout)img=ants.iMath(img,'TruncateIntensity',0.02,0.98)ants.plot(img,nslices=21,ncol=7,axis=2,crop=True)else:print(f+": "+'empty')k=k+1build docspdoc -o ./docs antspymm --htmlssl errorif you get an odd certificate error when callingforce_download, try:importsslssl._create_default_https_context=ssl._create_unverified_contextto publish a releaserm -r -f build/ antspymm.egg-info/ dist/
python3 setup.py sdist bdist_wheel
twine upload --repository antspymm dist/*