Package: inkid
inkidis a Python package and collection of scripts for identifying ink in volumetric CT data using machine learning.RequirementsPython >=3.8 is required.Installation$gitclonehttps://code.cs.uky.edu/seales-research/ink-id.git&&cdink-id# From code.cs server$gitclonehttps://gitlab.com/educelab/ink-id.git&&cdink-id# From gitlab.com$python-mvenv.venv# Create a virtual environment$..venv/bin/activate# Activate the environment(.venv)$pipinstall-Upip# Upgrade pip(.venv)$pipinstall-e.# Install ink-id and dependencies(.venv)$deactivate# When finished, deactivate the environmentCython modules are automatically built during the installation.
After changes to Cython files (.pyxand.pxd), those modules must be rebuilt:$pythonsetup.pybuild_ext--inplaceUsageThe package can be used as a Python library:importinkidparams=inkid.ops.json_schema('dataSource0.1')regions=inkid.data.Dataset(['./training_dataset.txt'])A script is also included for running a training job and/or generating prediction images:$ inkid-train-and-predict
usage: inkid-train-and-predict [options]ExamplesSLURM JobsThis code is often used in Singularity containers, run as SLURM jobs on a compute cluster.
For documentation of this usage, seesingularity/inkid.def.K-Fold Cross Validation (and Prediction)inkid/scripts/train_and_predict.py``typicallytakes a dataset files as input and trains on the specified training regions, validates on the validation regions, and predicts on the prediction regions. However if the``--cross-validate-onargument is passed, the behavior is slightly different. The nth training source will be removed
from the training set and added to the validation and prediction sets. Example:$inkid-train-and-predict--training-set~/data/dri-datasets-drive/LunateSigma/grid-2x5.txt\--cross-validate-on7\--final-prediction-on-all\--output~/data/LunateSigmaGridTest00It is possible to schedule all of these jobs with one command if using SLURM’ssbatch. Example:$./submit_with_summary.shsbatch-pP4V12_SKY32M192_L--time=1-00:00:00--mem=187G--array=0-1submit.sh\--training-set/pscratch/seales_uksr/dri-datasets-drive/Dummy/DummyTest.volpkg/working/DummyTest_1x2Grid.txt\--subvolume-shape-voxels484848\--final-prediction-on-all\--prediction-grid-spacing2\--label-typergb_values\--subvolume-shape-microns3002020\--output/pscratch/seales_uksr/dri-experiments-drive/inkid/results/DummyTest/check_gpu/03After performing a run for each value of--cross-validate-on, each will have created a subdirectory of output.Generating Summary ImagesThere is a scriptinkid/scripts/misc/create_summary_images.pythat takes the parent output directory and will
generate various output images combining the cross-validation results. Example:$pythoninkid/scripts/misc/add_k_fold_prediction_images.py~/data/out/carbon_phantom_col1_test/Grid TrainingWhen working with only one surface PPM, it is often desirable to split that single region into a grid to be used with
k-fold cross-validation. There is a script to automatically create the grid dataset file:$pythoninkid/scripts/split_region_into_grid.py~/data/dri-datasets-drive/Dummy/DummyTest.volpkg/working/DummyTest.json12Then use this dataset for standard k-fold cross validation and prediction.MiscellaneousThere is a dummy test dataset in the DRI Datasets Drive that is meant to be a small volume to quickly validate
training and prediction code. If something major has been broken such as dimensions in the neural network model, this
will make that clear without having to wait for large volumes to load. Example:$./submit_with_summary.shsbatch-pP4V12_SKY32M192_L--time=1-00:00:00--mem=150Gsubmit.sh\--training-set$PSCRATCH/seales_uksr/dri-datasets-drive/Dummy/DummyTest.volpkg/working/DummyTest_grid1x2.txt\--subvolume-shape-voxels4822\--final-prediction-on-all\--prediction-grid-spacing2\--label-typergb_values\--cross-validate-on0\--output~/temp/test00Texture a region using an existing trained model (important parts:--modeland--skip-training:$./submit_with_summary.shsbatch-pP4V12_SKY32M192_L--time=1-00:00:00--mem=187Gsubmit.sh\--training-set$PSCRATCH/seales_uksr/dri-datasets-drive/MorganM910/MS910.volpkg/working/segmentation/quire.json\--prediction-set$PSCRATCH/seales_uksr/dri-datasets-drive/MorganM910/MS910.volpkg/working/segmentation/p60.json\--subvolume-shape484848\--final-prediction-on-all\--prediction-grid-spacing8\--label-typergb_values\--skip-training\--model$PSCRATCH/seales_uksr/dri-experiments-drive/inkid/results/MS910/p60/initial/09/2021-02-08_09.15.07/checkpoints/checkpoint_0_175000.pt\--output$PSCRATCH/seales_uksr/dri-experiments-drive/inkid/results/MS910/p60/fromSavedWeights/02ContributingWhen contributing to this repository, please first discuss the change you wish to make via issue, email, or another method with the owners of this repository.We follow the git branching model describedhereand document code based on theGoogle Python Style Guide standards.LicenseThis package is licensed under the GNU General Public License (GPLv3) - seeLICENSEfor details.