Package: aisegcell
aiSEGcell - OverviewThis repository contains atorchimplementation of U-Net (Ronneberger et al., 2015).
We providetrainedmodels to semantically segment nuclei and whole cells in bright field images.
Please citethis paperif you are using this code in your research.ContentsInstallationVirtual environmentpip installationSource installationDataTrainingTrained modelsTestingPredictingnapari pluginImage annotation toolsTroubleshooting & supportCitationInstallationIf you do not have python installed already, we recommend installing it using theAnaconda distribution.aisegcellwas tested withpython 3.8.6.Virtual environment setupIf you do not use and IDE that handlesvirtual environmentsfor you (e.g.PyCharm) use your command line application (e.g.Terminal) and
one of the many virtual environment tools (seehere). We will
usecondaCreate new virtual environmentcondacreate-naisegcellpython=3.8.6Activate virtual environmentcondaactivateaisegcellpip installationRecommended if you do not want to develop theaisegcellcode base.Installaisegcell# update pippipinstall-Upip==23.2.1
pipinstallaisegcell(Optional)GPUsgreatly speed up training and inference of U-Net and are available fortorch(v1.10.2) forWindowsandLinux. Check if yourGPU(s)are CUDA compatible
(Windows,Linux) and
update their drivers if necessary.Installtorch/torchvisioncompatible with your system.aisegcellwas tested withtorchversion1.10.2,torchvisionversion0.11.3, andcudaversion11.3.1. Depending on your OS, yourCPUorGPU(andCUDAversion) the installation may change# Windows/Linux CPUpipinstalltorch==1.10.2+cputorchvision==0.11.3+cpu-fhttps://download.pytorch.org/whl/cpu/torch_stable.html# Windows/Linux GPU (CUDA 11.3.X)pipinstalltorch==1.10.2+cu113torchvision==0.11.3+cu113-fhttps://download.pytorch.org/whl/cu113/torch_stable.html# macOS CPUpipinstalltorch==1.10.2torchvision==0.11.3Installpytorch-lightning.aisegcellwas tested with version1.5.9.# note the installation of v1.5.9 does not use pip install lightningpipinstallpytorch-lightning==1.5.9Source installationInstallation requires a command line application (e.g.Terminal) withgitandpythoninstalled.
If you operate onWindowswe recommend usingUbuntu on Windows.
Alternatively, you can installAnacondaand
useAnaconda Powershell Prompt. An introductory tutorial on how to usegitand GitHub can be foundhere.(Optional) If you useAnaconda Powershell Prompt, installgitthroughcondacondainstall-canacondagitclone the repository (considersshalternative)# change directorycd/path/to/directory/to/clone/repository/to

gitclonehttps://github.com/CSDGroup/aisegcell.gitNavigate to the cloned directorycdaisegcellInstallaisegcell# update pippipinstall-Upip==23.2.1as a userpipinstall.as a developer (in editable mode with development dependencies and pre-commit hooks)pipinstall-e".[dev]"pre-commitinstall(Optional)GPUsgreatly speed up training and inference of U-Net and are available fortorch(v1.10.2) forWindowsandLinux. Check if yourGPU(s)are CUDA compatible
(Windows,Linux) and
update their drivers if necessary.Installtorch/torchvisioncompatible with your system.aisegcellwas tested withtorchversion1.10.2,torchvisionversion0.11.3, andcudaversion11.3.1. Depending on your OS, yourCPUorGPU(andCUDAversion) the installation may change# Windows/Linux CPUpipinstalltorch==1.10.2+cputorchvision==0.11.3+cpu-fhttps://download.pytorch.org/whl/cpu/torch_stable.html# Windows/Linux GPU (CUDA 11.3.X)pipinstalltorch==1.10.2+cu113torchvision==0.11.3+cu113-fhttps://download.pytorch.org/whl/cu113/torch_stable.html# macOS CPUpipinstalltorch==1.10.2torchvision==0.11.3Installpytorch-lightning.aisegcellwas tested with version1.5.9.# note the installation of v1.5.9 does not use pip install lightningpipinstallpytorch-lightning==1.5.9DataU-Net is currently intended for single-class semantic segmentation. Input images are expected to be 8-bit or
16-bit greyscale images. Segmentation masks are expected to decode background as 0 intensity and all intensities
>0 are converted to a single intensity value (255). Consequently, different instances of a class (instance
segmentation) or multi-class segmentations are handled as single-class segmentations. Have a look atthis notebookfor a data example.TrainingTraining U-Net is as simple as calling the commandaisegcell_train. We provide anotebookon how to train
U-Net with a minimal working example.aisegcell_trainis available if you activate the virtual environment youinstalledand can be called with the following arguments:--help: show help message--data: Path to CSV file containing training image file paths. The CSV file must have the columnsbfandmask.--data_val: Path to CSV file containing validation image file paths (same format as--data).--output_base_dir: Path to output directory.--model: Model type to train (currently only U-Net). Default is "Unet".--checkpoint: Path to checkpoint file matching--model. Only necessary if continuing a model training.
Default isNone.--devices: Devices to use for model training. If you want to use GPU(s) you have to provideintIDs.
Multiple GPU IDs have to be listed separated by spacebar (e.g.2 5 9). If you want to use the CPU you have
to use "cpu". Default is "cpu".--epochs: Number of training epochs. Default is 5.--batch_size: Number of samples per mini-batch. Default is 2.--lr: Learning rate of the optimizer. Default is 1e-4.--base_filters: Number of base_filters of Unet. Default is 32.--shape: Shape [heigth, width] that all images will be cropped/padded to before model submission. Height
and width cannot be smaller than--receptive_field. Default is [1024,1024].--receptive_fieldReceptive field of a neuron in the deepest layer. Default is 128.--log_frequency: Log performance metrics every N gradient steps during training. Default is 50.--loss_weight: Weight of the foreground class compared to the background class for the binary cross entropy loss.
Default is 1.--bilinear: If flag is used, use bilinear upsampling, else transposed convolutions.--multiprocessing: If flag is used, all GPUs given in devices will be used for traininig. Does not support CPU.--retrain: If flag is used, best scores for model saving will be reset (required for training on new data).--transform_intensity: If flag is used random intensity transformations will be applied to input images.--seed: None or Int to use for random seeding. Default isNone.The commandaisegcell_generate_listcan be used to write CSV files for--dataand--data_valand
has the following arguments:--help: show help message--bf: Path (globpattern) to input images (e.g. bright field). Naming convention must match naming convention of--mask.--mask: Path (globpattern) to segmentation masks corresponding to--bf.--out: Directory to which output file is saved.--prefix: Prefix for output file name (i.e.{PREFIX}_paths.csv). Default is "train".Usewildcard characterslike*to select all files you want to
input to--bfand--mask(see example below).Consider the following example:# activate the virtual environmentcondaactivateaisegcell# generate CSV files for data and data_valaisegcell_generate_list\--bf"/path/to/train_images/*/*.png"# i.e. select all PNG files in all sub-directories of /path/to/train_images\--mask"/path/to/train_masks/*/*mask.png"# i.e. select all files in all sub-directories that end with "mask.png"\--out/path/to/output_directory\--prefixtrain

aisegcell_generate_list\--bf"/path/to/val_images/*.png"\--mask"/path/to/val_masks/*.png"\--out/path/to/output_directory\--prefixval# starting multi-GPU trainingaisegcell_train\--data/path/to/output_directory/train_paths.csv\--data_val/path/to/output_directory/val_paths.csv\--modelUnet\--devices24# use GPU 2 and 4 \--output_base_dir/path/to/results/folder\--epochs10\--batch_size8\--lr1e-3\--base_filters32\--shape1024512\--receptive_field128\--log_frequency5\--loss_weight1\--bilinear\--multiprocessing# required if you use multiple --devices \--transform_intensity\--seed123# OR retrain an existing checkpoint with single GPUaisegcell_train\--data/path/to/output_directory/train_paths.csv\--data_val/path/to/output_directory/val_paths.csv\--modelUnet\--checkpoint/path/to/checkpoint/file.ckpt--devices0\--output_base_dir/path/to/results/folder\--epochs10\--batch_size8\--lr1e-3\--base_filters32\--shape10241024\--receptive_field128\--log_frequency5\--loss_weight1\--bilinear\--transform_intensity\--seed123The output ofaisegcell_trainwill be stored in subdirectories{DATE}_Unet_{ID1}/lightning_logs/version_{ID2}/at--output_base_dir. Its contents are:hparams.yaml: stores hyper-parameters of the model (used bypytorch_lightning.LightningModule)metrics.csv: contains all metrics tracked during trainingloss_step: training loss (binary cross-entropy) per gradient stepepoch: training epochstep: training gradient steploss_val_step: validation loss (binary cross-entropy) per validation mini-batchf1_step:f1 scoreper validation mini-batchiou_step: average ofiou_small_stepandiou_big_stepper validation mini-batchiou_big_step:intersection over unionof objects with
> 2000 px in size per validation mini-batchiou_small_step:intersection over unionof objects
with <= 2000 px in size per validation mini-batchloss_val_epoch: averageloss_val_stepover all validation steps per epochf1_epoch: averagef1_stepover all validation steps per epochiou_epoch: averageiou_stepover all validation steps per epochiou_big_epoch: averageiou_big_epochover all validation steps per epochiou_small_epoch: averageiou_small_epochover all validation steps per epochloss_epoch: averageloss_stepover all training gradient steps per epochcheckpoints: model checkpoints are stored in this directory. Path to model checkpoints are used as input to--checkpointofaisegcell_trainor--modelofaisegcell_testandaisegcell_predict.best-f1-epoch={EPOCH}-step={STEP}.ckpt: model weights with the (currently) highestf1_epochbest-iou-epoch={EPOCH}-step={STEP}.ckpt: model weights with the (currently) highestiou_epochbest-loss-epoch={EPOCH}-step={STEP}.ckpt: model weights with the (currently) lowestloss_val_epochlatest-epoch={EPOCH}-step={STEP}.ckpt: model weights of the (currently) latest checkpointTrained modelsWe provide trained models:modalityimage formatexample imagedescriptionavailabilitynucleus segmentation2D grayscaleTrained on a data set (link to data set) of 9849 images (~620k nuclei).ETH Research Collectionwhole cell segmentation2D grayscaleTrained on a data set (link to data set) of 224 images (~12k cells).ETH Research CollectionTestingA trained U-Net can be tested withaisegcell_test. We provide anotebookon how to test
with U-Net.aisegcell_testreturns predicted masks and performance metrics.aisegcell_testcan be called with the
following arguments:--help: show help message--data: Path to CSV file containing test image file paths. The CSV file must have the columnsbfand--mask.--model: Path to checkpoint file of trained pytorch_lightning.LightningModule.--suffix: Suffix to append to all mask file names.--output_base_dir: Path to output directory.--devices: Devices to use for model training. If you want to use GPU(s) you have to provideintIDs.
Multiple GPU IDs have to be listed separated by spacebar (e.g.2 5 9). If multiple GPUs are provided only
the first ID will be used. If you want to use the CPU you have to use "cpu". Default is "cpu".Make sure to activate the virtual environment created duringinstallationbefore callingaisegcell_test.Consider the following example:# activate the virtual environmentcondaactivateaisegcell# generate CSV file for dataaisegcell_generate_list\--bf"/path/to/test_images/*.png"\--mask"/path/to/test_masks/*.png"\--out/path/to/output_directory\--prefixtest# run testingaisegcell_test\--data/path/to/output_directory/test_paths.csv\--model/path/to/checkpoint/file.ckpt\--suffixmask\--output_base_dir/path/to/results/folder\--devices0# predict with GPU 0The output ofaisegcell_testwill be stored in subdirectorieslightning_logs/version_{ID}/at--output_base_dir. Its contents are:hparams.yaml: stores hyper-parameters of the model (used bypytorch_lightning.LightningModule)metrics.csv: contains all metrics tracked during testing. Column IDs are identical tometrics.csvduringtrainingtest_masks: directory containing segmentation masks obtained from U-NetPredictingA trained U-Net can used for predictions withaisegcell_predict. We provide anotebookon how to
predict with U-Net.aisegcell_predictreturns only predicted masks metrics and can be called with the following
arguments:--help: show help message--data: Path to CSV file containing predict image file paths. The CSV file must have the columnsbfand--mask.--model: Path to checkpoint file of trained pytorch_lightning.LightningModule.--suffix: Suffix to append to all mask file names.--output_base_dir: Path to output directory.--devices: Devices to use for model training. If you want to use GPU(s) you have to provideintIDs.
Multiple GPU IDs have to be listed separated by spacebar (e.g.2 5 9). If multiple GPUs are provided only
the first ID will be used. If you want to use the CPU you have to use "cpu". Default is "cpu".Make sure to activate the virtual environment created duringinstallationbefore callingaisegcell_predict.Consider the following example:# activate the virtual environmentcondaactivateaisegcell# generate CSV file for dataaisegcell_generate_list\--bf"/path/to/predict_images/*.png"\--mask"/path/to/predict_images/*.png"# necessary to provide "--mask" for aisegcell_generate_list \--out/path/to/output_directory\--prefixpredict# run predictionaisegcell_predict\--data/path/to/output_directory/predict_paths.csv\--model/path/to/checkpoint/file.ckpt\--suffixmask\--output_base_dir/path/to/results/folder\--devices0# predict with GPU 0The output ofaisegcell_predictwill be stored in subdirectorieslightning_logs/version_{ID}/at--output_base_dir. Its contents are:hparams.yaml: stores hyper-parameters of the model (used bypytorch_lightning.LightningModule)predicted_masks: directory containing segmentation masks obtained from U-Netnapari pluginaisegcell_predictis also available as a plug-in fornapari(link to napari-hub page and github page).Image annotation toolsAvailable tools to annotate segmentations include:napariLabkitforFijiQuPathilastikTroubleshooting & supportIn case you are experiencing issues withaisegcellinform us via theissue tracker.
Before you submit an issue, check if it has been addressed in a previous issue.Citationt.b.d.