Package: emcaps
Source code for "Genetically encoded barcodes for correlative volume electron microscopy"This repository contains all code for the sequential ML pipeline of the paperGenetically encoded barcodes for correlative volume electron microscopy.Pretrained models can be found as release artifactshere. They are automatically downloaded and cached when needed by the code.InstallationOption 1: From PyPI (withpip)You can install the project with scripts and all dependencies by runningpip install emcapsNotes:It is recommended to use a newvirtual environmentfor this.If you encounter PyTorch issues with this setup, pleaseinstall PyTorch manually following the official instructionsIf you want to use the napari-based GUI, you will also need toinstall one of qtpy's supported Qt backends, for example PyQt5:pip install pyqt5This has to be done manually since none of the backends is compatible with all platforms and pip can't auto-select an optimal one.Option 2: From sources (withpiporconda)First obtain theproject sources(either clone ordownload zipand extract) andcdto the project root.If you want to install all dependencies and the package itself withpip, just runpip install .Alternatively, if you want to install the dependencies withconda, run the following commands:conda env create -f environment.yml
conda activate emcaps
pip install .Running the codeAll scripts can be executed from the project root directory usingpython3 -m, for example:$ python3 -m emcaps.inference.segment -hAlternatively you can use the entry points provided by the pip installation:$ emcaps-segment -hEntry points for testing on custom dataThese entry points just require raw images and don't require GPU resources. Labels are not needed.Napari-based interactive GUI tool for segmentation and EMcapsulin particle classification$ emcaps-encarior$ python3 -m emcaps.analysis.encariPerforming batch inference on a directory of images or single image files$ emcaps-segment segment.inp_path=<PATH_TO_FILE_OR_FOLDER>or$ python3 -m emcaps.inference.segment segment.inp_path=<PATH_TO_FILE_OR_FOLDER>Entry points for reproduction, retraining or evaluationThe following steps require a local copy of theofficial datasetor a dataset in the same structure. A GPU is highly recommended.Splitting labeled image dataset into training and validation images and normalizing the data format$ emcaps-splitdatasetor$ python3 -m emcaps.utils.splitdatasetTraining new segmentation models$ emcaps-segtrainor$ python3 -m emcaps.training.segtrainSegmentation inference and evaluationSegment and optionally also perform particle-level classification if a model is available, render output visualizations (colored classification overlays etc.) and compute segmentation metrics.$ emcaps-segmentor$ python3 -m emcaps.inference.segmentFor a usage example featuring config sweeps, see_scripts/seg_cls_test.shProducing a patch dataset based on image segmentationBased on segmentation (from a model or human annotation), extract particle-centered image patches and store them as separate files in addition to metadata. The resulting patch dataset can be used for training models for patch-based classification. In addition, A random sample of the validation patches is prepared for evaluation of human and model-based classification evaluation.$ emcaps-patchifysegor$ python3 -m emcaps.inference.patchifysegTraining new patch classifiersRequires the outputs ofpatchifyseg(see above).$ emcaps-patchtrainor$ python3 -m emcaps.training.patchtrainQuantitative evaluation of patch classification resultsRequires the outputs ofpatchifyseg(see above).$ emcaps-patchevalor$ python3 -m emcaps.inference.patchevalFor a usage example featuring config sweeps, see_scripts/patcheval.shRendering average images of patch collections and grouping patches by EMcapsulin typesRequires the outputs ofpatchifyseg(see above).$ emcaps-averagepatchesor$ python3 -m emcaps.analysis.averagepatchesConfiguration systemWe are using a common configuration system for the runnable code, based onHydraandOmegaConf.
A central default config file with explanatory comments is located atconf/conf.yaml.
It is written to be as automatic and minimal as possible, but it can still be necessary to change some of the values for experiments or adapting to a different system.For the syntax of such yaml-based config files please refer to the OmegaConf docs onaccess and manipulationandvariable interpolationFor running hydra-enabled code with custom configuration you can either point to a different config file with the-cpCLI flagor change config values directly on the CLI usingHydra's override syntaxDatasetIf you want to train own models and/or do quantitative evaluation on the official data, please find the datahereand extract it to~/emc/emcapsulin.Further notesFor more details see top-level docstrings in each file.