Package: mmv-im2im
MMV Im2Im TransformationA generic python package for deep learning based image-to-image transformation in biomedical applications(We are actively working on the documentation and tutorials. Submit a feature request if there is anything you need.)OverviewThe overall package is designed with a generic image-to-image transformation framework, which could be directly used for semantic segmentation, instance segmentation, image restoration, image generation, labelfree prediction, staining transformation, etc.. The implementation takes advantage of the state-of-the-art ML engineering techniques for users to focus on researches without worrying about the engineering details. In our pre-printarxiv link, we demonstrated the effectiveness ofMMV_Im2Imin more than ten different biomedical problems/datasets.For computational biomedical researchers (e.g., AI algorithm development or bioimage analysis workflow development), we hope this package could serve as the starting point for their specific problems, since the image-to-image "boilerplates" can be easily extended further development or adapted for users' specific problems.For experimental biomedical researchers, we hope this work provides a comprehensive view of the image-to-image transformation concept through diversified examples and use cases, so that deep learning based image-to-image transformation could be integrated into the assay development process and permit new biomedical studies that can hardly be done only with traditional experimental methodsInstallationBefore starting, we recommend tocreate a new conda environmentora virtual environmentwith Python 3.9+.Please note that the proper setup of hardware is beyond the scope of this pacakge. This package was tested with GPU/CPU on Linux/Windows and CPU on MacOS. [Special note for MacOS users: Directly pip install in MacOS may needadditional setup of xcode.]Install the development head of MONAIDue to this bug in MONAI:https://github.com/Project-MONAI/MONAI/pull/6523, which has been fixed in the dev branch but not in the stable release yet, we need to install the current dev head to avoid this bug. To do this:git clone https://github.com/Project-MONAI/MONAI.git
cd ./MONAI
# note: the dev branch is the default branch after cloning MONAI
pip install .Install MMV_Im2Im for basic usage:(For users only using this package, not planning to change any code or make any extension):Option 1: core functionality onlypip install mmv_im2imOption 2: advanced functionality (core + logger)pip install mmv_im2im[advance]Option 3: to reproduce paper:pip install mmv_im2im[paper]Option 4: install everything:pip install mmv_im2im[all]For MacOS users, additional ' ' marks are need when using installation tags in zsh. For example,pip install mmv_im2im[paper]should bepip install mmv_im2im'[paper]'in MacOS.Install MMV_Im2Im for customization or extension:git clone https://github.com/MMV-Lab/mmv_im2im.git
cd mmv_im2im
pip install -e .[all]Note: The-eoption is the so-called "editable" mode. This will allow code changes taking effect immediately. The installation tags,embedseg,advance,paper,all, are be selected based on your needs.(Optional) Install using DockerIt is also possible to use our package throughdocker. The installation tutorial ishere.(Optional) Use MMV_Im2Im with Google ColabWe provide a web-based demo, if cloud computing is preferred. you can. The same demo can de adapted for different applications.Quick startYou can try out on a simple example followingthe quick start guideBasically, you can specify your training configuration in a yaml file and run training withrun_im2im --config /path/to/train_config.yaml. Then, you can specify the inference configuration in another yaml file and run inference withrun_im2im --config /path/to/inference_config.yaml. You can also run the inference as a function with the provided API. This will be useful if you want to run the inference within another python script or workflow.  Here is an example:from pathlib import Path
from aicsimageio import AICSImage
from aicsimageio.writers import OmeTiffWriter
from mmv_im2im.configs.config_base import ProgramConfig, parse_adaptor, configuration_validation
from mmv_im2im import ProjectTester

# load the inference configuration
cfg = parse_adaptor(config_class=ProgramConfig, config="./paper_configs/semantic_seg_2d_inference.yaml")
cfg = configuration_validation(cfg)

# define the executor for inference
executor = ProjectTester(cfg)
executor.setup_model()
executor.setup_data_processing()

# get the data, run inference, and save the result
fn = Path("./data/img_00_IM.tiff")
img = AICSImage(fn).get_image_data("YX", Z=0, C=0, T=0)
# or using delayed loading if the data is large
# img = AICSImage(fn).get_image_dask_data("YX", Z=0, C=0, T=0)
seg = executor.process_one_image(img)
OmeTiffWriter.save(seg, "output.tiff", dim_orders="YX")Tutorials, examples, demonstrations and documentationsThe overall package aims to achieve both simplicty and flexibilty with the modularized image-to-image boilerplates. To help different users to best use this package, we provide documentations from four different aspects:Examples (i.e., scripts and config files)for reproducing all the experiments in ourpre-printA bottom-up tutorials onhow to understand the modularized image-to-image boilerplates(for extending or adapting the package) andhow to understand the configuration system in details(for advance usage to make specific customization).A top-down tutorials asFAQ, which will continuously grow as we receive more questions.Contribute models toBioImage Model ZooWe highly appreciate the BioImage Model Zoo's initiative to provide a comprehensive collection of pre-trained models for a wide range of applications. To make MMV_Im2Im trained models available as well, the first step involves extracting the state_dict from the PyTorch Lightning checkpoint.
This can be done via:importtorchckpt_path="./lightning_logs/version_0/checkpoints/last.ckpt"checkpoint=torch.load(ckpt_path,map_location=torch.device('cpu'))state_dict=checkpoint['state_dict']torch.save(state_dict,"./state_dict.pt")All further steps to provide models can be found in theofficial documentation.DevelopmentSeeCONTRIBUTING.mdfor information related to developing the code.MIT license