amorsi/micro-sam-vit-l-lm

micro-sam vit_l_lm

Light microscopy generalist, backbone ViT-L (best quality of the three, higher compute cost). Trained for cells and nuclei in light microscopy.

Unmodified mirror of a pretrained checkpoint from micro-sam (Segment Anything for Microscopy; code is MIT-licensed). Hashes match the model registry in micro_sam/util.py. The license of the weights is not stated here; check the original sources below.

Files

• model.pt — SAM checkpoint; original: https://uk1s3.embassy.ebi.ac.uk/public-datasets/bioimage.io/idealistic-rat/1.2/files/vit_l.pt
• decoder.pt — the instance-segmentation decoder used for automatic instance segmentation (AIS); original: https://uk1s3.embassy.ebi.ac.uk/public-datasets/bioimage.io/idealistic-rat/1.2/files/vit_l_decoder.pt

Usage

vit_l_lm is one of micro-sam's built-in model_type names, so it is normally downloaded automatically:

from micro_sam.util import get_sam_model

predictor = get_sam_model(model_type="vit_l_lm")

To use this mirror instead, download model.pt and pass it as checkpoint_path (get_sam_model(model_type="vit_l_lm", checkpoint_path="model.pt")). The library has submodules for prompt_based_segmentation (2D/3D), instance_segmentation (automatic), training (finetuning) and evaluation; see the documentation.

Recommended model sizes: vit_l or vit_b give the best trade-off between speed and quality; the authors report no advantage of vit_h, and vit_t is much faster but lower quality for many applications.

Training notes

Per the paper's supplementary Table 1, the light-microscopy generalist was trained on LIVECell, DeepBacs, TissueNet, PlantSeg (root), DSB (fluorescence nuclei), NeurIPS CellSeg and Cell Tracking Challenge data; CovidIF, HPA, Lizard, MouseEmbryo, PlantSeg (ovules) and PanNuke were held out for evaluation.

Models are finetuned from the Segment Anything weights. For the paper's models, the previous mask prompt is used with 50% probability during training, and the AIS decoder predicts a foreground map plus center- and boundary-distance maps (supplementary Fig. 2). These details come from the supplement; the checkpoints mirrored here may be later versions than those in the paper, so see the docs for current training data and performance.

Citation

Please cite the micro-sam paper (Nature Methods) and the original Segment Anything publication if you use these weights.