SAM 2: Segment Anything in Images and Videos
Paper โข 2408.00714 โข Published โข 123
How to use openEuler/sam2.1_hiera_tiny with sam2:
# Use SAM2 with images
import torch
from sam2.sam2_image_predictor import SAM2ImagePredictor
predictor = SAM2ImagePredictor.from_pretrained(openEuler/sam2.1_hiera_tiny)
with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
predictor.set_image(<your_image>)
masks, _, _ = predictor.predict(<input_prompts>) # Use SAM2 with videos
import torch
from sam2.sam2_video_predictor import SAM2VideoPredictor
predictor = SAM2VideoPredictor.from_pretrained(openEuler/sam2.1_hiera_tiny)
with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
state = predictor.init_state(<your_video>)
# add new prompts and instantly get the output on the same frame
frame_idx, object_ids, masks = predictor.add_new_points(state, <your_prompts>):
# propagate the prompts to get masklets throughout the video
for frame_idx, object_ids, masks in predictor.propagate_in_video(state):
...Segment Anything Model 2.1 (SAM2.1) with Hiera-Tiny backbone, packaged for the IB-Robot framework. Performs prompted segmentation: given an image and point/box prompts, generates instance masks.
inference_manifest.json โ deployment routing (schema v3)assets/ โ adapter config, torch weights (.pt)artifacts/ascend_310b/ โ encoder + decoder OM (batch=1)artifacts/ascend_310p/ โ encoder + decoder OM (batch=4)| Target | Backend | Runtime | Batch | Hardware |
|---|---|---|---|---|
ascend_310b |
ascend | ACL | 1 | Ascend 310B1 |
ascend_310p |
ascend | ACL | 4 | Ascend 310P1 |
torch_cpu |
torch | PyTorch | โ | CPU |
torch_cuda |
torch | PyTorch | โ | NVIDIA GPU |
Each Ascend deployment runs a two-stage pipeline: encoder then decoder.
Encoder input: host.sam2.image float32 [N,3,1024,1024] NCHW
Decoder outputs: host.sam2.mask_logits [N,1,256,256], host.sam2.iou_predictions [N,1], host.sam2.low_res_masks [N,1,256,256]
This bundle's torch weights originate from the upstream SAM2.1 checkpoint:
https://dl.fbaipublicfiles.com/segment_anything_2/092824/sam2.1_hiera_tiny.ptThe Ascend OM artifacts were converted from these torch weights. Download script: scripts/download_perception_models.sh.
@inproceedings{sam2,
title = {SAM 2: Segment Anything in Images and Videos},
author = {Ravi, Nikhila and Gabeur, Valentin and Hu, Yuan-Ting and others},
booktitle = {arXiv preprint arXiv:2408.00714},
year = {2024}
}
@software{ib_robot,
title = {IB-Robot: Intelligence Boom Robot},
url = {https://gitcode.com/openeuler/IB_Robot},
license = {Apache-2.0}
}
Base model
facebook/sam2.1-hiera-tiny