browser-perception models
Collection
4 items β’ Updated
How to use imagingforgood/sam2.1-hiera-tiny-onnx with sam2:
# Use SAM2 with images
import torch
from sam2.sam2_image_predictor import SAM2ImagePredictor
predictor = SAM2ImagePredictor.from_pretrained("imagingforgood/sam2.1-hiera-tiny-onnx")
with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
predictor.set_image(<your_image>)
masks, _, _ = predictor.predict(<input_prompts>) # Use SAM2 with videos
import torch
from sam2.sam2_video_predictor import SAM2VideoPredictor
predictor = SAM2VideoPredictor.from_pretrained("imagingforgood/sam2.1-hiera-tiny-onnx")
with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16):
state = predictor.init_state(<your_video>)
# add new prompts and instantly get the output on the same frame
frame_idx, object_ids, masks = predictor.add_new_points(state, <your_prompts>)
# propagate the prompts to get masklets throughout the video
for frame_idx, object_ids, masks in predictor.propagate_in_video(state):
...An unchanged copy of the ONNX export of Meta's SAM 2.1 Hiera Tiny that browser-perception runs in production, kept by Imaging for Good so it stays available. Only the four files the library uses are copied, plus the export's configuration files. We did not train, convert or modify this model.
814a066640debee5a91e70aa401fb8e17e030503. That repository also has
quantized variants (int8, q4, β¦), not copied here.| File | Bytes | SHA-256 |
|---|---|---|
onnx/vision_encoder_fp16.onnx |
314,925 | 7773e79d589dada8e1e630cc7eb18b17b2c5b4faeb1eaab5ebf9df618ecbd50a |
onnx/vision_encoder_fp16.onnx_data |
67,005,504 | a4dd3759e9b6a476d991fb3493787992e78777e51462695e1bede71ae258103e |
onnx/vision_encoder.onnx |
354,238 | 4f30aacd3aaefbca81a0b7fe4c1fc96345570ea0a6f80ced599493d1b3be2e8c |
onnx/vision_encoder.onnx_data |
134,084,864 | e83df9866a5afe68ea7f0f721f18f65137fc3acbf0da1c74e946d363e09c69cc |
onnx/prompt_encoder_mask_decoder_fp16.onnx |
229,799 | 00ac7200451f246e8307aa1a1648113f47dc45625738406403a0e6b86c69fc55 |
onnx/prompt_encoder_mask_decoder_fp16.onnx_data |
10,454,016 | 027b457a53f5bd76bb401c59555448451d04b4eab6d2f6a6758406b40133ee2c |
onnx/prompt_encoder_mask_decoder.onnx |
213,114 | 874414704c5d686db7d206a35f6e15d26563d50c8c4468fccc6739bd7e491dcf |
onnx/prompt_encoder_mask_decoder.onnx_data |
20,958,208 | e9874d900dd4134ed60eab1e97910327c2419e0b2954485d8fd6e7f1a1470f47 |
Also config.json, preprocessor_config.json and processor_config.json, unchanged. |
||
Each .onnx file keeps its weights in the .onnx_data file next to it. |
vision_encoder: pixel_values [1, 3, 1024, 1024] float32 β image_embeddings.0
[1, 32, 256, 256], .1 [1, 64, 128, 128], .2 [1, 256, 64, 64].prompt_encoder_mask_decoder: input_points, input_labels, input_boxes and the
three embeddings β iou_scores [1, 1, 3], pred_masks [1, 1, 3, 256, 256] (logits),
object_score_logits.shader-f16 on WebGPU; the fp32 files run anywhere.Apache-2.0, as the original model; see LICENSE (copied from facebookresearch/sam2).
Base model
facebook/sam2.1-hiera-tiny