Instructions to use Extreme-Media/video-redactor-sam2.1-tiny-onnx with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sam2
How to use Extreme-Media/video-redactor-sam2.1-tiny-onnx with sam2:
# Use SAM2 with images import torch from sam2.sam2_image_predictor import SAM2ImagePredictor predictor = SAM2ImagePredictor.from_pretrained("Extreme-Media/video-redactor-sam2.1-tiny-onnx") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): predictor.set_image(<your_image>) masks, _, _ = predictor.predict(<input_prompts>)# Use SAM2 with videos import torch from sam2.sam2_video_predictor import SAM2VideoPredictor predictor = SAM2VideoPredictor.from_pretrained("Extreme-Media/video-redactor-sam2.1-tiny-onnx") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): state = predictor.init_state(<your_video>) # add new prompts and instantly get the output on the same frame frame_idx, object_ids, masks = predictor.add_new_points(state, <your_prompts>) # propagate the prompts to get masklets throughout the video for frame_idx, object_ids, masks in predictor.propagate_in_video(state): ... - Notebooks
- Google Colab
- Kaggle
SAM 2.1 tiny β ONNX export for Video Redactor
These files are an ONNX export of Meta's SAM 2.1 tiny (sam2.1_hiera_tiny.pt)
used by Video Redactor for its
Accurate AI tracking mode. Video Redactor downloads them on first use and
verifies every file's SHA-256 before loading it; it runs them locally with
ONNX Runtime β no video ever leaves the computer.
They are published here only so the app can download them. They are tied to the app's JavaScript glue code (single-object, box-prompted video propagation) and are not a general-purpose SAM 2 runtime.
| File | Contents | SHA-256 |
|---|---|---|
enc.onnx |
image encoder (+ mask decoder's high-res feature convs) | dc0b5a163d35ac94f2f6f9495acd08b07d01727fa9f51d9339aabae85b278aa4 |
dec.onnx |
prompt encoder + mask decoder | 5bae8dcc82db7c640e80791f207c700b0c99ed9455fd3aa44bbfccc5102e8d77 |
menc.onnx |
memory encoder | db9fefce908f5402f0838f9317545b4029ac496c45d6c54db2a1c552bd00fe43 |
mattn.onnx |
memory attention (fixed 7-frame bank + 16 object pointers) | c18925fa3e961591dd5ea7fa3dcbcc4f210c779fa4e2d39c57f09f7b84a412cb |
consts.npz |
constant tensors (position encodings, no-memory embedding, no-object pointer) | 4c7f2efb06b3f1bf79d349762354091ce4cfe78672f07784d8b3d47380db598e |
Total β 168 MB, float32, ONNX opset 17, image size 512.
Licence and modifications
SAM 2 is Β© Meta Platforms, Inc. and licensed under the Apache License 2.0 (facebookresearch/sam2). This is a modified form of the model: the predictor was split into four ONNX graphs plus a constants file, the memory attention takes a fixed-size memory bank, and the rotary position encoding was rewritten with real arithmetic so it could be exported. The trained weights are unchanged. See NOTICE for the full modification notice.
Model tree for Extreme-Media/video-redactor-sam2.1-tiny-onnx
Base model
facebook/sam2.1-hiera-tiny