Instructions to use lkibbalam/scut-edgetam-coreml with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- sam2
How to use lkibbalam/scut-edgetam-coreml with sam2:
# Use SAM2 with images import torch from sam2.sam2_image_predictor import SAM2ImagePredictor predictor = SAM2ImagePredictor.from_pretrained("lkibbalam/scut-edgetam-coreml") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): predictor.set_image(<your_image>) masks, _, _ = predictor.predict(<input_prompts>)# Use SAM2 with videos import torch from sam2.sam2_video_predictor import SAM2VideoPredictor predictor = SAM2VideoPredictor.from_pretrained("lkibbalam/scut-edgetam-coreml") with torch.inference_mode(), torch.autocast("cuda", dtype=torch.bfloat16): state = predictor.init_state(<your_video>) # add new prompts and instantly get the output on the same frame frame_idx, object_ids, masks = predictor.add_new_points(state, <your_prompts>) # propagate the prompts to get masklets throughout the video for frame_idx, object_ids, masks in predictor.propagate_in_video(state): ... - Notebooks
- Google Colab
- Kaggle
EdgeTAM for Core ML (Scut)
Meta's EdgeTAM, the on-device Track Anything Model (SAM 2 with a memory), converted to four compiled Core ML models. The Scut iOS video editor downloads them as an optional feature, Better cutout, which keeps the person you tap cut out through a clip.
| Model | What it does |
|---|---|
EdgeTAMImageEncoder.mlmodelc |
encodes a frame |
EdgeTAMInitializer.mlmodelc |
seeds the track from a tap or a box |
EdgeTAMPropagator.mlmodelc |
carries the mask to the next frame from the memory |
EdgeTAMMemoryEncoder.mlmodelc |
writes a frame's mask into the memory |
Source
- Code:
facebookresearch/EdgeTAMat7711e012a30a2402c4eaab637bdb00a521302c91 - Checkpoint:
facebook/EdgeTAMat14d7ecc48c656b94e5184519f698cd5386c5a2bf,edgetam.pt, SHA-256ed2d4850b8792c239689b043c47046ec239b6e808a3d9b6ae676c803fd8780df - Converted from Meta's own code and checkpoint by Scut's
tools/edgetam/convert.pyand packaged bytools/edgetam/package.sh. These are not a third party's converted binaries.
The app fetches the files at a pinned commit of this repository and checks each file's size and SHA-256 before using it.
Licence
Apache License 2.0, © Meta Platforms, Inc. and affiliates. The converted models keep the licence of the code and checkpoint they were made from.
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for lkibbalam/scut-edgetam-coreml
Base model
facebook/EdgeTAM