SAM2 Lip Segmentation

Fine-tuned Segment Anything Model 2 (SAM2.1 Hiera Small) weights for semi-automated mouth landmark localization evaluation. These weights were produced by training the model on 1,600 images from the LaPa dataset and 1,600 images from the new MetaHuman Lip Segmentation dataset (linked below).

Model and Training Parameters

Base model: SAM2.1 Hiera Small
Fine-tuning datasets: Landmark-guided face Parsing dataset (LaPa) and MetaHuman Lip Segmentation Dataset
Prompt encoder and mask decoder were fine-tuned for 30 epochs
Learning rate: 1e-5
Batch size: 8

Included Files

sam2 config file: sam2.1_hiera_s.yaml
weights: finetuned_lip_seg_sam2.1s.pt sample annotated video: sample_lip_segmentation.mp4

Training Datasets

This model was fine-tuned using MetaHuman Lip Segmentation Dataset. and Landmark-Guided Face Parsing Dataset

Citation

TBD

Downloads last month: -; Downloads are not tracked for this model. How to track

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support