EfficientViT-SAM: Accelerated Segment Anything Model Without Performance
  Loss
Authors: name: Zhuoyang Zhang; name: Han Cai; name: Song Han
Published: 2024-02-07 16:28:36

Abstract: We present EfficientViT-SAM, a new family of accelerated segment anything
models. We retain SAM's lightweight prompt encoder and mask decoder while
replacing the heavy image encoder with EfficientViT. For the training, we begin
with the knowledge distillation from the SAM-ViT-H image encoder to
EfficientViT. Subsequently, we conduct end-to-end training on the SA-1B
dataset. Benefiting from EfficientViT's efficiency and capacity,
EfficientViT-SAM delivers 48.9x measured TensorRT speedup on A100 GPU over
SAM-ViT-H without sacrificing performance. Our code and pre-trained models are
released at https://github.com/mit-han-lab/efficientvit.