Package: segment-anything-fast
Fast Segment Anything[📕Paper] [🤗HuggingFace Demo] [Colab demo] [Replicate demo & API] [Model Zoo] [BibTeX]TheFast Segment Anything Model(FastSAM)is a CNN Segment Anything Model trained using only 2% of the SA-1B dataset published by SAM authors. FastSAM achieves comparable performance with
the SAM method at50× higher run-time speed.🍇 Updates2023/07/06Added toUltralytics (YOLOv8) Model Hub. Thanks toUltralyticsfor help 🌹.2023/06/29Supporttext modein HuggingFace Space. Thanks a lot togaoxingefor help 🌹.2023/06/29ReleaseFastSAM_Awesome_TensorRT. Thanks a lot toChuRuaNh0for providing the TensorRT model of FastSAM 🌹.2023/06/26ReleaseFastSAM Replicate Online Demo. Thanks a lot toChenxifor providing this nice demo 🌹.2023/06/26Supportpoints modein HuggingFace Space. Better and faster interaction will come soon!2023/06/24Thanks a lot toGrounding-SAMfor Combining Grounding-DINO with FastSAM inGrounded-FastSAM🌹.InstallationClone the repository locally:pipinstallsegment-anything-fastGetting StartedFirst download amodel checkpoint.Then, you can run the scripts to try the everything mode and three prompt modes.# Everything modepythonInference.py--model_path./weights/FastSAM.pt--img_path./images/dogs.jpg# Text promptpythonInference.py--model_path./weights/FastSAM.pt--img_path./images/dogs.jpg--text_prompt"the yellow dog"# Box prompt (xywh)pythonInference.py--model_path./weights/FastSAM.pt--img_path./images/dogs.jpg--box_prompt"[[570,200,230,400]]"# Points promptpythonInference.py--model_path./weights/FastSAM.pt--img_path./images/dogs.jpg--point_prompt"[[520,360],[620,300]]"--point_label"[1,0]"You can use the following code to generate all masks, make mask selection based on prompts, and visualize the results.fromfastsamimportFastSAM,FastSAMPromptmodel=FastSAM('./weights/FastSAM.pt')IMAGE_PATH='./images/dogs.jpg'DEVICE='cpu'everything_results=model(IMAGE_PATH,device=DEVICE,retina_masks=True,imgsz=1024,conf=0.4,iou=0.9,)prompt_process=FastSAMPrompt(IMAGE_PATH,everything_results,device=DEVICE)# everything promptann=prompt_process.everything_prompt()# bbox default shape [0,0,0,0] -> [x1,y1,x2,y2]ann=prompt_process.box_prompt(bbox=[[200,200,300,300]])# text promptann=prompt_process.text_prompt(text='a photo of a dog')# point prompt# points default [[0,0]] [[x1,y1],[x2,y2]]# point_label default [0] [1,0] 0:background, 1:foregroundann=prompt_process.point_prompt(points=[[620,360]],pointlabel=[1])prompt_process.plot(annotations=ann,output_path='./output/dog.jpg',)You are also welcomed to try our Colab demo:FastSAM_example.ipynb.Different Inference OptionsWe provide various options for different purposes, details are inMORE_USAGES.md.Web demoGradio demoWe also provide a UI for testing our method that is built with gradio. You can upload a custom image, select the mode and set the parameters, click the segment button, and get a satisfactory segmentation result. Currently, the UI supports interaction with the 'Everything mode' and 'points mode'. We plan to add support for additional modes in the future. Running the following command in a terminal will launch the demo:# Download the pre-trained model in "./weights/FastSAM.pt"
python app_gradio.pyThis demo is also hosted onHuggingFace Space.Replicate demoReplicate demohas supported all modes, you can experience points/box/text mode.Model CheckpointsTwo model versions of the model are available with different sizes. Click the links below to download the checkpoint for the corresponding model type.defaultorFastSAM:YOLOv8x based Segment Anything Model|Baidu Cloud (pwd: 0000).FastSAM-s:YOLOv8s based Segment Anything Model.ResultsAll result were tested on a single NVIDIA GeForce RTX 3090.1. Inference timeRunning Speed under Different Point Prompt Numbers(ms).methodparams110100E(16x16)E(32x32*)E(64x64)SAM-H0.6G44646462785220996972SAM-B136M11012523043213835417FastSAM68M4040404040402. Memory usageDatasetMethodGPU Memory (MB)COCO 2017FastSAM2608COCO 2017SAM-H7060COCO 2017SAM-B46703. Zero-shot Transfer ExperimentsEdge DetectionTest on the BSDB500 dataset.methodyearODSOISAPR50HED2015.788.808.840.923SAM2023.768.786.794.928FastSAM2023.750.790.793.903Object ProposalsCOCOmethodAR10AR100AR1000AUCSAM-H E6415.545.667.732.1SAM-H E3218.549.562.533.7SAM-B E3211.439.659.127.3FastSAM15.747.363.732.2LVISbbox AR@1000methodallsmallmed.largeViTDet-H65.053.283.391.2zero-shot transfer methodsSAM-H E6452.136.675.188.2SAM-H E3250.333.176.289.8SAM-B E3245.029.368.780.6FastSAM57.144.377.185.3Instance Segmentation On COCO 2017methodAPAPSAPMAPLViTDet-H.510.320.543.689SAM.465.308.510.617FastSAM.379.239.434.5004. Performance VisualizationSeveral segmentation results:Natural ImagesText to Mask5.Downstream tasksThe results of several downstream tasks to show the effectiveness.Anomaly DetectionSalient Object DetectionBuilding ExtractingLicenseThe model is licensed under theApache 2.0 license.AcknowledgementSegment Anythingprovides the SA-1B dataset and the base codes.YOLOv8provides codes and pre-trained models.YOLACTprovides powerful instance segmentation method.Grounded-Segment-Anythingprovides a useful web demo template.ContributorsOur project wouldn't be possible without the contributions of these amazing people! Thank you all for making this project better.Citing FastSAMIf you find this project useful for your research, please consider citing the following BibTeX entry.@misc{zhao2023fast,
      title={Fast Segment Anything},
      author={Xu Zhao and Wenchao Ding and Yongqi An and Yinglong Du and Tao Yu and Min Li and Ming Tang and Jinqiao Wang},
      year={2023},
      eprint={2306.12156},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}