InsectSAM: Insect Segmentation and Monitoring

Overview

InsectSAM is an advanced machine learning model tailored for the https://diopsis.eu camera systems and https://www.arise-biodiversity.nl/, dedicated to Insect Biodiversity Detection and Monitoring in the Netherlands. Built on Meta AI's segment-anything model, InsectSAM is fine-tuned to be accurate at segmenting insects from complex backgrounds, enhancing the accuracy and efficiency of biodiversity monitoring efforts.

Purpose

This model has been meticulously trained to identify and segment insects against a variety of backgrounds that might otherwise confuse traditional algorithms. It is specifically designed to adapt to future changes in background environments, ensuring its long-term utility in the DIOPSIS / ARISE project.

Model Architecture

InsectSAM utilizes the advanced capabilities of the segment-anything architecture, enhanced by our custom training on an insect-centric dataset. The model is further refined by integrating with GroundingDINO, improving its ability to distinguish fine details and subtle variations in insect appearances.

Quick Start

Prerequisites

Python
Hugging Face Transformers
PyTorch

Usage

Install

!pip install --upgrade -q git+https://github.com/huggingface/transformers
!pip install torch

Load model directly via HF Transformers 🤗

from transformers import AutoProcessor, AutoModelForMaskGeneration

processor = AutoProcessor.from_pretrained("martintmv/InsectSAM")
model = AutoModelForMaskGeneration.from_pretrained("martintmv/InsectSAM")

Notebooks

Three Jupyter notebooks are provided to demonstrate the model's capabilities and its integration with GroundingDINO:

InsectSAM.ipynb: Covers the training process, from data preparation to model evaluation.
InsectSAM_GroundingDINO.ipynb: Demonstrates how InsectSAM is combined with GroundingDINO for enhanced segmentation performance.
Run_InsectSAM_Inference_Transformers.ipynb: Run InsectSAM using Transformers.

Check out the notebooks on RB-IBDM's GitHub page - https://github.com/martintmv-git/RB-IBDM/tree/main/InsectSAM