SAM3: Optimized for AMD ROCm
SAM3 (Segment Anything Model 3) is a promptable instance segmentation model that generates high-quality masks from text, box, or point prompts. This repository packages inference for instance segmentation using PyTorch, exported and validated for AMD ROCm so it runs efficiently on AMD GPUs and CPUs.
This is based on the implementation of SAM3 found here. This repository contains configurations and scripts optimized for AMD® ROCm™ platforms. You can use the sam3 AMD scripts to reproduce results or export with custom configurations. More details on model performance can be found here.
Task Overview
Task: Instance segmentation
Dataset: COCO val2017 (5,000 images, 80 categories)
Output metrics: Mask mAP@0.5:0.95, Mask mAP@0.5, Mask mAP@0.75, Mean Mask IoU, Mask Precision, Mask Recall, Mask F1
PyTorch note: CPU runs FP32 only; GPU supports FP32/FP16/BF16 via
torch.autocast(no INT8 — SAM3 is PyTorch-only, not ONNX/MIGraphX). NPU is not available since ONNX export is unsupported upstream. Hugging Face access is required — SAM3 weights are gated; you must accept the license at facebook/sam3 and exportHF_TOKEN.
AMD ROCm Optimization
This model export has been adapted and validated for AMD Instinct™ / Radeon™ GPUs running ROCm, as well as AMD CPUs. Key points:
- Validated backend: PyTorch (native ROCm HIP kernels via
torch.autocast), FP32/FP16/BF16 on GPU, FP32 on CPU. - No code changes required versus the upstream SAM3 implementation — only environment/runtime configuration differs.
- No NPU path — SAM3 ONNX export is unsupported upstream, and VitisAI requires an ONNX graph.
| Runtime | Precision | Backend | Hardware | Notes |
|---|---|---|---|---|
| PyTorch | FP32 | HIP (ROCm) | AMD CPU | — |
| PyTorch | FP32 / FP16 / BF16 | HIP (ROCm) | AMD Instinct™ / Radeon™ GPU | Via torch.autocast; no INT8 path |
Getting Started
For setup instructions, evaluation scripts, and custom configuration options, see the sam3 on GitHub.
Model Details
Model Type: Promptable instance segmentation (text, box, or point prompts)
Base Model: facebook/sam3 (gated)
Model Stats:
- Precision tested: FP32 (CPU); FP32, FP16, BF16 (GPU)
Accuracy Pipeline
Higher Mask mAP means the model's predicted masks agree more closely with the ground-truth segmentation annotations — 1.0 would be perfect mask overlap, 0.0 means no correct masks. Because SAM3 is a promptable segmentation model (not a detector), these metrics measure pure mask quality independent of detection accuracy. Since full-dataset inference is too slow for a complete 5,000-image COCO val2017 pass, evaluation uses three nested random subsets (N=100, 300, 500) drawn once with a fixed seed for reproducibility.
Metrics Explained
| Metric | Description |
|---|---|
| Mask mAP@0.5:0.95 | Primary COCO mask metric — mean AP averaged across IoU thresholds 0.5–0.95. The strictest, most holistic mask accuracy number; higher means predicted masks overlap ground truth tightly across a range of thresholds. |
| Mask mAP@0.5 | AP at a single, looser IoU threshold of 0.5 — a predicted mask only needs 50% overlap to count as correct, so this is typically higher than mAP@0.5:0.95 and reflects "did it segment the object" more than "how precisely." |
| Mask mAP@0.75 | AP at a stricter IoU threshold of 0.75 — the predicted mask must overlap ground truth by 75%, rewarding precise segmentation boundaries over merely finding the object. |
| Mean Mask IoU | Average intersection-over-union across all predicted masks — a direct geometric measure of how well each individual mask covers its ground-truth region, complementing the AP-based aggregate metrics. |
| Mask Precision | Of all masks the model produced, what fraction were correct — high precision means few false-positive masks (segments that don't correspond to real objects). |
| Mask Recall | Of all ground-truth objects, what fraction did the model successfully segment — high recall means few missed objects (false negatives). |
| Mask F1 | Harmonic mean of Mask Precision and Mask Recall — balances the two failure modes (false positives vs. false negatives) into a single number. |
Accuracy Results
Sampled Evaluation (COCO val2017) — nested subsets, seed=42:
| Device | Precision | N | Mask mAP@0.5:0.95 | Mask mAP@0.5 | Mean IoU | Mask Precision | Mask Recall | Mask F1 |
|---|---|---|---|---|---|---|---|---|
| CPU | FP32 | 100 | 0.6269 | 0.8982 | 0.7933 | 0.5895 | 0.6166 | 0.6027 |
| CPU | FP32 | 300 | 0.5991 | 0.8937 | 0.7878 | 0.5525 | 0.6026 | 0.5765 |
| CPU | FP32 | 500 | 0.5813 | 0.9045 | 0.7886 | 0.5390 | 0.5938 | 0.5651 |
| GPU | FP32 | 100 | 0.6269 | 0.8982 | 0.7933 | 0.5895 | 0.6166 | 0.6027 |
| GPU | FP32 | 300 | 0.5991 | 0.8937 | 0.7878 | 0.5525 | 0.6026 | 0.5765 |
| GPU | FP32 | 500 | 0.5813 | 0.9045 | 0.7886 | 0.5390 | 0.5938 | 0.5651 |
Dig Deeper
Want to explore the full evaluation scripts, config options, and other AMD-optimized model examples?
📂 View the full project on GitHub
The GitHub repository includes:
- Setup and prerequisites for ROCm environments
- Text/box/point prompt-based inference scripts
- COCO val2017 sampled evaluation pipeline with fixed-seed reproducibility
- Benchmarking and reproduction instructions for CPU and GPU