MedMLX/sam3-interactive

SAM 3 interactive head for MLX

Native float32 MLX weights for the point, box and mask prompt head of SAM 3, used by sam3-mlx for SAM 1-style interactive segmentation. The tensors are the prompt encoder, mask decoder and no_mem_embed of the tracker in Meta's sam3.pt (facebook/sam3 at 3c879f39826c281e95690f02c7821c4de09afae7), renamed to inst_interactive_predictor.model.* as upstream's image builder does, with convolution kernels in MLX's channels-last layout. They have not been retrained.

This file complements the image weights in mlx-community/sam3-image, which do not contain the interactive head. sam3-mlx downloads it on first use:

import numpy as np
from sam3_mlx import build_sam3_image_model
from sam3_mlx.model.sam3_image_processor import Sam3Processor

model = build_sam3_image_model(enable_inst_interactivity=True)
state = Sam3Processor(model).set_image("image.png")
masks, scores, _ = model.predict_inst(
    state, point_coords=np.array([[320, 240]]), point_labels=np.array([1])
)

Validation and use

Every converted value equals the corresponding tensor in sam3.pt. On three medical images with point, box and combined prompts, the MLX head and upstream PyTorch SAM 3 (commit 2814fa6) given the same image features produced the same predicted IoU scores, low-resolution logits within 3e-4 and masks differing by at most one pixel. This is a conversion check, not a clinical validation. SAM 3's accuracy on medical images is not established. Research and education use only; outputs require human review.

License

The weights are a derivative of SAM Materials and are distributed under the SAM License, reproduced in LICENSE. Publications that use them must acknowledge SAM. Not affiliated with Meta.

Citation

If you use MedMLX, please cite the MedMLX paper:

@article{sandoval2026medmlx,
  title   = {MedMLX: On-Device Medical Imaging AI for Apple Silicon},
  author  = {Sandoval, A. and Sandoval, J. and Saleem, A.},
  journal = {medRxiv},
  year    = {2026},
  doi     = {10.64898/2026.10.28.26366762}
}