SAM 3.1 Multiplex (ConvRot INT8)
High-fidelity Native ConvRot INT8 quantized weights for the SAM 3.1 Multiplex (Segment Anything Model 3.1) architecture.
🌟 Model Overview
This repository provides Native ConvRot INT8 quantized weights for the SAM 3.1 Multiplex foundation model. By applying orthogonal Hadamard rotation before per-channel INT8 quantization, activation outliers are evenly distributed across matrix dimensions, drastically suppressing quantization noise while halving VRAM and storage footprint:
- Architecture: SAM 3.1 Multiplex (Vision Backbone + FPN Neck + Decoupled Memory Attention + 16-Object Multiplex Mask Decoder)
- Base Checkpoint: Comfy-Org/sam3.1 (
sam3.1_multiplex_fp16.safetensors) - Native ComfyUI Support: Features native
int8_tensorwiseandcomfy_quantmetadata stamps, loading seamlessly with ComfyUI's standardUNetLoader/DiffusionModelLoaderwithout requiring external custom quantization nodes.
📦 Model Details & Comparison
| Filename | Base Architecture | Precision / Format | File Size | Memory Footprint | Native ComfyUI Support |
|---|---|---|---|---|---|
sam3.1_multiplex_convrot_int8.safetensors |
SAM 3.1 Multiplex | ConvRot INT8 (int8_tensorwise) |
~0.84 GB (899 MB) | ~50% VRAM Reduction | ✅ Supported natively (UNetLoader) |
sam3.1_multiplex_fp16.safetensors (Reference) |
SAM 3.1 Multiplex | FP16 | ~1.63 GB (1,746 MB) | Baseline | ✅ Supported natively |
Quantization Breakdown
- ConvRot INT8 Layers: 647 Linear projection layers (Hadamard rotation
groupsize=256+ per-out-channel scale) - Plain INT8 Fallback Layers: 4 layers (non-power-of-4 channel dimensions safely quantized with row-wise scaling)
- Preserved High-Precision Tensors: 1,183 tensors (LayerNorm, Embedding, Biases, and 4D Conv2d weights preserved in full float precision for boundary and spatial accuracy)
🛠️ Key Highlights
- Hadamard Orthogonal Rotation (ConvRot): Pre-rotates weight matrices ($W_{rot} = W \cdot H^T$) along the input channel axis using normalized Sylvester Hadamard blocks, eliminating channel-wise outlier spikes and preventing mask degradation.
- Full Multiplex Support: Preserves the complete 16-object simultaneous tracking, interactive segmentation, and decoupled cross-attention capabilities of the SAM 3.1 architecture.
- 50% Disk & VRAM Savings: Reduces weight storage from 1.63 GB down to 0.84 GB, enabling lightweight deployment on consumer hardware and multi-model video segmentation workflows.
🚀 Usage in ComfyUI
Model Placement
Download sam3.1_multiplex_convrot_int8.safetensors and place it in your ComfyUI models directory:
ComfyUI/models/unet/sam3.1_multiplex_convrot_int8.safetensors
Loading
- Load the model using the standard
UNetLoader(orLoad Diffusion Model) node in ComfyUI. - Connect the loaded model output directly into your SAM3 image / video segmentation workflow nodes.
- ComfyUI automatically detects the
comfy_quantheader metadata and executes optimized INT8 Tensor Core operations natively.
📜 Credits & License
- Base Architecture & Model: Comfy-Org/sam3.1 / Segment Anything Team
- Quantization Framework: HSWQ / ConvRot INT8 Pipeline
- License: Apache-2.0