ussoewwin/SAM3.1-HSWQ-ConvRot-INT8

SAM 3.1 Multiplex (HSWQ ConvRot INT8)

High-fidelity HSWQ ConvRot INT8 (Hybrid-Sensitivity-Weighted-Quantization) quantized weights for the SAM 3.1 Multiplex (Segment Anything Model 3.1) architecture.

• Technical Details & Quantization Framework: Hybrid-Sensitivity-Weighted-Quantization (HSWQ)
• ComfyUI Loader for ConvRot INT8: ComfyUI-HSWQ-Loader-and-Tools

---

🌟 Model Overview

This repository provides production-grade HSWQ ConvRot INT8 quantized weights for the SAM 3.1 Multiplex foundation model. By combining orthogonal Hadamard rotation with HSWQ sensitivity profiling, activation outlier spikes are suppressed across projection layers while critical memory attention, boundary refinement, and mask decoder modules are protected in higher precision (FP16/BF16):

• Architecture: SAM 3.1 Multiplex (Vision Backbone + FPN Neck + Decoupled Memory Attention + 16-Object Multiplex Mask Decoder)
• Base Checkpoint: Upstream SAM 3.1 Multiplex foundation weights from Comfy-Org/sam3.1 / facebook/sam3.1
• Native ComfyUI Support: Features native int8_tensorwise and comfy_quant metadata stamps, loading seamlessly with ComfyUI's standard UNetLoader / DiffusionModelLoader or HSWQ SAM3 Loader without requiring external manual dequantization workflows.

---

📦 Model Details

Filename	Base Architecture	Precision / Format	File Size	Memory Footprint vs FP16	Native ComfyUI Support	Status
sam3.1_multiplex_hswq_convrot_int8.safetensors	SAM 3.1 Multiplex	HSWQ ConvRot INT8 (int8_tensorwise)	~1.74 GB (1,738 MB)	~0.4% vs the 1.66 GB FP16 source (41 of 221 runtime-live layers quantized)	✅ Supported natively (UNetLoader)	Available

Quantization Breakdown

• HSWQ ConvRot INT8 Linear Layers: Bulk transformer and feedforward projection layers quantized with orthogonal Sylvester Hadamard rotation (groupsize=256) and per-channel scaling.
• Sensitivity-Protected High-Precision Tensors: Critical cross-attention heads, memory encoder/decoder boundaries, LayerNorm, Embeddings, Biases, and 4D Conv2d weights preserved in full float precision under HSWQ sensitivity analysis to prevent mask jitter, boundary collapse, and video tracking loss.

---

🛠️ Key Highlights

• Hadamard Orthogonal Rotation (ConvRot): Pre-rotates weight matrices ($W_{rot} = W \cdot H^T$) along the input channel axis using normalized Sylvester Hadamard blocks, eliminating channel-wise outlier spikes and preventing mask degradation.
• Hybrid Sensitivity Protection (HSWQ): Specifically designed to prevent precision loss in multi-object video segmentation, guaranteeing sharp mask boundaries and long-term tracking stability across all 16 multiplex object slots.
• Optimized Storage & Memory Footprint: 1.74 GB, near the 1.66 GB FP16 source, because only 41 of the checkpoint's runtime-live layers are both 256-wide and unprotected. The gain here is the INT8 execution path (Tensor Core INT8 matmul), not the disk size - the same trade the guide documents. Quality is preserved: measured tracker mask IoU 0.9978 against FP16.

---

🚀 Usage in ComfyUI

Model Placement

Download sam3.1_multiplex_hswq_convrot_int8.safetensors and place it in your ComfyUI models directory:

ComfyUI/models/unet/sam3.1_multiplex_hswq_convrot_int8.safetensors

Loading

1. Load the model using the standard UNetLoader (or Load Diffusion Model) node in ComfyUI, or use HSWQ SAM3 Loader from ComfyUI-HSWQ-Loader-and-Tools.
2. Connect the loaded model output directly into your SAM3 image / video segmentation workflow nodes.
3. ComfyUI automatically detects the comfy_quant header metadata and executes optimized INT8 Tensor Core operations natively.

---

📜 Credits & License

• Base Architecture & Model: Comfy-Org/sam3.1 / facebook/sam3.1 (Meta SAM / Segment Anything Team)
• Quantization Framework: HSWQ / ConvRot INT8 Pipeline
• License: SAM License (Meta SAM License)