realrebelai/FastH3_GGUFs

FastH3 4-step (VSA, DataFree) — GGUF

Rebels Q4

Kijais INT8

GGUF quantizations of FastVideo/FastVideo-FastH3-4-step-Preview-v1-VSA-DataFree — MiniMax-H3 distilled to 4 steps with DMD2 and trained with Video Sparse Attention.

The upstream release is 65 GB of bf16 shards in FastVideo's own layout. These are converted to ComfyUI's MiniMax-H3 layout and quantized, so a 35B video model fits on consumer hardware.

Original licensing terms carry over from MiniMax-H3.

---

⚠️ Read this first

These will not work on stock ComfyUI. FastH3's attention is trained with VSA — every block has a to_gate_compress layer that gates a coarse attention branch. Stock ComfyUI has no code for it, discards those 50 tensors on load ("unet unexpected"), runs dense attention instead, and outputs uniform noise at every quant tier. Nothing about the file is wrong when that happens; the attention path is missing.

Running these needs three unmerged pieces, all from Kijai:

1. ComfyUI's vsa branch — the H3 model code that keeps the gate weights
2. comfy-kitchen's sol_attn branch, compiled — the sparse attention kernels
3. A test node that turns VSA on at runtime

Full walkthrough below. Budget 30-60 minutes the first time; most of it is the CUDA build.

---

Files

Quant	Size	Notes
Q8_0	~37 GB	Reference
Q6_K	~29 GB
Q5_K_M	~25 GB
Q4_K_M	~21 GB
Q4_K_S	~21 GB	Quality minimum, Q3 and Q2 are unusable

Sizes are large because this is a 35B model. Pick by system RAM, not VRAM — the weights stream from RAM, and a tier that doesn't fit will page badly.

---

Setup — step by step

Paths below use <COMFY> for your ComfyUI portable root (the folder containing ComfyUI\ and python_embeded\) and <PY> for <COMFY>\python_embeded\python.exe.

1. ComfyUI on the vsa branch

cd /d <COMFY>\ComfyUI
git fetch https://github.com/kijai/ComfyUI.git vsa:vsa
git checkout vsa

Verify the gate detection exists:

findstr /n "gate_compress" comfy\model_detection.py

You want a line setting dit_config["gate_compress"]. If nothing prints, the checkout didn't take.

Any ComfyUI update puts you back on master and breaks this. Re-check the branch (git branch --show-current) after updating. git checkout master returns to normal ComfyUI.

2. Build comfy-kitchen with sol_attn

This compiles CUDA kernels. You need Visual Studio 2022 with the C++ tools and the Windows 11 SDK, plus CUDA Toolkit 12.8 or newer (12.9 used here).

cd /d <COMFY>\comfy-kitchen
git fetch https://github.com/kijai/comfy-kitchen.git sol_attn:sol_attn
git checkout sol_attn
git submodule update --init --recursive

The submodule step matters — flash.h lives in one and the build fails without it.

Now open x64 Native Tools Command Prompt for VS 2022 from the Start menu. A normal command prompt will not work: it lacks rc.exe and mt.exe and linking fails.

set "PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9\bin;%PATH%"
set "CUDA_HOME=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9"
set "CUDA_PATH=C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.9"
set "CMAKE_GENERATOR=Ninja"
set "COMFY_CUDA_ARCHS=80-real;86-real"
cd /d <COMFY>\comfy-kitchen
rmdir /s /q build
<PY> -m pip install . --no-deps

Why each line:

• PATH first — setup.py reads nvcc from PATH. A stale CUDA 12.1 elsewhere on PATH fails the 12.8+ check even when 12.9 is installed.
• Ninja — the Visual Studio generator falls back to CUDA 12.1.targets and errors on a toolkit directory that no longer exists.
• COMFY_CUDA_ARCHS — the default list includes Blackwell, whose NVFP4 kernel fails to compile on 12.9. 80-real;86-real covers RTX 30-series; use 89-real for 40-series, 120-real for 50-series.
• rmdir build — CMake caches the previous failure otherwise.

Confirm:

<PY> -c "import comfy_kitchen as ck; print('sol_attn:', hasattr(ck,'sol_attn'))"

Must print True.

3. The Sol-Attn node

Download sol_attn_minimax_v5.py from the attachment on comfy-kitchen PR #117 and drop it in <COMFY>\ComfyUI\custom_nodes\.

4. Companion models

From Comfy-Org/MiniMax-H3:

File	Folder
qwen3vl-32B-MiniMax-H3 text encoder (GGUF or safetensors)	models/text_encoders
minimax_h3_video_vae_fp16.safetensors	models/vae
minimax_h3_audio_vae_fp32.safetensors	models/vae

Both VAEs are required — the audio one is what people forget.

5. Workflow

Standard MiniMax-H3 graph, with one addition: Patch Sol-Attn (MiniMax) between the Unet Loader (GGUF) and the sampler (MODEL in, MODEL out).

Setting	Value
selection	VSA (FastVideo)
vsa_keep_percent	10 — what the FastH3-VSA checkpoints were trained at (90% sparsity)
start_percent	0
end_percent	1
min_tokens	0 to force it on; raise later
sink_conditioning	exact_kv_and_rows

start 0 / end 1 is not optional on a 4-step model. Anything outside that window runs dense, and dense attention on VSA-trained weights is noise.

Sampler: 4 steps, CFG 1.0 (distilled — guidance is baked in).

6. Check the console

A correct load shows:

[sol_attn] chunked qkv producer on 50 blocks

And must not show either of these:

unet unexpected: ['blocks.0.attn.to_gate_compress.weight', ...]
[sol_attn] VSA: checkpoint has no to_gate_compress weights

Both of those mean the gate weights were discarded at load — almost always because ComfyUI is on master instead of vsa.

---

Conversion notes

The upstream checkpoint is FastVideo/diffusers layout; ComfyUI's MiniMax-H3 expects a different one. The conversion is structural, not just renaming:

• attn.to_q / to_k / to_v → concatenated into attn.qkv_proj (ComfyUI splits it back with .split(heads * head_dim, dim=-1), so the order is q, k, v)
• attn.to_out.0 → attn.out_proj
• attn.norm_q / norm_k → attn.q_norm / k_norm
• ff.net.0.proj / ff.net.2 → mlp.fc1 / mlp.fc2
• transformer_blocks. → blocks., token_refiner.refiner_blocks. → token_refiner.blocks.
• top-level: audio_proj_in → audio_patch_proj, proj_in → video_patch_proj, context_embedder → condition_proj, norm_out.* → final_layer.*, time_embedder.linear_1/2 → time_embedder.proj_in/proj_out
• rope.inv_freq synthesized — it's a computed buffer FastVideo doesn't ship: 1/(theta ** (arange(0, 2*16, 2) / 32)), theta 10000
• to_gate_compress kept — required for VSA

Kept at high precision through quantization (llama-quantize crushes these otherwise, and the timestep embedder in particular corrupts every step of a 4-step schedule):

audio_patch_proj, video_patch_proj, condition_proj,
final_layer.*, time_embedder.*, token_refiner, rope.inv_freq

---

Credits

• FastVideo (hao-ai-lab) — FastH3 distillation and VSA
• MiniMax — MiniMax-H3
• Kijai — the ComfyUI VSA branch, Sol-Attn kernels, and the test node
• city96 — ComfyUI-GGUF

Quantized by RealRebelAI · GitHub · X