WaveCut/LingBot-Video-MoE-30B-A3B-SDNQ-uint4-static

LingBot Video MoE 30B-A3B SDNQ UINT4 Static

This is a complete, loadable derivative of robbyant/lingbot-video-moe-30b-a3b with the diffusion transformer stored using static SDNQ UINT4 weights. It is tied to source model revision f2e538f64afe00cc4ae674db2aeb52e2945edfd5, LingBot Video code a2bb04b78edd848500dc27a26e035a95442ae186, and SDNQ d841c383ff7be38728d4df829e17af4f15d4fd66 (v0.2.1-17-gd841c38).

Text encoder, tokenizer/processor, scheduler, and VAE remain at their upstream precision. Convolutions and embeddings are not quantized. The recipe is uint4-static-transformer-only-with-3d-expert-adapter: weights_dtype=uint4, auto group size (group_size=0), no dynamic quantization, SVD, Hadamard transform, convolution quantization, or embedding quantization.

Coverage

Coverage is calculated from the original parameter inventory, not from model-file sizes.

Component	Total logical params	Quantized params	Original parameter bytes covered	Packed storage	Packed grouped experts
transformer	30,084,506,176	99.8268%	99.6550%	17.42 GiB	28,991,029,248 params / 16.88 GiB stored
refiner	30,084,506,176	99.8268%	99.6550%	17.42 GiB	28,991,029,248 params / 16.88 GiB stored

Both the base transformer's and refiner's raw 3-D w1/w2/w3 expert tensors are packed. They are not excluded from the reported coverage. Exact module-level coverage and unquantized tensors are in quantization_manifest.json and benchmark/coverage.

Reproducible base benchmark

All five pairs use identical prompts, negative prompt, seeds 4201-4205, scheduler inputs, 832x480 dimensions, 73 frames, 24 fps, 40 steps, guidance 3.0, shift 3.0, batch_cfg=False, and null_cond_clone_zero=False. Resources were sampled every 250 ms from /proc, psutil, and nvidia-smi.

Variant	Load (s)	Cold generation (s)	Hot mean (s)	Peak VRAM (MiB)	Peak Torch allocated (MiB)	Process RSS (GiB)	System RAM used (GiB)
Original BF16	50.29	140.90	139.63	126274	112470	3.00	86.52
SDNQ UINT4	43.81	159.77	158.62	42504	36068	3.08	129.39

Observed base peak-VRAM reduction: 66.34%. Timing and memory are measurements on the environment recorded in benchmark/environment, not universal performance claims.

Frame-aligned aggregate quality across the five pairs: MAE 0.160701, RMSE 0.227453, PSNR 13.561 dB, SSIM 0.573403, LPIPS-Alex 0.484469.

The complete contact sheets and side-by-side MP4s are under assets/comparison/base. Quantized sample MP4s are under samples/base. Raw per-prompt CSV/JSONL, resource samples, commands, ffprobe records, output sizes, and SHA-256 values are under benchmark.

Refiner A/B

The refiner pair used the same 502b10f841d96aa101e69421b20083aeb60427c054b42fb0db29c5a6e70824cf initial latent generated from the same base MP4. Both outputs are 1920x1088, 73 frames at 24 fps, with 8 refiner steps and upstream-default null_cond_clone_zero=True.

Variant	Load (s)	Generation (s)	Peak VRAM (MiB)	Process RSS (GiB)	System RAM used (GiB)
Original BF16	139.01	478.60	179178	4.26	107.63
SDNQ UINT4	105.56	460.47	98232	3.85	105.73

Frame-aligned refiner quality: MAE 0.014650, RMSE 0.034053, PSNR 29.357 dB, SSIM 0.954816, LPIPS-Alex 0.063118.

Visual inspection found the same strong pink/red color clipping and cyan/magenta speckling in both the original-BF16 and SDNQ refiner outputs. Their close frame metrics therefore demonstrate pairwise similarity, not natural-color reconstruction quality; treat this as a shared refiner quality failure in this recorded sample.

Load the refiner by passing transformer_subfolder="refiner" to load_pipeline.

Installation and load

Use the exact pinned dependencies shipped with the repository:

git clone https://huggingface.co/WaveCut/LingBot-Video-MoE-30B-A3B-SDNQ-uint4-static
cd LingBot-Video-MoE-30B-A3B-SDNQ-uint4-static
python -m pip install -r runtime-requirements.txt

The tested MoE backend also needs the pinned SGLang userspace packages without replacing Torch:

python -m pip install --no-deps sglang==0.5.13.post1 apache-tvm-ffi==0.1.9 tilelang==0.1.8 triton==3.6.0

The repository includes the runtime adapter; no unmerged LingBot branch or local hidden file is needed:

import sys
from huggingface_hub import snapshot_download

root = snapshot_download("WaveCut/LingBot-Video-MoE-30B-A3B-SDNQ-uint4-static")
sys.path.insert(0, root)

from lingbot_sdnq_runtime import load_pipeline

pipe = load_pipeline(root, device="cuda")
# For the MoE refiner: load_pipeline(root, transformer_subfolder="refiner", device="cuda")

See prompts.json for the exact A/B inputs and benchmark/summary.json for portable metrics. Recorded consumer/offload smoke artifacts: benchmark/smokes/moe-refiner-sdnq-standard.json, benchmark/smokes/moe-refiner-sdnq-standard.mp4, benchmark/smokes/moe-sdnq-model.json, benchmark/smokes/moe-sdnq-model.mp4, benchmark/smokes/moe-sdnq-sequential.json, benchmark/smokes/moe-sdnq-sequential.mp4, benchmark/smokes/moe-sdnq-standard.json, benchmark/smokes/moe-sdnq-standard.mp4.

Runtime behavior and limitations

• Generic SDNQ Linear layers use eager BF16 dequantization followed by F.linear in the tested Torch 2.8/CUDA 12.8 environment because the current SDNQ Triton quantized-matmul path is incompatible there.
• Packed MoE experts are dequantized for each expert call and executed by the pinned SGLang Triton fused-MoE path. The adapter does not keep a persistent BF16 expert-weight cache.
• Static UINT4 materially changes generated pixels. Inspect the published matrices and per-prompt metrics before choosing this derivative for quality-sensitive work.
• The MoE SDNQ factory prompt is a severe framing/adherence regression: the valid 832x480 MP4 contains a smaller portrait-like factory view centered on a white canvas (MAE 0.338658). Treat that sample as a quality failure, not as a successful match to the original.
• Peak residency and speed depend strongly on resolution, frame count, attention backend, offload mode, and GPU. The numbers above describe the exact recorded B200 run only.
• The Apache-2.0 upstream license is retained. Users remain responsible for evaluating generated content for their application.

Evidence map

• quantization_manifest.json: recipe, revisions, per-component and expert coverage.
• prompts.json: exact structured prompts, negative prompt, seeds, and generation settings.
• benchmark/summary.json: portable aggregate benchmark record.
• benchmark/base: unmodified original and SDNQ raw metrics and resource samples.
• benchmark/comparison: frame-aligned MAE/RMSE/PSNR/SSIM/LPIPS records.
• SHA256SUMS: hashes for all published files, including model shards.