igorls/LingBot-World-V2-14B-Causal-Fast-NVFP4

LingBot-World V2 14B Causal Fast — Hybrid NVFP4

This is an unofficial, transformer-only hybrid NVFP4 quantization of robbyant/lingbot-world-v2-14b-causal-fast-diffusers, created for single-GPU live-inference experiments on NVIDIA Blackwell.

It is not a standalone Diffusers pipeline. Download the base repository for the tokenizer, text encoder, VAE, scheduler, and pipeline metadata, then use this repository as the replacement transformer path.

License

The model weights are distributed under CC BY-NC-SA 4.0, matching the exact base model. They are for non-commercial use only. See LICENSE and ATTRIBUTION.md. The SGLang compatibility patch is separately covered by Apache-2.0; see THIRD_PARTY_NOTICES.md.

What is quantized

This is deliberately labelled hybrid NVFP4, not full-model NVFP4:

• 340 transformer weights are packed NVFP4.
• 430 weights remain at higher precision for runtime compatibility.
• 1,020 ModelOpt scale tensors are included.
• The eight-shard transformer export is 28,310,448,928 bytes.
• The LingBot camera-conditioning linears and unsupported boundary layers are retained at higher precision.

config.json contains the complete ModelOpt quantization configuration and ignored-layer list. build.json records the export counts and versions.

Tested environment

Component	Tested value
GPU	NVIDIA RTX PRO 6000 Blackwell Workstation Edition (SM120, 96 GB)
GPU power limit	498.6 W
OS	WSL2 / Ubuntu
Python	3.12.3
PyTorch	2.11.0
SGLang	0.5.15 plus the included compatibility patch
ModelOpt	0.45.0
FlashAttention	4.0.0b21

Stock SGLang 0.5.15 will not load this checkpoint correctly without patches/sglang-0.5.15-lingbot-nvfp4-sm120.patch. The patch fixes ModelOpt exclusion-prefix matching and propagates quantization prefixes into causal Wan cross-attention and FFN layers.

Download and serve

Download the exact tested base revision and this transformer:

hf download robbyant/lingbot-world-v2-14b-causal-fast-diffusers \
  --revision 59cccf49f2d2dd27418ae7a04b82b10868d455c2 \
  --local-dir ./lingbot-world-v2-base

hf download igorls/LingBot-World-V2-14B-Causal-Fast-NVFP4 \
  --local-dir ./lingbot-world-v2-nvfp4

Apply the pinned compatibility patch to the SGLang environment:

SGLANG_PYTHON=/path/to/sglang-env/bin/python \
  bash ./lingbot-world-v2-nvfp4/apply_sglang_patch.sh

Serve with the base pipeline and this repository as --transformer-path:

sglang serve \
  --model-path ./lingbot-world-v2-base \
  --model-id lingbot-world-fast-diffusers \
  --transformer-path ./lingbot-world-v2-nvfp4 \
  --backend sglang \
  --pipeline-class-name LingBotWorldCausalDMDPipeline \
  --num-gpus 1 \
  --ulysses-degree 1 \
  --dit-cpu-offload false \
  --text-encoder-cpu-offload false \
  --vae-cpu-offload false \
  --attention-backend fa4 \
  --enable-torch-compile false \
  --host 127.0.0.1 \
  --port 30000

The tested SM120 configuration also used SGLANG_DIFFUSION_FLASHINFER_FP4_GEMM_BACKEND=cutlass. CUDA library/include paths must match the local PyTorch/SGLang installation.

Measured performance

Measurements below are from one RTX PRO 6000 limited to 498.6 W, four inference steps, guidance scale 1.0, sink 9, KV window 18, WebP transport, and no CPU offload.

Workload	Throughput	Median chunk	First frame	Peak VRAM
384×192 warm	21.158 unpaced E2E FPS	568 ms	1.969 s	52,254 MiB
Character portrait 240×448	14.098 E2E / 14.210 generated FPS	856 ms	2.446 s	55,941 MiB

The 384×192 result was 23.0% faster than the measured FP8 baseline of 17.196 E2E FPS. A clean SM120 CUTLASS kernel cache incurred a one-time 99.177-second first request; subsequent warm requests started in about two seconds.

These are low-resolution engineering throughput measurements, not advertised native-quality results. In the 240×448 character test, broad identity and the red/blue hair split remained recognizable, but faces and eyes were visibly smeared and the raised hand had fused fingers. Use a native resolution and perform independent quality evaluation for production decisions.

Reproducibility and integrity

• Export: NVIDIA ModelOpt 0.45.0 unified Hugging Face NVFP4 format.
• Calibration: four causal examples; 1,710 quantizers calibrated in 30.48 s.
• Manifest: SHA256SUMS.
• Export metadata: build.json.
• Runtime patch: patches/sglang-0.5.15-lingbot-nvfp4-sm120.patch.

This repository contains no UI code or character reference images.

Citation

Please cite the original LingBot-World V2 work and respect its attribution requirements. See ATTRIBUTION.md for the exact source model, revision, paper citation, and modification notice.