Qwen3.8-Flash-Next-NVFP4-W4A16

Built with Qwen.

Weight-only NVFP4 quantization of the routed MoE experts. Everything else — attention, linear attention, shared experts, routers, hyper-connections, PLE n-gram tables, MTP, vision tower, lm_head — stays in BF16.

What is quantized

Scheme	NVFP4, 4-bit weights, group size 16, BF16 activations
Scope	routed experts of the 48 text MoE layers (120.8B of 125B params)
Tool	NVIDIA TensorRT Model Optimizer 0.46.0
Calibration	data-free (algorithm: max)
Size	169 GB (from 336 GB BF16)

How this differs from other NVFP4 builds

Existing NVFP4 checkpoints of this model are W4A4 — activations are also 4-bit — and are validated for SGLang on datacenter Blackwell (GB300/B300). This one keeps activations at 16-bit, which costs memory but avoids activation quantization error.

Status

vLLM support for qwen4_exp is not merged yet (vllm-project/vllm#53896). This checkpoint is published ahead of that. Serving has not been validated end to end.

The PLE n-gram tables are 97.7 GiB of the total and remain BF16; serving on a single 96 GB GPU requires host-RAM PLE offload (vllm-project/vllm#53899).