jarrelscy/DeepSeek-V4-Pro-0813-NVFP4

DeepSeek-V4-Pro-0813 — NVFP4 (routed experts)

NVFP4 quantization of the routed experts of deepseek-ai/DeepSeek-V4-Pro-0813, in the ModelOpt MIXED_PRECISION layout (the format nvidia's NVFP4 checkpoints and vLLM's ModelOpt path consume).

What was quantized

All 61 layers' ffn.experts → NVFP4 (group-16 e4m3 block scale + per-tensor f32 global scale). Excluded and left in their native precision (matches nvidia's ignore list): *.attn.*, *.ffn.shared_experts.*, head, mtp.*.

Per expert linear (w1 gate, w3 up, w2 down):

.weight          uint8         [out, in/2]    e2m1, low nibble first
.weight_scale    float8_e4m3   [out, in/16]   group-16 block scale
.weight_scale_2  float32       []             per-tensor global scale

dequant = e2m1_val * weight_scale * weight_scale_2.

How it was made

The 0813 release already ships experts as MXFP4 (e2m1 + UE8M0 group-32 scales); there is no BF16 master on the Hub. This checkpoint transcodes those experts MXFP4 → NVFP4 (dequant to bf16, requantize to group-16 e4m3). Weight-only and data-free. Median requant error vs the MXFP4 source ≈ 4.8% (4-bit regrid noise); note this is a double quantization (MXFP4→NVFP4), so error is measured against the already-4-bit source, not full-precision weights.

Provenance

• Base: deepseek-ai/DeepSeek-V4-Pro-0813 (MIT).
• Every other public DeepSeek-V4-Pro-NVFP4 is built on the older deepseek-ai/DeepSeek-V4-Pro base; this one is on 0813.