Qwen3.8-Flash-Next-NVFP4

GSM8K unchanged at 97.27 and AIME26 majority@8 unchanged at 100, while a decode step at low concurrency reads 38% fewer weight bytes — 6.13 GB/token against the base's 9.95 GB.

Designed for single-device DGX Spark serving, where the PLE n-gram tables stream from SSD and the dense path is what a decode step spends its bandwidth on. DGX Spark inference results coming soon; the numbers on this card were measured on 2xB200.

Derived from RadixArk/Qwen3.8-Flash-Next-NVFP4, which quantizes the routed MoE experts to NVFP4 and leaves everything else in BF16. The dense path it left in BF16 — attention, the gated-delta-net projections, the shared experts — is what a low-concurrency decode step spends most of its bandwidth on.

This checkpoint takes that dense path to FP8 W8A8 (static per-tensor, with calibrated activation scales) and the LM head to NVFP4 W4A16. Routed experts and the PLE n-gram tables are byte-identical to the base.

301 tensors are requantized; 1,562 unchanged tensors are byte-identical to the base.

What changed

component	base	this checkpoint
routed MoE experts (48 x 512)	NVFP4 W4A4	unchanged, byte-identical
PLE n-gram embeddings	FP8 E4M3	unchanged, byte-identical
GDN in_proj_qkv / in_proj_z / out_proj (36 layers)	BF16	FP8 W8A8
QSA q/k/v/o_proj (12 layers)	BF16	FP8 W8A8
shared-expert gate/up/down_proj (48 layers)	BF16	FP8 W8A8
lm_head	BF16	NVFP4 W4A16
norms, routers, hyper-connections, in_proj_ba, conv1d, MTP, vision	BF16	unchanged

Checkpoint size: 131.4 GB (base 135.2 GB).

Activation scales were calibrated by max over MoE-block input activations captured from live serving; the LM head is weight-only and needs no activation scale.

Decode weight traffic

Bytes of weights read per generated token, computed from the real tensor shapes (top-10 of 512 routed experts active):

	base	this checkpoint
batch 1 / low concurrency	9.95 GB/token	6.13 GB/token (-38.4%)

At low concurrency the dense path dominates what a decode step reads, and the dense path is exactly what this checkpoint quantizes — the routed experts, already NVFP4 in the base, are left alone.

Evaluation

Base column is the number published on the base model's card. This checkpoint was measured on SGLang, TP2, 2xB200, using the base card's protocols verbatim — GSM8K full 1319 at t0.6 / top-p 0.95 / max 8192, AIME26 30 problems x 8 at t1.0 / max_tokens 130000, both with the NeMo-Skills boxed prompt.

eval	base (published)	this checkpoint
GSM8K	97.27	97.27
AIME26 pass@1	98.75	97.50
AIME26 majority@8	100	100
AIME26 stop rate	99.17	99.17

Usage

The architecture (Qwen4ExpForConditionalGeneration) is not yet in mainline SGLang. Build from PR #36497, which is sgl-project's own branch — no third-party patch is needed.

python -m sglang.launch_server \
  --model-path senfu/Qwen3.8-Flash-Next-NVFP4 \
  --tp 2 \
  --quantization modelopt_mixed \
  --fp4-gemm-backend flashinfer_cutlass \
  --moe-runner-backend flashinfer_trtllm \
  --page-size 64 \
  --mamba-scheduler-strategy extra_buffer \
  --mamba-track-interval 64 \
  --chunked-prefill-size 4096 \
  --max-running-requests 36 \
  --context-length 262144 \
  --mem-fraction-static 0.80 \
  --allow-auto-truncate \
  --port 30000

--quantization modelopt_mixed is the one flag that differs from the base's recipe: this checkpoint declares ModelOpt MIXED_PRECISION, with a per-module quantized_layers map covering 396 FP8 modules, 48 NVFP4 MoE layers and 1 W4A16 head.

Keep --mamba-scheduler-strategy extra_buffer: it is what makes --page-size 64 legal for the GDN cache, and page 64 is required by compressed-QSA addressing.

License

Qwen Community License 1.0, inherited from the base model. Derivative works are permitted; the license and copyright notice must travel with them. Operating a Model-as-a-Service or AI Work Assistant business on it requires a separate agreement with Qwen.

Credits

Qwen for the base model. RadixArk for the NVFP4 expert quantization this builds on. SGLang for the serving stack.