sdworld/Qwen3.8-Next-SDnvfp2

Qwen3.8-Next SDnvfp2

A quantised build of Qwen3.8-Flash-Next (180B-parameter MoE, 262k context) that runs on one DGX Spark (GB10, 128 GB unified memory) with sglang: routed experts at 2 bits per weight (read by a dedicated Triton kernel), NVFP4 dense layers and PLE table, NEXTN speculative decoding with a reduced draft vocabulary. Text-only as published (the vision tower is included but unmeasured).

Serving code, launcher and benchmarks: GitHub repository Qwen3.8-Next-SDnvfp2 (this checkpoint does not load on stock sglang or vLLM; it needs the patched files and run.sh from there).

Contents

• model-*.safetensors (206 shards, ~98 GB): expert tensors carry 2-bit codes, 4 per byte, under the NVFP4 tensor names and scale hierarchy; everything else is as in the NVFP4 source.
• dense_codes/ (1.8 GB): NVFP4 codes for the dense GDN / attention / lm_head projections, loaded at start-up (SGLANG_DENSE_NVFP4_CODES_DIR).
• tokenizer, chat template, generation config, config.json + hf_quant_config.json.

Measured on one DGX Spark (2026-09-06)

	SDnvfp2	MiaAI vLLM NVFP4 (same day, same probes)
decode 1 / 2 / 4 streams, short prose, tok/s per stream	45.8 / 33.5 / 23.5	31.3 / 27.5 / 22.6
decode at ~4k context, 1 / 2 / 4 streams	48.3 / 32.1 / 23.1	29.9 / 25.3 / 20.3
time to first token, short / 4k	0.12 s / 1.3 s	0.32 s / 2.3 s
GSM8K-300 greedy	97.0 %	99.3 %
tool-eval-bench (88 scenarios)	148 / 176	146 / 176
on disk / on GPU while serving	100 GB / 83 GB	99 GB / 74 GB + 27 GB mmap

Run

hf download sdworld/Qwen3.8-Next-SDnvfp2 --local-dir ./Qwen3.8-Next-SDnvfp2
git clone <GitHub Qwen3.8-Next-SDnvfp2> && cd Qwen3.8-Next-SDnvfp2
CKPT=/path/to/Qwen3.8-Next-SDnvfp2 ./run.sh     # OpenAI-compatible API on :8934, model name "sdnvfp2"

License

Derived from Qwen3.8-Flash-Next via the RadixArk NVFP4 checkpoint. The weights are distributed under the Qwen Community License 1.0 (LICENSE in this repo, copied from the base model): derivatives may be redistributed with this notice; no commercial model-as-a-service / AI-assistant use without a separate license from Qwen; naming requirement above 100M monthly users or US$20M monthly revenue. Serving code provenance is in the GitHub repository's NOTICE.