neroued/Qwen3.8-Flash-Next-NInfer

Qwen3.8-Flash-Next for NInfer

A mixed-precision NInfer v3 artifact of Qwen3.8-Flash-Next, combining the BF16 checkpoint with calibrated experts and MTP weights from NVIDIA's NVFP4 checkpoint. It contains Text, Vision, native MTP and an optimized full-vocabulary draft head.

Use it with NInfer for CLI generation or OpenAI-/Anthropic-compatible serving. The native architecture is Qwen4Exp: Gated DeltaNet, sparse attention, gated residual streams, MoE and PLE embeddings. The artifact carries its tokenizer, processor resources and the maintained Flash Next chat template. .ninfer files are consumed directly by NInfer.

Download

hf download neroued/Qwen3.8-Flash-Next-NInfer \
  --include 'qwen3_8_flash_next.ninfer*' \
  --local-dir models/Qwen3.8-Flash-Next-NInfer

The complete artifact is 101.00 GiB across four files:

qwen3_8_flash_next.ninfer
qwen3_8_flash_next.ninfer.part-0001
qwen3_8_flash_next.ninfer.part-0002
qwen3_8_flash_next.ninfer.part-0003

Keep all four in one directory and pass only qwen3_8_flash_next.ninfer to NInfer. These are storage parts; loading selects any required rank shards. File sizes and the artifact identity are recorded in artifact-manifest.json.

Representation

Weights	Stored representation
Main routed experts	Imported NVIDIA NVFP4, with calibrated scales
Large GDN projections; Main QSA query/gate and output	Q8 A16
MTP routed experts	Imported block-FP8
PLE table	NF4
Main head, shared experts, HC, Vision and other retained projections	BF16; source FP32 control fields remain FP32
Optimized MTP head	Additional full-vocabulary Q4 A16; Main head stays BF16

Imported NVFP4 codes/scales are retained. Additional quantization is performed by the qwen3_8_flash_next recipe. The optimized head does not crop the vocabulary. Select it with --spec mtp --draft-tokens 3 --proposal-head optimized.

Run on Spark

Build the current NInfer checkout using its build instructions. One Spark uses the dgx-spark-release preset. For two Sparks, build the same revision on each node with -DNINFER_ENABLE_DISTRIBUTED=ON and NCCL development dependencies installed.

From the NInfer repository root:

cp examples/configs/spark-qwen3_8-flash-next.toml ninfer.local.toml
# Set model.path to the downloaded entry file.
./build/dgx-spark-release/apps/ninfer-serve --config ninfer.local.toml

For two Sparks, start from spark2-qwen3_8-flash-next.toml, place all artifact parts on both nodes, and edit worker paths, model paths and network settings. Run the server once on the coordinator; it launches both workers. TP2 uses expert TP by default; ep = 2 selects whole-expert EP. Both examples enable Vision, FP8 KV, MTP3, the optimized head and CUDA Graph decode, with a 262,144-token logical context limit.

For one CLI request, use apps/ninfer with the same config and --prompt or --messages. See the deployment guide for commands.

RTX 5090 can run this artifact with CPU expert offload. That path needs approximately 63.3 GiB of Host Main-expert backing plus other runtime/system memory. On one Spark, experts reside on the GPU and PLE is disk-backed; dual Spark shards PLE across Device memory. File size is not a runtime memory requirement: KV/State, workspace and enabled components also affect capacity.

Features

• Text, image/video prompts, streaming, tool calls and constrained output.
• Native MTP with 1..5 drafts, full or optimized head; enable Vision with --vision.
• CUDA Graph decode, prefix reuse and up to eight active requests.
• CLI, OpenAI Chat/Responses and Anthropic Messages through the same Engine.
• Thinking enabled by default. The maintained template supports low, medium, xhigh effort; high/max map to xhigh, minimal to low, and none disables thinking.

The context limit is per request. Actual shared KV/State capacity is reported at startup and depends on the deployment and selected resources. DFlash and DFlash2 are not used by this architecture. NInfer returns tool calls to the client; it does not execute them.

Evaluation

Recorded on 2026-10-21–22 through NInfer's public OpenAI-compatible service and EvalScope 1.10.0. The server used two DGX Sparks, TP2/EP2, BF16 KV, Vision, MTP3, optimized draft head, CUDA Graphs, prefill chunk 4096, context limit 262,144 and concurrency 8.

Benchmark	Accuracy	Correct / scored samples	Repeats per question
GPQA-Diamond	91.79%	727 / 792	4
AIME 2026	97.78%	88 / 90	3
ERQA	68.00%	272 / 400	1
RealWorldQA	85.75%	656 / 765	1
MMMU-Pro (standard, 10 options)	80.81%	1,398 / 1,730	1

Scores are mean accuracy across generated samples, not pass@k. GPQA contains 198 questions with four samples each; AIME26 contains 30 questions with three samples each. The other datasets use one sample per question. MMMU-Pro uses the standard 10-option format.

All jobs completed with no failed samples. Two GPQA responses reached the output limit and had no parsed answer; one MMMU-Pro response was unparsed. They remain in the scored totals. Original dataset labels and scorer outputs are retained. Vision results were collected before the subsequent JPEG decoder fix.

Generation used thinking with xhigh effort, temperature 1.0, top-p 0.95, top-k 20, min-p 0, zero presence/frequency penalties and a 131,072-token output budget. Evaluation used 0-shot prompts, rule-based scoring and evaluator seed 42. evaluation.json preserves the run identity, configuration, counts and diagnostics.

Sources and license

• BF16 parameters and frontend resources: Qwen/Qwen3.8-Flash-Next.
• Calibrated experts and MTP: nvidia/Qwen3.8-Flash-Next-NVFP4.
• Conversion and runtime: NInfer.

The source model uses the Qwen Community License 1.0. The NVIDIA checkpoint lists the NVIDIA Open Model License and the Qwen source terms. Those source weight terms apply to this derivative. NInfer's Apache-2.0 software license is separate from the model licenses.