huginnfork/Qwen3.8-Flash-Next-NVFP4-qwen4exp

Qwen3.8-Flash-Next-NVFP4 (qwen4_exp naming)

A fork of local-inference-lab/Qwen3.8-Flash-Next-NVFP4.

The only change is config.json. All weight shards are byte-identical to the source repo (same LFS object hashes); no tensor was touched.

What changed and why

Upstream vLLM merged this architecture as qwen4_exp (PR #53896). The source checkpoint is exported under the downstream names, which vLLM's registry does not recognise, so serving it required rewriting config.json at container start. This fork bakes the rename in so vllm serve works directly against the repo.

key	source	this fork
model_type	qwen3_8_flash_next	qwen4_exp
architectures[0]	Qwen3_8FlashNextForConditionalGeneration	Qwen4ExpForConditionalGeneration
text_config.model_type	qwen3_8_flash_next_text	qwen4_exp_text
vision_config.model_type	qwen3_5_vision	qwen4_exp

These match the classes vLLM registers in vllm/models/qwen4_exp/config.py — including the vision tower, whose Qwen4ExpVisionConfig.model_type is qwen4_exp rather than the inherited qwen3_5_vision.

Nothing else in config.json differs: same keys, same order, same values, same formatting, including the quantization_config block. The edit is textual and line-local, so diff against the source shows exactly four changed lines.

Serving

vllm serve <path-to-this-repo> \
    --tensor-parallel-size 1 \
    --max-model-len 180224 \
    -cc.cudagraph_mode=PIECEWISE

PIECEWISE is required: the PLE mmap forward is a blocking host round-trip and cannot be captured. Leave the KV cache dtype at the bf16 default — the Qwen4Exp QSA kernel rejects anything else.

License

Inherited from the source repository, which declares no license metadata. Refer to local-inference-lab/Qwen3.8-Flash-Next-NVFP4 and the upstream Qwen release terms.