Qwen3.8-27B-DFlash2-NVFP4-modelopt

Re-quantized DFlash2 draft for sglang's DFLASH speculative decoding, used by hamichok/Qwen3.8-27B-NVFP4-RTX5090-LMHead4.

This is a re-quantization of incoai/Qwen3.8-27B-DFlash2 (BF16) to modelopt-NVFP4 so sglang can load it as a speculative draft. The upstream compressed-tensors NVFP4 variant does not load in sglang; this one does.

Why this exists

• DFLASH needs a draft the target lm_head-quantized checkpoint can verify; the quantized head breaks DSPARK but not DFLASH.
• fc and q/k/v stay dense; only o_proj + MLP projections are quantized (the dflash.py loader shape-checks fc.weight as dense).

Usage (serve command)

Point --speculative-draft-model-path at this repo and --speculative-draft-model-quantization modelopt_fp4, with --speculative-dflash-block-size 6 (measured throughput peak). See the main model card for the full recipe and measurements.

Provenance

Built with quantize-dflash2.py (Apache-2.0, from the main recipe repo). Base: incoai/Qwen3.8-27B-DFlash2 (Apache-2.0).