Sparkulator-Laguna-S-2.1-NVFP4 (W4A16 DFlash draft for the NVFP4 target)

A W4A16-quantized DFlash speculator for poolside/Laguna-S-2.1, tuned for single-user speculative decoding on the NVIDIA DGX Spark (GB10, sm_121).

It is a group-128 symmetric-int4 (W4A16, compressed-tensors pack-quantized) quant of poolside's Laguna-S-2.1-DFlash-NVFP4 draft. Draft weights shrink 2.08 GiB → 0.82 GiB (~2.5×); measured resident cost inside vLLM falls from 69.34 GiB → 68.08 GiB of total model memory, i.e. 1.26 GiB handed back.

⚠️ Target precision matters — use this one with the NVFP4 target

DFlash drafts ship no lm_head of their own — they share the target's and are calibrated against the target's hidden states — so a draft only works with the target precision it was built for. Cross-pairing gives 0% acceptance (a silent failure: output stays correct, decode drops below the no-draft baseline).

serve this target	use this draft
poolside/Laguna-S-2.1-NVFP4	this repo (Sparkulator-Laguna-S-2.1-NVFP4)
poolside/Laguna-S-2.1-FP8	sapidlabs/Sparkulator-Laguna-S-2.1

Measured

1× DGX Spark, vLLM 0.25.1, poolside/Laguna-S-2.1-NVFP4 target (71.9 GB, one node), k=6, --max-num-seqs 4, KV pinned to 12 GiB, temp 0, concurrency 1, warm, thinking off, code-continuation workload, 12 repeats per cell. Both arms were run on both Sparks with the arm order counterbalanced (Spark-1 ran BF16 first, Spark-2 ran W4A16 first), on 2026-07-27:

prompt	poolside BF16 DFlash	this (W4A16)	delta
512 (Spark-1)	44.14 tok/s	45.28	+2.6%
2,048 (Spark-1)	42.72 tok/s	45.85	+7.3%
8,192 (Spark-1)	44.34 tok/s	47.00	+6.0%
512 (Spark-2)	41.80 tok/s	44.79	+7.2%
2,048 (Spark-2)	43.59 tok/s	44.82	+2.8%
8,192 (Spark-2)	44.11 tok/s	45.90	+4.1%

All six comparisons favour the quantized draft, against a per-request stdev of 1.7–6.0%. Pooled, it is worth roughly +5% single-stream decode.

Acceptance is unchanged, which is the point — spec-decode is verified, so the draft can only cost speed, never quality, and the gate is whether the quant damaged the draft's guesses:

arm	drafted	accepted	acceptance
Spark-1 BF16	15,876	9,818	61.84%
Spark-1 W4A16	15,924	9,821	61.67%
Spark-2 BF16	15,948	9,782	61.34%
Spark-2 W4A16	15,990	9,778	61.15%

(Warm-up requests were subtracted from the /metrics counters before differencing — they accept far worse than real work and otherwise drag the ratio down.)

Against the same recipe's no-draft baseline (18.9 tok/s at a 2,048-token prompt), this draft is worth about 2.4×.

How it was made

Data-free group-128 symmetric RTN over the raw safetensors — no calibration, no model class, no GPU — emitting a standard compressed-tensors pack-quantized checkpoint that vLLM loads through its marlin path.

• Quantized: every layer's mlp.{gate,up,down}_proj, self_attn.o_proj, and the DFlash fc encoder.
• Left BF16: self_attn.qkv_proj — vLLM's _build_context_kv_buffers reads qkv_proj.weight directly with no dequant path — and the tiny softplus gate self_attn.g_proj.
• The ignore list is written as re: regexes, because vLLM matches ignore entries against the prefixed module name (model.layers.N…); literal names silently miss and the layer gets quantized anyway, then KeyErrors on .weight.

This draft has 6 layers, which is why data-free RTN is enough: quantization error has somewhere to wash out. A 1-layer EAGLE3-style head given the same treatment loses several points of acceptance.

Serving on a DGX Spark (vLLM 0.25.1)

vllm serve poolside/Laguna-S-2.1-NVFP4 \
  --max-model-len 262144 \
  --gpu-memory-utilization 0.85 \
  --kv-cache-memory-bytes 12884901888 \
  --max-num-seqs 4 --max-num-batched-tokens 8192 \
  --speculative-config '{"method":"dflash","model":"sapidlabs/Sparkulator-Laguna-S-2.1-NVFP4","num_speculative_tokens":6}'

One vLLM patch is needed for a quantized DFlash draft to load (0.25.1 assumes an unquantized/FP8 draft), on every node — in vllm/model_executor/models/laguna_dflash.py, guard the FP8-only get_cache_scale call:

if self.quant_config is not None and hasattr(self.quant_config, "get_cache_scale") and (
        scale_name := self.quant_config.get_cache_scale(name)):

CompressedTensorsConfig does not implement that method. Keeping qkv_proj in BF16 (as this checkpoint does) is the second half of the fix.

If you A/B two drafts of different quantization on one node, point VLLM_CACHE_ROOT at a fresh directory per variant, or serve --enforce-eager — vLLM's compile cache does not re-key on draft quantization and you will hit KeyError: 'weight_packed'.

License & attribution

Derived from poolside/Laguna-S-2.1-DFlash-NVFP4; released under the same OpenMDW-1.1. All credit for the base model and the DFlash speculator design to poolside. This is not an official poolside release — it only re-encodes their draft weights to 4-bit for the DGX Spark. Built and measured by How To Spark.