Sparkulator-Laguna-S-2.1 (W4A16 DFlash draft)

A W4A16-quantized DFlash speculator for poolside/Laguna-S-2.1, tuned for single-user speculative decoding on the NVIDIA DGX Spark (GB10, sm_121).

It is a group-128 symmetric-int4 (W4A16, compressed-tensors pack-quantized) quant of poolside's own Laguna-S-2.1-DFlash-FP8 draft. The draft weights shrink 2.08 GiB → 0.82 GiB (~2.5×); the attention qkv_proj and the softplus gate g_proj are deliberately left in BF16 (see How it was made).

Why

On a DGX Spark, a DFlash draft runs on every decode step, so a smaller/faster draft lifts single-stream decode even when acceptance is unchanged. Measured on 2× DGX Spark (TP2, vLLM 0.25.1, --enforce-eager) serving poolside/Laguna-S-2.1-FP8, single-stream decode at a 2048-token code prompt, warm:

draft	decode tok/s	draft acceptance	draft size
none (baseline)	26.5	—	—
poolside FP8 DFlash (k=8)	58.4	52%	2.08 GiB
this — W4A16 DFlash (k=6)	60.9	52%	0.82 GiB

So +4% decode at identical acceptance, from a draft 2.5× smaller. Decode is a gentle plateau over k (k=6 60.9, k=8 60.3, k=10 55.9) — k=6 is the measured peak. Correction (2026-07-27): an earlier version of this card said poolside's -DFlash-NVFP4 draft "produces 0% acceptance" on this stack. That was a mispairing on our side, not a broken draft. DFlash drafts are target-precision-specific: -DFlash-NVFP4 gives 0% against an FP8 target and ~62% against the NVFP4 target it was built for. If you serve poolside/Laguna-S-2.1-NVFP4 instead, use sapidlabs/Sparkulator-Laguna-S-2.1-NVFP4, the W4A16 quant of that draft (+2.6 to +7.3% decode over the BF16 original, acceptance unchanged, measured on two DGX Sparks with the arm order counterbalanced).

How it was made

Data-free group-128 symmetric RTN over the raw safetensors (no calibration, no model class needed), producing a standard compressed-tensors W4A16 checkpoint that vLLM loads through its marlin path. Quantized: every layer's mlp.{gate,up,down}_proj, self_attn.o_proj, and the DFlash fc encoder. Left BF16: self_attn.qkv_proj (vLLM's DFlash context-KV buffer reads qkv_proj.weight directly with no dequant path) and self_attn.g_proj (tiny gate). The ignore list uses re: regexes so vLLM's prefix-matched module names are excluded correctly.

Serving on a DGX Spark (vLLM 0.25.1)

Serve poolside/Laguna-S-2.1-FP8 as the target with this as the DFlash draft:

vllm serve poolside/Laguna-S-2.1-FP8 \
  --enforce-eager \
  --speculative-config '{"method":"dflash","model":"sapidlabs/Sparkulator-Laguna-S-2.1","num_speculative_tokens":6}'

Two vLLM patches are needed for a quantized DFlash draft to load (vLLM 0.25.1 assumes an unquantized/FP8 draft), on every node:

1. In laguna_dflash.py, guard the FP8-only get_cache_scale call: ... and hasattr(self.quant_config, "get_cache_scale") and (scale_name := ...).
2. Keep self_attn.qkv_proj in BF16 (as this checkpoint does) so _build_context_kv_buffers can read qkv_proj.weight; otherwise patch a dequant there.

Under --enforce-eager the vLLM compile-cache-vs-draft-quant collision does not apply (there is no compiled graph).

License & attribution

Derived from poolside/Laguna-S-2.1-DFlash-FP8; released under the same OpenMDW-1.1. All credit for the base model and the DFlash speculator design to poolside. This repository only re-encodes their draft weights to 4-bit for DGX Spark.