DFlash configuration issue causing acceptance rate collapse
tl;dr the DFlash model has the wrong YaRN scaling factors compared to the model GGUFs, which causes the acceptance rate to plummet rapidly. Pass these args to fix it:
--rope-scaling yarn --rope-scale 32 --yarn-orig-ctx 8192
The model is set to 256k total context, while the DFlash drafter still has YaRN set for 1M context?
I've also found on my DGX Spark that the n-max is probably better closer to 3 than it is to 15, but maybe I need to re-test with these new args. The performance with DFlash still isn't amazing, so I wonder if it is a DFlash issue, or if llama-server just needs an improved DFlash implementation.
GPT-5.6-Sol investigated, and this was its conclusion:
The acceptance collapse is caused by a positional-encoding mismatch in the llama.cpp DFlash integration.
The target and draft contexts were running different RoPE scales:
- Laguna target: YaRN ×32, freq_scale = 0.03125
- DFlash draft: unscaled, freq_scale = 1.0
That divergence becomes increasingly destructive as the prompt grows.
| Test | Default DFlash | Draft forced to YaRN ×32 |
|---|---|---|
| 3,019-token prompt | 7.07%, 12.67 tok/s | 39.08%, 23.59 tok/s |
| 19,459-token prompt | 0/351 DFlash tokens, 16.35 tok/s | 51/120 DFlash tokens, 38.35 tok/s |
The long test’s reported 62.5% aggregate acceptance included ngram-mod; DFlash itself accepted 42.5%.
Also ruled out:
- Slot/LCP reuse: fresh and reused slots both reached zero DFlash acceptance.
- Split prefill: one 4,096-token batch behaved the same as normal 2,048-token batches.
llama-server creates the draft context by copying the general runtime parameters, but unspecified RoPE parameters then resolve independently from each GGUF’s metadata: llama.cpp/tools/server/server-context.cpp:1172 and llama.cpp/common/speculative.cpp:2247. Poolside’s target GGUF advertises the recommended 256K configuration as YaRN ×32, while the draft retains its native unscaled metadata. Poolside’s model card (https://huggingface.co/poolside/Laguna-S-2.1-GGUF) publishes this 256K target and DFlash pairing without compensating draft-specific flags.
The working configuration workaround is:
--rope-scaling yarn --rope-scale 32 --yarn-orig-ctx 8192
Hey, thanks for the comment. We encountered this issue independently & we'll repackage the GGUF shortly to tackle them. Once it's done, I'll update here.
