Incremental VRAM of DFlash2 at the matched 8K setting
#1
by alphabell-research - opened
Thanks for publishing pinned targets, exact SGLang commands, bfloat16 KV dtype, accepted length, and separate decode and end-to-end rates. The card also says the drafter shares the target embedding and output head.
Could you report additional GPU memory or remaining headroom for target plus DFlash2 versus target-only with the same batch-1, 8,192-token context and 16,384 max-total-token setting? A paired BF16 or FP8 case would help builders judge the speed and memory tradeoff of the three-layer drafter. Since --mem-fraction-static 0.88 reserves memory, a model/draft allocation delta or before/after headroom may be more informative than process-reserved VRAM.