yushengsu/deepseek-v4.1-flash-dspark-8000step

DeepSeek-V4.1-Flash DSpark draft, SpecForge fine-tune (block size 7)

Draft model (3 layers, 128-expert MoE, 14.2B parameters) of DeepSeek-V4.1-Flash, fine-tuned with SpecForge from the official mtp.* draft on regenerated thinking-mode rows (global batch 128, no row reused, training block size 7; serve with --speculative-dspark-block-size 5), trained until the regenerated data ran out.

• Code: https://github.com/sgl-project/SpecForge/pull/1073 (branch v4.1-dspark)
• W&B: https://forge.coreweave.com/wandb/eigent_radixark_training/deepseek-v4.1-flash-dspark/runs/v41-b200-8000step-b7
• Data: yushengsu/deepseek-v4-flash-0731-thinking-regen (data/deepseek-v4.1-flash/, private)
• Recipe: examples/configs/online/disaggregated/external/deepseek-v4.1-flash-moe-dspark.yaml (global batch 128, constant learning rate 2e-5, warm start from the official draft)

Every 500 global steps: step-<g>/specforge-hf/ (SpecForge HF export, bf16, specforge export --to hf) and step-<g>/official-mtp/ (official mtp.* layout for SGLang --speculative-algorithm DSPARK, scripts/convert_deepseek_v41_dspark.py --to-official).