Prefill reads ~2x the experts it needs — fixing it is 1.7x on one SSD, 2.7x on three
If you're streaming this Q4 from NVMe with ds4 because it doesn't fit in RAM, there's
free performance in the prefill path.
The layer-major prefill sweep reads all 384 experts per layer. A 512-token chunk only
routes to 187 of them — 48.8%. So it reads roughly twice what the model actually touches.
Measured on unmodified ds4 (bd66c40), single internal SSD: 301.9 GB read where the
prompt touches 149.2 GB.
Running the router first and staging only the selected experts fixes it. Numbers on an
M5 Max 128 GB, 512-token prompt / 200 generated, interleaved against upstream, identical
output SHA-256 on every arm:
| upstream | ours | ||
|---|---|---|---|
| prompt processing, one drive, no replicas | 16.50 | 28.04 | 1.70× |
| steady decode, one drive | 10.44 | 14.38 | 1.38× |
| prompt processing, three drives | 16.23 | 43.62 | 2.69× |
| steady decode, three drives | 10.61 | 17.38 | 1.64× |
The one-drive rows are the point: that gain needs no extra hardware, just not reading half
the bytes. The three-drive rows add weighted split reads across byte-identical replicas,
which is what more devices buy on top.
One caveat — it's adaptive by prompt length. At 512 tokens the routed union is 48.8% and it
wins clearly; above ~1024 tokens the union passes 60% and full staging with read-ahead is
better. The threshold is a tunable.
Branch: https://github.com/argonautlabsai/ds4-argodrive/tree/argonaut-v41-benchmark
A fresh clone builds ds4 and ds4-bench with the multi-source reader compiled in.
Method, every arm and the raw sampler data:
https://argonautlabs.ai/research/deepseek-2026-09-15.html
Reported to antirez by email before posting.
Awesome optimization! Will integrate it into DwarfStar soon.
Awesome optimization! Will integrate it into DwarfStar soon.
Thank you!
Everything's on the argonaut-v41-benchmark branch of argonautlabsai/ds4-argodrive.
If you only pick one commit, 38e200a: the selective prefill path was gated behind "two or more sources", and with the gate at one it runs on a single drive. On the internal SSD alone that's 16.5 → 28 tok/s prefill and 10.4 → 14.4 decode, same output SHA. Only kicks in for prompts over 256 tokens.
One question: any plans for a DSpark file for V4.1 Flash?
Decode is GPU-bound for us now, so a drafter is the one thing left that could move it a lot.
Happy to help on the DwarfStar side if useful.
New result: 20.115 generation-inclusive tok/s, or 20.375 steady.
Setup: V4.1 Flash Q4, M5 Max 128 GB, internal SSD plus two TB5 enclosures.
Method: 512 prompt tokens, 512 generated tokens, two runs per configuration in BAAB order.
https://github.com/argonautlabsai/ds4-argodrive/releases/tag/v41-router-qualified-20260921