Measured: MTP depth N=6 gives ~2.5x decode throughput on QAT Q4_0, at -0.83pt quality (single RTX 5070 Ti)
Sharing measurements in case they are useful to anyone running this model locally. Independent, single-operator study; I have no affiliation with the Gemma team.
Setup: paired benchmark of gemma-4-12B-it and gemma-4-26B-A4B-it, both QAT Q4_0, on one RTX 5070 Ti with a pinned CUDA llama.cpp build. Multi-token prediction depth swept over a fixed grid.
What I measured at N=6 (fastest depth in the tested grid for both models):
- 12B: aggregate decode throughput 89.20 to 226.39 tok/s
- 26B-A4B: 169.58 to 322.53 tok/s
- Paired objective macro-score change: -0.83 percentage points for both models, with 95% CIs above the prospectively frozen -5-point pilot margin
The number I did not expect, and the reason I am posting rather than just tweeting a speedup: exact byte equality with ordinary decoding was only 112/200 for 12B and 118/200 for 26B-A4B. So MTP here is not a free lunch that returns the same tokens faster - roughly 45% of completions differ textually, even though aggregate quality barely moves. If your pipeline diffs outputs, caches them, or has snapshot tests, that matters more than the throughput number does.
Boundaries, stated up front: this holds for the measured GPU, runtime commit, quantization, settings and workloads only. N=6 is an evidence-backed default for that stack, not a universal optimum, and I have not tested other GPUs or backends.
Protocol was frozen before collection; scripts, processed measurements, item hashes and the claims matrix are public if anyone wants to re-run or dispute it:
- Code and reproduction: https://github.com/mobius-style/gemma4-mtp-quality-study
- Dataset: https://huggingface.co/datasets/moebiusT7/gemma4-mtp-quality-study
- Paper (DOI): https://doi.org/10.5281/zenodo.21860461
If someone has run MTP on other hardware or a different llama.cpp commit, I would be interested in whether the byte-equality figure moves - that is the part I am least sure generalises.