Hi — great release. The retention numbers here (several benches at 96–102% of BF16) are the best I've seen at this bit-width for this model family, and the codec work is genuinely interesting.
One question about reach: I run Qwen3.8-Flash-Next on small machines (12 GB VRAM / 32 GB RAM class), where builds of this model are only usable with expert streaming plus a compressed n-gram table. Qwen ships that table at FP8 (~51 GB), and other builds of this model carry it around ~29 GB (IQ4_NL) with small per-value error. Mach-2's 26.7 GB of weights would be a huge win for that class of machine; right now the 102 GB bf16 table is what puts it out of reach — the whole file would need to land in the ~40–64 GB range to help these boxes.
So, three questions:
Is a smaller-table variant (or a lower-RAM target) something you're planning or considering?
Is bf16 on the table a measured requirement — have you tried it at FP8/IQ4_NL-class precision with the new codec, and how does retention move?
If it costs a few benchmark points, would you consider publishing a compressed-table option as experimental?
Since the table is only read a few rows per token, it looks like the natural place to trade precision for reach.
Thanks for the feedback! Yes, the model is robust to lower precision on the table. Typically you can store it in SSD which is why we left it in full precision, but we can release a smaller table variant if that's helpful!
