poolside

/

Laguna-S-2.1-NVFP4

like 171

Follow

Poolside 1.36k

The 2026-08-01 update added re:^model\.layers\.4[0-7]\.mlp\.experts(\..*)?$ to the quantization ignore list, so the routed experts in the last 8 layers now ship as BF16. That is the whole size change: 19.3B parameters at BF16 instead of NVFP4 is +27.8 GB, which is 71.9 GB to 99.7 GB.

Two options that work on one RTX PRO 6000 today:

1. Pin the last export that fits: --revision 07614121b3 on this repo, with draft revision 4cdcc6e9b2 on the DFlash repo. This is poolside's own 2026-07-22 export (spinquantless, includes the looping fixes, 256K config).
2. I requantized the 8 BF16 layers of the current release back to NVFP4, keeping everything else byte-identical, including the 1M context config: https://huggingface.co/kkuspa/Laguna-S-2.1-NVFP4-0804 (71.9 GB). The quantization procedure reproduces poolside's own export bit-exactly on tensors they ship in both forms, and the activation scales come from their 07-22 checkpoint. Recipe and caveats are in the model card.

Measured on one RTX PRO 6000 Blackwell with the DFlash draft at n=7: about 125 tok/s prose, 205+ tok/s code, single stream.

...

2. I requantized the 8 BF16 layers of the current release back to NVFP4, keeping everything else byte-identical, including the 1M context config: https://huggingface.co/kkuspa/Laguna-S-2.1-NVFP4-0804 (71.9 GB). The quantization procedure reproduces poolside's own export bit-exactly on tensors they ship in both forms, and the activation scales come from their 07-22 checkpoint. Recipe and caveats are in the model card.

Measured on one RTX PRO 6000 Blackwell with the DFlash draft at n=7: about 125 tok/s prose, 205+ tok/s code, single stream.

oh wow, so looks like DFlash works with your NVFP4-only!

I'll download your model and try! (ignore my question in the other thread, I was asking about this)