Can you make a HQ quant for SDNQ?
In comfyui I'm using HQ int8 quant from here https://huggingface.co/DmitryDB/MiniMax-H3-ComfyUI-Quants
It has every attention layers in BF16 and some other layers completely in BF16. There will be no much difference in size but quality will improve, I believe
I'll take a look, it should be easy to match it
I checked and they mostly have the same as this one, sdnq does a dynamic quantization and I do a profile before too to get the most affected layers, the remaining ones that they kept (~3B params) didn't have a significant loss.
Not sure if it's worth to have another version of the model with those, I will try to test a video with the same params with both, probably you should do the same, that's the best way to measure this since mathematically they don't make a real difference.