Quant pls
#5
by
Yhyu13
- opened
Hi, there is sft chat model released for mixtral, would you like to quant this?
This looks like it was based on the DiscoResearch unofficial Mixtral, which unfortunately means I can't quant it, because all the layer names are different to the official implementation. And all the recent quant code updates (GGUF, GPTQ, AWQ) assume the official Mixtral layer names