Quantized version?
#3
by
CR2022
- opened
Hello, which commands to use to make a quantized version of this model?
I have the hardware to do it.
Maybe using the converters found with llama.cpp?
This is a mpt 7b fined tuned model I believe, we should be able to get autogptq working