enable gguf
#3
by
ehartford
- opened
hello
@francislabounty
Can you please enable gguf?
I would really like to try sparsetral.
https://github.com/ggerganov/llama.cpp/issues/5365
GPTQ and AWQ too, I imagine changes need to be made to upstream for it to work
It's interesting that exl2 just supported it out of the box