MaziyarPanahi

/

WizardLM-2-8x22B-GGUF

like 106

Hi,
Will start today 😆

AWQ is ready here: https://huggingface.co/MaziyarPanahi/WizardLM-2-8x22B-AWQ

GPTQ plz

I thought AWQ was faster with the same quality than GPTQ, is there another reason to use GPTQ?

Because we use v100 gpu for inference，not support AWQ.

That's a very good reason! I see if I can do this by GPTQ.