4bit GPTQ

#1
by FrankWu - opened

can provide 4bit quantized by gptq? My gpu doesn't support awq, thank you

Checkout TheBloke's Yi-34B Chat GPTQ https://huggingface.co/TheBloke/Yi-34B-Chat-GPTQ

FrankWu changed discussion status to closed

Sign up or log in to comment