GPTQ quants

#5
by dazipe - opened

Hello,

Would you consider uploading GPTQ 4bit and 8bit quants for the Qwen/QwQ-32B?
The AWQ is supported on newer GPUs only. Support is particularly poor on AMD cards.

You have provided it previously. For example your GPTQ quants have the most downloads for the Qwen2.5-32B-Instruct:
https://huggingface.co/models?other=base_model:quantized:Qwen%2FQwen2.5-32B-Instruct&sort=downloads

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment