Exllamav2 quant needed
#1
by
zypcastles
- opened
@Apel-sin Hi, I've seen you've involved in QwQ series. Could you pls try to make an exl quant for this model? It would be interesting to test.
Hi! Yes, I can :)
I will do it in 1-2 days.
https://huggingface.co/Apel-sin/qwq-32b-instruct-abliterated-exl2
What bitrate do you need?
Thank you very much! In my intial test I think this combination is very good and meet my expectation, would you pls make a 4bit or 6bit quant as well? @Apel-sin
zypcastles
changed discussion status to
closed
zypcastles
changed discussion status to
open
zypcastles
changed discussion status to
closed