q5/q6 GGUF?
#1
by
johnnnna
- opened
Hi
@Nexesenex
thanks for this gguf version. I'm trying the Q4_K_M version and it's really good.
By any chance are you considering uploading other quantized versions such as Q5 M,L, Q6 etc.?
Hello John.
I will do Q4_K_S (for everyone), Q5_K_S, Q5_K_M and Q6_K and upload them this evening at the latest.