Request for another exl quant
#1
by
Clevyby
- opened
Nice, I liked the 13b version and am looking forwards to the 20b one. So I'd like to request a 4bpw quant. I did some testing and I can fit a 4bpw in free tier colab. I know that the quality of llm's are really sensitive and fluctuates in regards to 3 bpw range. It'd be nice to list this to the number of 20b's I can fully run in cloud.
Clevyby
changed discussion status to
closed
Hello, I would like a 4.55 bpw quant of this.
Clevyby
changed discussion status to
open