Would love 5.0bpw
#2
by
koesn
- opened
Thank's for 4.25bpw, works great. This model is my default choice for now, but I would like to maxxed out my 3x3060 vram. I hope you add 5.0bpw, if you don't mind. It perfectly suit full 32k ctx on those gpus with cache_4bit. Thanks a lot, @bartowski
I should be able to do that relatively soon yes, I'll add it to my queue and let you know when it's done