Other models?
#2
by
ekryski
- opened
Thanks for this @Green-Sky ! Any chance you're going to do the other models? Someone else did them here but only in 4 bit. Was hoping to try 5 and 8.
If not, I may try my hand at it but didn't want to duplicate work if it's already ongoing.
Go there and ask :), also looks like f16 is in that repository too, so you only need that and quantize locally. (way less hassle than converting to ggml f16 first yourself)