Composer
MosaicML
llm-foundry
ggml

Other models?

#2
by ekryski - opened

Thanks for this @Green-Sky ! Any chance you're going to do the other models? Someone else did them here but only in 4 bit. Was hoping to try 5 and 8.

If not, I may try my hand at it but didn't want to duplicate work if it's already ongoing.

Go there and ask :), also looks like f16 is in that repository too, so you only need that and quantize locally. (way less hassle than converting to ggml f16 first yourself)

Sign up or log in to comment