Can anyone make ggml 4bit q4_0 version?

#3
by 4eJIoBek - opened

I think it ll be usual

On the way! Got two bash windows git pushing my repos up now, might be a couple hours. I converted and quantized these with https://github.com/ggerganov/ggml/ on a MacBook pro M1 w/ 16GB RAM.

They are up! https://huggingface.co/oeathus/stablelm-7b-sft-v7-epoch-3-ggml-q4
My repo has the ggml f16 model as well as the 4 q4_X quantized versions.

Oh huge thx

Sign up or log in to comment