4bit quantization

#2
by llama-anon - opened

is 4bit quantization supported?
if not, is it planned?
if yes, is there a guide?

We would happily accept PRs, but don't have internal resources devoted to it

jfrankle changed discussion status to closed

Sign up or log in to comment