q4k_m doesn't work in oobabooga

#2
by ramzeez88 - opened

any remedies for it ?

Me too, Q4 can't work with Ollama.

Q5 can work but it's to slow for my 16G Macbook pro.

Could you please bulid a workable Q4 version, or 7b version?

Thank you very much!

Sign up or log in to comment