Any chance for models with lower quantization like q2_k (still usefull and better fit for mobile devices)?
Hi,I uploaded 2 and 3bit quentizations for you :)
Thanks a lot bro!;)
· Sign up or log in to comment
