Q4_0 quant possible?
#1
by
Zwenyn
- opened
Sorry for the annoyance, but would it be possible to get a q4_0 quant of this model too? Since it's a 4b model, it would fit in many phones and your examples were very promising; it'd be very nice to use it locally on my phone and q4_0 runs so much better on it. Thank you!
Excellent; thank you - I will put some up.
Putting up both Q4_0 and Q5_0 for both 4B "NEO" and "Horror"
Should be up an hour or two depending on upload gods.
That's fantastic, thank you very much!
Zwenyn
changed discussion status to
closed