Hi, is it possible to come out with the IQ4_NL model?

by Eilian - opened

I found it to be smaller, but closer to the Q5KM in performance.

Hmm, from the results I have seen, IQ4_NL is never better than Q4_K_S (for normal models), nor would it be expected to. Do you have some actual evidence (i.e. more than a feeling) that it is worth it? If yes, I can add it in general for models.

I'll add it to his repo in any case, it should be up in a few hours.

Sign up or log in to comment