IQ3_XS?
#1
by
person4268
- opened
IQ3_S is just slightly too big for my setup, and the quality gap between IQ3_S and Q2_K is insane. Could you quant an IQ3_XXS version?
no, because there is no such thing as a static IQ3_XXS quant. but i will provide an imatrix IQ3_XXS quant, which will, unfortunately, wait for another model to be quanted. you can check at http://hf.tst.eu/status.html: currently, it's scheduled on node "marco" and there is one other model in front of it (Meta-Llama-3.1-405B-Instruct-Uncensored).
mradermacher
changed discussion status to
closed