even smaller quants
#1
by
Samvanity
- opened
Hi, is it possible to provide a IQ2_XS? I have been able to use IQ2_XS with Llama3 70b with acceptable results. It's the perfect size for a 24GB card (RTX xx90 cards)
Thanks!
Generation is not finished, be patient, and most likely they will show up :)
mradermacher
changed discussion status to
closed