EXL2 Quants are up
#2
by
Dracones
- opened
Very nice model for its size. EXL2 quants are up with perplexity scoring so people can get an idea of size vs loss: https://huggingface.co/collections/Dracones/codeqwen15-7b-chat-661ed39a0325c1978192dba1
Thanks for your contribution!
Very nice model for its size. EXL2 quants are up with perplexity scoring so people can get an idea of size vs loss: https://huggingface.co/collections/Dracones/codeqwen15-7b-chat-661ed39a0325c1978192dba1
I downloaded the 8_0 quant and it gives me an error when trying to load it up in text-generation-webui. Something to do with a missing pre tokeniser Whitespace()? At least from my searches on the internet this seems to be the problem, although I don't know how to fix it.