4K_0 gone and 4K_XS?
What is the new quant? How does it compare with other 4K quants? And why did 4K0 disappear? K0 recently got boost in speed (couple of days ago, new release boosted speed, now all speeds will boost its speed to some extent, while generation quality remains the same, sorry forgot what PR number was that).
Thank you I found PR for this: https://github.com/ggerganov/llama.cpp/pull/5060
Inferring from discussion there, I think this is new K0 i.e. between Q3 and Q4.
There is a good discussion here: https://huggingface.co/MaziyarPanahi/Venomia-1.1-m7-Mistral-7B-Instruct-v0.2-slerp-GGUF/discussions/1#65e7e609a73ffb80b47fd7fd
I kept my list and just added one of these new _XS for the 4 bits for testings, hopefully, I can add more of the new ones once I know which ones are useful to the community.