Couldn't Find A GGUF Qwen
I'm not interested in using a GGUF Qwen. Just wondering if there's a technical, legal or other reason why Qwens can be found in GPTQ and AWQ, but not GGUF.
It would be really nice to have a GGUF version of it to use with Ollama.
Did a little research and apparently the token library of Qwen 14b is gigantic because of the vast number of Chinese symbols used, which someone said is incompatible with GGUF.
I don't think this is a big loss. I tested out the online chat version of Qwen 14b and it performed notably worse across the board compared to most Llama 2 13b and Mistral 7b fine-tunes, often outputting random nonsense. Which is odd because Qwen 14b scores notably higher on multi-shot LLM tests compared to Llama 13b and Mistral 7b. Perhaps this isn't an issue with the base model, but rather the inability of the official chat version to respond appropriately to 0-shot user prompts.