how to convert the model to gguf?

#1
by rinoa - opened

Hi, Could you please share the steps to convert this model into GGUF format? I tried to the convert.py from llama.cpp, but got an error:

Exception: Vocab size mismatch (model has 102400, but tokenizer.model has 32000).

deleted

It crashes when using GPT4ALL, so I'm not sure the Bloke was able to convert it to GGUF while maintaining full compatibility.

All the Chinese models (Yi, Qwen & Deepseek) are having an issue with GGUF, likely because of the gigantic token vocabulary required by the Chinese language.

Sign up or log in to comment