Error during converting to gguf
#3
by
apepkuss79
- opened
Hi, I'm trying to convert this model to gguf format. However, I ran into the following error:
Exception: Vocab size mismatch (model has 32000, but /home/ubuntu/workspace/models/sqlcoder-34b-alpha has 32004).
Hi there, I think this is because of the way the gguf converter handles special tokens. Following some of the fixes suggested here ought to work! https://github.com/ggerganov/llama.cpp/issues/2820
@rishdotblog Thanks for the reply. I'll give it a try.
This comment has been hidden