Error during converting to gguf

#3
by apepkuss79 - opened

Hi, I'm trying to convert this model to gguf format. However, I ran into the following error:

Exception: Vocab size mismatch (model has 32000, but /home/ubuntu/workspace/models/sqlcoder-34b-alpha has 32004).
Defog.ai org

Hi there, I think this is because of the way the gguf converter handles special tokens. Following some of the fixes suggested here ought to work! https://github.com/ggerganov/llama.cpp/issues/2820

@rishdotblog Thanks for the reply. I'll give it a try.

This comment has been hidden

Sign up or log in to comment