eos token
#1
by
mljxy
- opened
it works better with eos token set to 7, <|im_end|>
python3 gguf-py/scripts/gguf-set-metadata.py models/yi-34b-chat.Q5_K_M.gguf tokenizer.ggml.eos_token_id 7
i have the same problem as well, haha
I think it should be 7 indeed: https://huggingface.co/01-ai/Yi-34B-Chat/discussions/5#65603109842e0ce604d25ee2
I am not sure about the
"add_bos_token": false,
"add_eos_token": false,
though