Set `add_bos_token` to `false` to prevent auto-adding `<|bos|>` in prompts from GGUF files
#3
by
Felladrin
- opened
When converting to GGUF, if we have this set to true, it will automatically add a <|bos|> in front of the prompt, which negatively affects the generation. So I recommend setting it to false here.
PS: I've set it to false locally before converting M4-ai/TinyMistral-248M-v2-Instruct-GGUF.
Locutusque
changed pull request status to
merged