Failure to Properly Append EOS Token
#1
by
LoafyLemon
- opened
Hi, first of all, thanks for the quant!
There's a small issue; The model keeps adding the word "assistant" instead of properly ending the text with an end-of-sequence (EOS) token, the config files might need updating to reflect the proper EOS id of 128009
.
To users: To resolve this on your end, you need to adjust the configuration file generation_config.json
. Specifically, you should locate the parameter eos_token_id
within this file and change its value from 128001
to 128009
. By doing this, you're instructing the model to correctly recognise the end of the generated text and stop appending additional words.