Update tokenizer_config.json
It seems eos_token_id is <|end|> (32007) instead of <|endoftext|> (32000).
Context: https://twitter.com/altryne/status/1783567596467491109?t=k5HHVmTCGDt4-TkXF8KyNw&s=19
Hey, thanks for proposing this fix :)
I tried to follow the twitter fix in the meantime but passing eos_token_id=32007 to the generate method still resulted in endless generation on my side at eval time. This also happens after overriding eos_token_id's value to 32007 at model and tokenizer initialization! I'm also using unk_token for padding, as advised in the repo fine tuning's example
Could this be related to the chat template's behaviour?
Really curious to get more feedback on this from other users.
Please ensure that you are using the configuration defined in generation_config.json.