Change the value of add_eos_token to false
#1
by
mitmul
- opened
From what I can see in the tokenizer_config.json of plamo-13b-instruct, it seems better for the value of this option to be false.
https://huggingface.co/pfnet/plamo-13b-instruct/blob/main/tokenizer_config.json#L3
torotoki
changed pull request status to
merged