Eos token.
#1
by
arthrod
- opened
Hi,
I saw you played with different tokens as eos in different finetunes... Did you reach any conclusion?
Thanks!
I actually am finding that setting it to 128009 "<|eot_id|>" works the best, so for my future models I will set that. Since all my fine tunes are based on the instruct model that uses that eos token anyways.
And keep pad as endoftext?