Eos token.

#1
by arthrod - opened

Hi,

I saw you played with different tokens as eos in different finetunes... Did you reach any conclusion?

Thanks!

Owner

I actually am finding that setting it to 128009 "<|eot_id|>" works the best, so for my future models I will set that. Since all my fine tunes are based on the instruct model that uses that eos token anyways.

And keep pad as endoftext?

Sign up or log in to comment