why EOS=<|endoftext|> not </s> when <s> is used as BOS?
#70
by
shiyue
- opened
Hey, I'm curious about why <|endoftext|> is used as the EOS token not </ s>, when BOS token is < s>? Seems a little bit odd lol
I am confused too, as to why does phi-3 need a at the first place? I understand chat template is necessary, but this does not seem to be a part of it.