why vocab size is 32001

#3
by yechenzhi1 - opened

just out of curiosity, why do you increase the vocab size from 32000 to 32001.

because mistral does not have a padding token so we add a [PAD]. you cannot use the eos token as the pad token because in this case the multi-turn conversation does not work probably because the chat template of mistral.

Sign up or log in to comment