Vocabulary Size?

#10
by MH1P - opened

Hi!

Thank you for the model.
I am just wondering: the vocab size in config.json says 32032, but only 2 tokens are added. What are the extra 30 tokens?

Thanks!

NousResearch org

Axolotl doesnt actually add padded tokens for and so it didn't. I just merged a PR that does

Thanks!

MH1P changed discussion status to closed

Sign up or log in to comment