Question about tokenizer

#1
by LukeYang - opened

May I ask how did you generate the .spm file for the MarianTokenizer? I'm trying to train a google/sentencepiece model, but it only returns the .model and .vocab file. Should I convert them manually? or is there any other methods.

LukeYang changed discussion status to closed
LukeYang changed discussion status to open
LukeYang changed discussion status to closed

Sign up or log in to comment