EUBERT / added_tokens.json
scampion's picture
Upload tokenizer
cc82d33 verified
{
"</s>": 65537,
"<mask>": 65540,
"<pad>": 65539,
"<s>": 65536,
"<unk>": 65538,
"[CLS]": 2,
"[MASK]": 4,
"[PAD]": 1,
"[SEP]": 3,
"[UNK]": 0
}