LLaMA-1B-dj-refine-100B / added_tokens.json
zhijianma's picture
Upload tokenizer
6980f60
raw
history blame
94 Bytes
{
"<CLS>": 32000,
"<EOD>": 32002,
"<MASK>": 32003,
"<PAD>": 32004,
"<SEP>": 32001
}