lemur-70b-chat-v1 / added_tokens.json
ranpox's picture
Upload tokenizer
fdcec31
raw
history blame
120 Bytes
{
"<pad>": 32000,
"<|diff_marker|>": 32004,
"<|im_end|>": 32002,
"<|im_sep|>": 32003,
"<|im_start|>": 32001
}