vikhr_encoder / added_tokens.json
AlexWortega's picture
Upload tokenizer
8bed65b verified
raw
history blame
377 Bytes
{
"<|future_token_0|>": 79085,
"<|future_token_10|>": 79095,
"<|future_token_11|>": 79096,
"<|future_token_1|>": 79086,
"<|future_token_2|>": 79087,
"<|future_token_3|>": 79088,
"<|future_token_4|>": 79089,
"<|future_token_5|>": 79090,
"<|future_token_6|>": 79091,
"<|future_token_7|>": 79092,
"<|future_token_8|>": 79093,
"<|future_token_9|>": 79094
}