zsql-en-sqlite / added_tokens.json
sdiehl's picture
Upload tokenizer
2420851 verified
{
"<pad>": 32002,
"<|im_end|>": 32000,
"<|im_start|>": 32001
}