typo_correction_merged_16bit / added_tokens.json
iadithyan's picture
Upload tokenizer
0a5b017 verified
{
"<|im_end|>": 32000,
"<|im_start|>": 32001
}