promptgen / added_tokens.json
aksty's picture
add tokenizer
2449eaa
raw
history blame
51 Bytes
{
"<|pad|>": 50258,
"<|startoftext|>": 50257
}