autodescgpt2 / added_tokens.json
Amisha182001's picture
Upload tokenizer
d72d597
{
"<|BOS|>": 50257,
"<|EOS|>": 50258,
"<|PAD|>": 50260,
"<|SEP|>": 50261,
"<|UNK|>": 50259
}