aragpt2-mega-pos-msa / added_tokens.json
alsubari's picture
Upload tokenizer
578462b
raw
history blame
51 Bytes
{
"<|pad|>": 64001,
"<|startoftext|>": 64000
}