openthaigpt-gpt2-pantipwiki-poc / added_tokens.json
cwtpc's picture
Upload tokenizer
c9948d2
{
"<|endoftext|>": 50265,
"<|pad|>": 50267,
"<|startoftext|>": 50266
}