redditbot / added_tokens.json
lagodw's picture
add tokenizer
1d50d0b
{"<|startoftext|>": 50257, "<sep>": 50258, "<|pad|>": 50259, "<cls>": 50260, "<mask>": 50261}