iac-v1 / added_tokens.json
sivavee-train's picture
add tokenizer
725fd7a
{"<BOS>": 50257, "<EOS>": 50258, "<PAD>": 50259}