roberta-base-64k / tokenizer_config.json
bab2min's picture
Update tokenizer_config.json
743657d verified
{
"model_max_length": 512,
"tokenizer_class": "KiwiTokenizer",
"added_tokens_decoder": {}
}