SOBertBase / tokenizer_config.json
mmukh's picture
Upload 3 files
9edf289
{
"model_max_length": 2048,
"tokenizer_class": "PreTrainedTokenizerFast"
}