bertoslav-limited / tokenizer_config.json

bertoslav-limited

bc86466 over 2 years ago

No virus

228 Bytes

{"special_tokens_map_file": "/ext/ivan/c4/multilingual/tokenizer/trained_tokenizer/special_tokens_map.json", "name_or_path": "/ext/ivan/c4/multilingual/tokenizer/trained_tokenizer/", "tokenizer_class": "PreTrainedTokenizerFast"}