malbert-base-cased-128k / tokenizer_config.json
cservan's picture
Updated tokenizer config
e3b954c
{"keep_accents": true, "do_lower_case": false, "model_max_length": 512, "tokenizer_class": "AlbertTokenizer"}