barthez / tokenizer_config.json
moussaKam's picture
Adds missing tokenizer configuration file (#3)
8fe7a6d verified
{"model_max_length": 1024}