openai-gpt / tokenizer_config.json
lysandre's picture
lysandre HF staff
Adds the tokenizer configuration file (#6)
1e0d4f3 verified
{"model_max_length": 512}