tiny-random-openai-gpt / tokenizer_config.json
LysandreJik's picture
v2 update
f5ae251
{"unk_token": "<unk>", "model_max_length": 512, "special_tokens_map_file": null, "name_or_path": "openai-gpt", "tokenizer_class": "OpenAIGPTTokenizer"}