gpt2-small-indonesian-story / tokenizer_config.json

updated tokenizer config

69d41ac over 2 years ago

No virus

240 Bytes

	{"unk_token": "<\|endoftext\|>", "bos_token": "<\|endoftext\|>", "eos_token": "<\|endoftext\|>", "add_prefix_space": false, "special_tokens_map_file": null, "name_or_path": "cahya/gpt2-small-indonesian-story", "tokenizer_class": "GPT2Tokenizer"}