yoonhero
/

kogpt2-chat

Text Generation

generated_from_keras_callback

Inference Endpoints

text-generation-inference

Model card Files Files and versions Community

kogpt2-chat / tokenizer_config.json

yoonhero's picture

add tokenizer

d718a1d almost 2 years ago

raw history blame contribute delete

No virus

247 Bytes

	{
	"add_prefix_space": false,
	"bos_token": "</s>",
	"eos_token": "</s>",
	"name_or_path": "skt/kogpt2-base-v2",
	"pad_token": "<pad>",
	"special_tokens_map_file": null,
	"tokenizer_class": "GPT2Tokenizer",
	"unk_token": "<\|endoftext\|>"
	}