taeminlee
/

kogpt2

kogpt2 / tokenizer_config.json

system HF staff

Update tokenizer_config.json

2983390 almost 4 years ago

No virus

109 Bytes

{"do_lower_case": false, "unk_token": "<unk>", "pad_token": "<pad>", "bos_token": "<s>", "eos_token": "</s>"}