EleutherAI
/

polyglot-ko-1.3b

Text Generation

Inference Endpoints

text-generation-inference

Model card Files Files and versions Community

hyunwoongko commited on Sep 18, 2022

Commit

42da130

•

1 Parent(s): a9367b9

Update tokenizer_config.json

Files changed (1) hide show

tokenizer_config.json +1 -1

tokenizer_config.json CHANGED Viewed

@@ -1,6 +1,6 @@
 {
-  "eos_token": "<|endoftext|>",
   "name_or_path": "EleutherAI/polyglot-ko-1.3b",
   "pad_token": "<|endoftext|>",
   "tokenizer_class": "PreTrainedTokenizerFast"
 }

 {
   "name_or_path": "EleutherAI/polyglot-ko-1.3b",
+  "eos_token": "<|endoftext|>",
   "pad_token": "<|endoftext|>",
   "tokenizer_class": "PreTrainedTokenizerFast"
 }