yahavb
/

optimum-neuron-sd21

yahavb commited on May 10, 2024

Commit

6697b2f

verified ·

1 Parent(s): 8ad2c0b

Upload tokenizer/special_tokens_map.json with huggingface_hub

Files changed (1) hide show

tokenizer/special_tokens_map.json CHANGED Viewed

@@ -14,9 +14,9 @@
     "single_word": false
   },
   "pad_token": {
-    "content": "<|endoftext|>",
     "lstrip": false,
-    "normalized": true,
     "rstrip": false,
     "single_word": false
   },

     "single_word": false
   },
   "pad_token": {
+    "content": "!",
     "lstrip": false,
+    "normalized": false,
     "rstrip": false,
     "single_word": false
   },