teknium
/

Llama-3.1-AlternateTokenizer

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

teknium commited on Jul 24

Commit

5755a67

•

1 Parent(s): 732ea55

Upload tokenizer_config.json

Files changed (1) hide show

tokenizer_config.json +1 -1

tokenizer_config.json CHANGED Viewed

@@ -265,7 +265,7 @@
       "special": true
     },
     "128033": {
-      "content": "<|reserved_special_token_25|>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,

       "special": true
     },
     "128033": {
+      "content": "<SOLUTION>",
       "lstrip": false,
       "normalized": false,
       "rstrip": false,