Hugging Face's logo

NousResearch
/
Meta-Llama-3-8B-Alternate-Tokenizer

Text Generation
Transformers
Safetensors
PyTorch
English
llama
facebook
meta
llama-3
conversational
text-generation-inference
Inference Endpoints
Model card Files Community
2
teknium commited on
Commit
d05a141
•
1 Parent(s): 13a9e8c

Upload tokenizer.json

Browse files
Files changed (1) hide show
  1. tokenizer.json +4 -4
tokenizer.json CHANGED
@@ -127,7 +127,7 @@
127
  "lstrip": false,
128
  "rstrip": false,
129
  "normalized": false,
130
- "special": true
131
  },
132
  {
133
  "id": 128014,
@@ -136,7 +136,7 @@
136
  "lstrip": false,
137
  "rstrip": false,
138
  "normalized": false,
139
- "special": true
140
  },
141
  {
142
  "id": 128015,
@@ -145,7 +145,7 @@
145
  "lstrip": false,
146
  "rstrip": false,
147
  "normalized": false,
148
- "special": true
149
  },
150
  {
151
  "id": 128016,
@@ -154,7 +154,7 @@
154
  "lstrip": false,
155
  "rstrip": false,
156
  "normalized": false,
157
- "special": true
158
  },
159
  {
160
  "id": 128017,
 
127
  "lstrip": false,
128
  "rstrip": false,
129
  "normalized": false,
130
+ "special": false
131
  },
132
  {
133
  "id": 128014,
 
136
  "lstrip": false,
137
  "rstrip": false,
138
  "normalized": false,
139
+ "special": false
140
  },
141
  {
142
  "id": 128015,
 
145
  "lstrip": false,
146
  "rstrip": false,
147
  "normalized": false,
148
+ "special": false
149
  },
150
  {
151
  "id": 128016,
 
154
  "lstrip": false,
155
  "rstrip": false,
156
  "normalized": false,
157
+ "special": false
158
  },
159
  {
160
  "id": 128017,