Safetensors
mistral
intervitens commited on
Commit
07466bb
·
verified ·
1 Parent(s): 92d7de6

Upload tokenizer.json with huggingface_hub

Browse files
Files changed (1) hide show
  1. tokenizer.json +2 -2
tokenizer.json CHANGED
@@ -37,7 +37,7 @@
37
  "lstrip": false,
38
  "rstrip": false,
39
  "normalized": false,
40
- "special": false
41
  },
42
  {
43
  "id": 4,
@@ -46,7 +46,7 @@
46
  "lstrip": false,
47
  "rstrip": false,
48
  "normalized": false,
49
- "special": false
50
  },
51
  {
52
  "id": 5,
 
37
  "lstrip": false,
38
  "rstrip": false,
39
  "normalized": false,
40
+ "special": true
41
  },
42
  {
43
  "id": 4,
 
46
  "lstrip": false,
47
  "rstrip": false,
48
  "normalized": false,
49
+ "special": true
50
  },
51
  {
52
  "id": 5,