tkcho commited on
Commit
36a7fb8
1 Parent(s): 7007c4c

Upload tokenizer

Browse files
Files changed (4) hide show
  1. added_tokens.json +1 -1
  2. special_tokens_map.json +1 -1
  3. tokenizer.json +0 -0
  4. vocab.txt +0 -0
added_tokens.json CHANGED
@@ -1,3 +1,3 @@
1
  {
2
- "[2024-03-06 10:31:00+09:00]": 20000
3
  }
 
1
  {
2
+ "[2024-03-06 10:42:00+09:00]": 20000
3
  }
special_tokens_map.json CHANGED
@@ -1,6 +1,6 @@
1
  {
2
  "additional_special_tokens": [
3
- "[2024-03-06 10:31:00+09:00]"
4
  ],
5
  "cls_token": "[CLS]",
6
  "mask_token": "[MASK]",
 
1
  {
2
  "additional_special_tokens": [
3
+ "[2024-03-06 10:42:00+09:00]"
4
  ],
5
  "cls_token": "[CLS]",
6
  "mask_token": "[MASK]",
tokenizer.json CHANGED
The diff for this file is too large to render. See raw diff
 
vocab.txt CHANGED
The diff for this file is too large to render. See raw diff