tkcho commited on
Commit
6bdf96c
1 Parent(s): 5e76d25

Upload tokenizer

Browse files
Files changed (4) hide show
  1. added_tokens.json +1 -1
  2. special_tokens_map.json +1 -1
  3. tokenizer.json +0 -0
  4. vocab.txt +0 -0
added_tokens.json CHANGED
@@ -1,3 +1,3 @@
1
  {
2
- "[2024-03-21 09:16:00+09:00]": 20000
3
  }
 
1
  {
2
+ "[2024-04-29 12:09:00+09:00]": 20000
3
  }
special_tokens_map.json CHANGED
@@ -1,6 +1,6 @@
1
  {
2
  "additional_special_tokens": [
3
- "[2024-03-21 09:16:00+09:00]"
4
  ],
5
  "cls_token": "[CLS]",
6
  "mask_token": "[MASK]",
 
1
  {
2
  "additional_special_tokens": [
3
+ "[2024-04-29 12:09:00+09:00]"
4
  ],
5
  "cls_token": "[CLS]",
6
  "mask_token": "[MASK]",
tokenizer.json CHANGED
The diff for this file is too large to render. See raw diff
 
vocab.txt CHANGED
The diff for this file is too large to render. See raw diff