Commit
•
ec6488d
1
Parent(s):
172e503
add tokenizer
Browse files- vocab.json +1 -1
vocab.json
CHANGED
@@ -1 +1 @@
|
|
1 |
-
{"i": 0, "
|
|
|
1 |
+
{"i": 0, "k": 1, "o": 2, "r": 3, "h": 4, "c": 5, "t": 6, "l": 7, "n": 8, "m": 9, "s": 10, "p": 11, "f": 12, "a": 13, "d": 14, "b": 15, "e": 16, " ": 17, "|": 17, "UNK": 19, "[PAD]": 20}
|