dys_base_M16 / vocab.json
mprzibilla's picture
add tokenizer
5406dbb
raw
history blame
250 Bytes
{"h": 0, "b": 1, "r": 2, "x": 3, "p": 4, "t": 5, "j": 6, "o": 7, "m": 8, "k": 9, "v": 10, "n": 11, "s": 12, "w": 13, "y": 15, "g": 16, "c": 17, "f": 18, "e": 19, "q": 20, "u": 21, "l": 22, "i": 23, "a": 24, "d": 25, "|": 14, "[UNK]": 26, "[PAD]": 27}