wietsedv Edwin Rijgersberg commited on
Commit
25c8b2f
1 Parent(s): 0837ce3

Fix mixup of `<pad>` and `<s>` tokens in vocab (#1)

Browse files

- Fix mixup of `<pad>` and `<s>` tokens in vocab (93d00ba05016dd3dbebb49609770a3b672a9126f)


Co-authored-by: Edwin Rijgersberg <erijgersberg-nfi@users.noreply.huggingface.co>

Files changed (1) hide show
  1. vocab.json +1 -1
vocab.json CHANGED
@@ -1 +1 @@
1
- {"<s>": 0, "<pad>": 1, "</s>": 2, "<unk>": 3, "|": 4, "A": 5, "B": 6, "C": 7, "D": 8, "E": 9, "F": 10, "G": 11, "H": 12, "I": 13, "J": 14, "K": 15, "L": 16, "M": 17, "N": 18, "O": 19, "P": 20, "Q": 21, "R": 22, "S": 23, "T": 24, "U": 25, "V": 26, "W": 27, "X": 28, "Y": 29, "Z": 30, "È": 31, "É": 32, "Ë": 33, "?": 34, "'": 35, "-": 36}
1
+ {"<pad>": 0, "<s>": 1, "</s>": 2, "<unk>": 3, "|": 4, "A": 5, "B": 6, "C": 7, "D": 8, "E": 9, "F": 10, "G": 11, "H": 12, "I": 13, "J": 14, "K": 15, "L": 16, "M": 17, "N": 18, "O": 19, "P": 20, "Q": 21, "R": 22, "S": 23, "T": 24, "U": 25, "V": 26, "W": 27, "X": 28, "Y": 29, "Z": 30, "È": 31, "É": 32, "Ë": 33, "?": 34, "'": 35, "-": 36}