alvin commited on
Commit
eb84e5f
1 Parent(s): 83f2ae2

fixed mismatched vocab_size between model and tokenizer

Browse files
Files changed (3) hide show
  1. merges.txt +0 -8
  2. tokenizer.json +0 -0
  3. vocab.json +0 -0
merges.txt CHANGED
@@ -49995,11 +49995,3 @@ st ep
49995
  ĠKet ersediaan
49996
  ĠMon key
49997
  ĠSIPI LIS
49998
- L ondon
49999
- ĠS perma
50000
- res or
50001
- ĠMed ika
50002
- ĠPD KT
50003
- ĠTeb al
50004
- Ġmentor ing
50005
- Ġmenyebr angi
 
49995
  ĠKet ersediaan
49996
  ĠMon key
49997
  ĠSIPI LIS
 
 
 
 
 
 
 
 
tokenizer.json CHANGED
The diff for this file is too large to render. See raw diff
 
vocab.json CHANGED
The diff for this file is too large to render. See raw diff