magungh1 commited on
Commit
7c52a83
1 Parent(s): 3a415c6

add tokenizer

Browse files
Files changed (3) hide show
  1. merges.txt +0 -1
  2. tokenizer.json +0 -0
  3. vocab.json +0 -0
merges.txt CHANGED
@@ -49994,4 +49994,3 @@ st ep
49994
  ĠUm p
49995
  ĠKet ersediaan
49996
  ĠMon key
49997
- ĠSIPI LIS
 
49994
  ĠUm p
49995
  ĠKet ersediaan
49996
  ĠMon key
 
tokenizer.json CHANGED
The diff for this file is too large to render. See raw diff
 
vocab.json CHANGED
The diff for this file is too large to render. See raw diff