hezarai
/

whisper-small-fa

arxyzan commited on Feb 5

Commit

4916574

•

1 Parent(s): 021ecaa

Hezar: Upload tokenizer and config

Files changed (1) hide show

preprocessor/tokenizer.json CHANGED Viewed

@@ -1119,6 +1119,7 @@
     "continuing_subword_prefix": "",
     "end_of_word_suffix": "",
     "fuse_unk": false,
     "vocab": {
       "!": 0,
       "\"": 1,

     "continuing_subword_prefix": "",
     "end_of_word_suffix": "",
     "fuse_unk": false,
+    "byte_fallback": false,
     "vocab": {
       "!": 0,
       "\"": 1,