matthias-wright commited on
Commit
5d5f94c
1 Parent(s): a30ac74

updated tokenizer files

Browse files
Files changed (2) hide show
  1. tokenizer/merges.txt +2 -2
  2. tokenizer/vocab.json +2 -2
tokenizer/merges.txt CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:2ee34673ec9e84786533ccaa8c41bbb209e7c889aca32a21f7654e09e3879cb7
3
- size 456519
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1ce1664773c50f3e0cc8842619a93edc4624525b728b188a9e0be33b7726adc5
3
+ size 456318
tokenizer/vocab.json CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:caeeb94ff0ee0729b311b35c8500d1a3b21bc01ee50ac52e928981ac3e72e615
3
- size 1042504
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:196139668be63f3b5d6574427317ae82f612a97c5d1cdaf36ed2256dbf636783
3
+ size 1042301