Hugging Face's logo

alasdairforsythe
/
tokenmonster

Model card Files Community
alasdairforsythe commited on
Commit
34a355a
•
1 Parent(s): f9b4f73

Upload 15 files

Browse files
Files changed (16) hide show
  1. .gitattributes +5 -0
  2. vocabs/code-16000-balanced-v1.vocab +0 -0
  3. vocabs/code-8000-balanced-v1.vocab +0 -0
  4. vocabs/code-8000-clean-v1.vocab +0 -0
  5. vocabs/english-100256-consistent-v1.vocab +3 -0
  6. vocabs/english-1024-balanced-v1.vocab +0 -0
  7. vocabs/english-32000-balanced-v1.vocab +0 -0
  8. vocabs/english-8000-balanced-v1.vocab +0 -0
  9. vocabs/englishcode-16000-clean-v1.vocab +0 -0
  10. vocabs/englishcode-40000-strict-v1.vocab +3 -0
  11. vocabs/englishcode-65536-consistent-v1.vocab +3 -0
  12. vocabs/englishcode-65536-unfiltered-v1.vocab +3 -0
  13. vocabs/englishcode-8000-balanced-v1.vocab +0 -0
  14. vocabs/fiction-2048-clean-v1.vocab +0 -0
  15. vocabs/fiction-65536-consistent-v1.vocab +3 -0
  16. vocabs/fiction-8000-consistent-v1.vocab +0 -0
.gitattributes CHANGED
@@ -94,3 +94,8 @@ vocabs/fiction-40000-unfiltered-v1.vocab filter=lfs diff=lfs merge=lfs -text
94
  vocabs/code-40000-strict-v1.vocab filter=lfs diff=lfs merge=lfs -text
95
  vocabs/code-65536-consistent-v1.vocab filter=lfs diff=lfs merge=lfs -text
96
  vocabs/fiction-40000-clean-v1.vocab filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
 
94
  vocabs/code-40000-strict-v1.vocab filter=lfs diff=lfs merge=lfs -text
95
  vocabs/code-65536-consistent-v1.vocab filter=lfs diff=lfs merge=lfs -text
96
  vocabs/fiction-40000-clean-v1.vocab filter=lfs diff=lfs merge=lfs -text
97
+ vocabs/english-100256-consistent-v1.vocab filter=lfs diff=lfs merge=lfs -text
98
+ vocabs/englishcode-40000-strict-v1.vocab filter=lfs diff=lfs merge=lfs -text
99
+ vocabs/englishcode-65536-consistent-v1.vocab filter=lfs diff=lfs merge=lfs -text
100
+ vocabs/englishcode-65536-unfiltered-v1.vocab filter=lfs diff=lfs merge=lfs -text
101
+ vocabs/fiction-65536-consistent-v1.vocab filter=lfs diff=lfs merge=lfs -text
vocabs/code-16000-balanced-v1.vocab ADDED
Binary file (432 kB). View file
 
vocabs/code-8000-balanced-v1.vocab ADDED
Binary file (210 kB). View file
 
vocabs/code-8000-clean-v1.vocab ADDED
Binary file (207 kB). View file
 
vocabs/english-100256-consistent-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:f37f99ffe224463dc39f2005e4f4edb632ce3111e908eaabf196dea9b19cef99
3
+ size 2727516
vocabs/english-1024-balanced-v1.vocab ADDED
Binary file (25.7 kB). View file
 
vocabs/english-32000-balanced-v1.vocab ADDED
Binary file (835 kB). View file
 
vocabs/english-8000-balanced-v1.vocab ADDED
Binary file (200 kB). View file
 
vocabs/englishcode-16000-clean-v1.vocab ADDED
Binary file (409 kB). View file
 
vocabs/englishcode-40000-strict-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:20055741f1a9eb3a128e9faf7811093334bbf97d02bc5bfbbe959ab360d0ec6e
3
+ size 1056367
vocabs/englishcode-65536-consistent-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:98bb0373dfb7e2689cce1b0c43a799c3452a99f31672effe6c3688b697fc0812
3
+ size 1764825
vocabs/englishcode-65536-unfiltered-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:a414e2294d0e05b3c9c95d324ab7e7c955ea2a2e250542693bd400314d9def5a
3
+ size 1787845
vocabs/englishcode-8000-balanced-v1.vocab ADDED
Binary file (200 kB). View file
 
vocabs/fiction-2048-clean-v1.vocab ADDED
Binary file (51.1 kB). View file
 
vocabs/fiction-65536-consistent-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1748055a3414f18661848e4f2224b7825bdf9a4bb2a6f9091dfca6c3ec65337b
3
+ size 1763846
vocabs/fiction-8000-consistent-v1.vocab ADDED
Binary file (199 kB). View file