Hugging Face's logo

alasdairforsythe
/
tokenmonster

Model card Files Community
alasdairforsythe commited on
Commit
1be91c1
•
1 Parent(s): 7dadf99

Upload 11 files

Browse files
Files changed (12) hide show
  1. .gitattributes +4 -0
  2. vocabs/code-1024-consistent-v1.vocab +0 -0
  3. vocabs/code-40000-clean-v1.vocab +3 -0
  4. vocabs/code-40000-unfiltered-v1.vocab +3 -0
  5. vocabs/code-8000-strict-v1.vocab +0 -0
  6. vocabs/englishcode-1024-clean-v1.vocab +0 -0
  7. vocabs/englishcode-32000-unfiltered-v1.vocab +0 -0
  8. vocabs/englishcode-40000-balanced-v1.vocab +3 -0
  9. vocabs/englishcode-4096-strict-v1.vocab +0 -0
  10. vocabs/fiction-2048-consistent-v1.vocab +0 -0
  11. vocabs/fiction-40000-balanced-v1.vocab +3 -0
  12. vocabs/fiction-8000-strict-v1.vocab +0 -0
.gitattributes CHANGED
@@ -109,3 +109,7 @@ vocabs/english-50256-strict-v1.vocab filter=lfs diff=lfs merge=lfs -text
109
  vocabs/englishcode-65536-strict-v1.vocab filter=lfs diff=lfs merge=lfs -text
110
  vocabs/fiction-65536-balanced-v1.vocab filter=lfs diff=lfs merge=lfs -text
111
  vocabs/fiction-40000-consistent-v1.vocab filter=lfs diff=lfs merge=lfs -text
 
 
 
 
 
109
  vocabs/englishcode-65536-strict-v1.vocab filter=lfs diff=lfs merge=lfs -text
110
  vocabs/fiction-65536-balanced-v1.vocab filter=lfs diff=lfs merge=lfs -text
111
  vocabs/fiction-40000-consistent-v1.vocab filter=lfs diff=lfs merge=lfs -text
112
+ vocabs/code-40000-clean-v1.vocab filter=lfs diff=lfs merge=lfs -text
113
+ vocabs/code-40000-unfiltered-v1.vocab filter=lfs diff=lfs merge=lfs -text
114
+ vocabs/englishcode-40000-balanced-v1.vocab filter=lfs diff=lfs merge=lfs -text
115
+ vocabs/fiction-40000-balanced-v1.vocab filter=lfs diff=lfs merge=lfs -text
vocabs/code-1024-consistent-v1.vocab ADDED
Binary file (25.6 kB). View file
 
vocabs/code-40000-clean-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ac5b3e10293e4f7de4f0bea9f3e870bbad24045e2a099ff7d868b5e9619a718d
3
+ size 1094257
vocabs/code-40000-unfiltered-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:0c1336bc3793060c44cfffbaef289af6c513356728b5f9a09ddab31c0767f424
3
+ size 1143542
vocabs/code-8000-strict-v1.vocab ADDED
Binary file (205 kB). View file
 
vocabs/englishcode-1024-clean-v1.vocab ADDED
Binary file (25.4 kB). View file
 
vocabs/englishcode-32000-unfiltered-v1.vocab ADDED
Binary file (853 kB). View file
 
vocabs/englishcode-40000-balanced-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d8e36514589a9e007c572958b1dbd4e18cb21143ed2cd8347c74dacf86892835
3
+ size 1056055
vocabs/englishcode-4096-strict-v1.vocab ADDED
Binary file (100 kB). View file
 
vocabs/fiction-2048-consistent-v1.vocab ADDED
Binary file (50.6 kB). View file
 
vocabs/fiction-40000-balanced-v1.vocab ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3d47d07fc0375179d84f71a8cf14b8120f8a41d898ae07649b3524af5a3f8cfb
3
+ size 1061196
vocabs/fiction-8000-strict-v1.vocab ADDED
Binary file (199 kB). View file