Hugging Face
Models
Datasets
Spaces
Posts
Docs
Solutions
Pricing
Log In
Sign Up
Spaces:
yhavinga
/
dutch-tokenizer-arena
like
5
Running
App
Files
Files
Community
main
dutch-tokenizer-arena
2 contributors
History:
79 commits
yhavinga
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
c78da21
29 days ago
css
add compression leaderboard
about 1 month ago
images
update
9 months ago
js
add grok mixtral
about 1 month ago
patcher
add compression leaderboard
about 1 month ago
stats
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
29 days ago
utils
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
29 days ago
vocab
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
29 days ago
.gitattributes
2.32 kB
add grok mixtral
about 1 month ago
.gitignore
171 Bytes
update
9 months ago
README.md
25.5 kB
Hack some Dutch tokenizers into it
5 months ago
app.py
565 Bytes
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
29 days ago
app_compression.py
4.75 kB
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
29 days ago
app_playground.py
9.81 kB
update
about 1 month ago
config.py
1.72 kB
Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.
29 days ago
examples.py
3.16 kB
add compression leaderboard
about 1 month ago
requirements.txt
96 Bytes
Pin protobuf dep
3 months ago
util.py
8.15 kB
Hack some Dutch tokenizers into it
5 months ago