Spaces:

yhavinga
/

dutch-tokenizer-arena

Running

App Files Files Community

dutch-tokenizer-arena / stats

3 contributors

History: 4 commits

yhavinga's picture

Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX.

c78da21 7 months ago

README.md

0 Bytes

update compress rate 7 months ago
compress_rate.json

105 kB

Add Llama tokenizer creation for Dutch, English, Code, Markdown and TeX. 7 months ago