Models
Datasets
Spaces
Posts
Docs
Pricing
Log In
Sign Up

Spaces:

yhavinga
/

dutch-tokenizer-arena

Running

App Files Files Community

dutch-tokenizer-arena / tokenizer

2 contributors

History: 7 commits

xu-song's picture

add compress rate

814ee6b 6 months ago

chinese_sptokenizer_patch.py

27 Bytes

add compress rate 6 months ago
sptokenizer_patch.py

2.82 kB

add compress rate 6 months ago
tiktoken_patch.py

2.97 kB

add compress rate 6 months ago
tokenizer_patcher.py

211 Bytes

add compress rate 6 months ago