File size: 493 Bytes
57f1fd6 428b731 751936e 57f1fd6 428b731 2d550af 11379e2 f4973d4 11379e2 f4973d4 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
---
title: Tokenizer Arena
emoji: ⚡
colorFrom: red
colorTo: gray
sdk: gradio
sdk_version: 3.41.2
app_file: app.py
pinned: false
---
Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
## ss
## TODO
- 搜索栏
-
## 统计
## vocabsize
- 增大能提到压缩率，副作用是增大计算量和内存 （getting the most out of your tokenizer for pre-training and）
-
https://huggingface.co/spaces/yenniejun/tokenizers-languages |