Hugging Face
Models
Datasets
Spaces
Posts
Docs
Solutions
Pricing
Log In
Sign Up
Spaces:
xu-song
/
tokenizer-arena
like
56
Running
App
Files
Files
Community
1
a173fe5
tokenizer-arena
/
vocab
/
gpt_neox_chinese_v1
/
to_v2
2 contributors
History:
2 commits
xu-song
update
d10ecd7
about 1 year ago
20B_tokenizer.1.append.json
2.75 MB
update
about 1 year ago
20B_tokenizer.1.insert.json
2.75 MB
update
about 1 year ago
20B_tokenizer.1.json
2.68 MB
update
about 1 year ago
20B_tokenizer.2.json
3.65 MB
update
about 1 year ago
20B_tokenizer.tmp.json
2.47 MB
update
about 1 year ago
README.md
21 Bytes
update
about 1 year ago
add_token_utils.py
6.23 kB
update
about 1 year ago
get_unused_id.py
8.56 kB
update
about 1 year ago
oov.add.txt
150 kB
update
about 1 year ago
oov.txt
893 kB
update
about 1 year ago
sort_test.py
182 Bytes
update
about 1 year ago
test2.py
1.44 kB
update
about 1 year ago
test_oov.py
2.34 kB
update
about 1 year ago
test_queue.py
285 Bytes
update
about 1 year ago
word_count.corpus.remove.jsonl
2.87 MB
update
about 1 year ago
word_count.corpus.sort_by_count.jsonl
6.8 MB
update
about 1 year ago
word_count.corpus.txt
631 kB
update
about 1 year ago