Models
Datasets
Spaces
Posts
Docs
Pricing
Log In
Sign Up

Spaces:

xu-song
/

tokenizer-arena

Running

App Files Files Community

tokenizer-arena / vocab /gpt_neox_chinese_v1

2 contributors

History: 3 commits

xu-song's picture

update

7156337 about 1 year ago

to_v2
update about 1 year ago
tokenizer
update about 1 year ago
20B_tokenizer.tmp.json

2.47 MB

update about 1 year ago
20B_tokenizer_chinese.json

2.75 MB

update about 1 year ago
20B_tokenizer_chinese.mock.json

2.15 MB

update about 1 year ago
README.md

1.09 kB

update about 1 year ago
__init__.py

375 Bytes

update about 1 year ago
build_tokenizer_chinese.py

1.81 kB

update about 1 year ago
build_tokenizer_chinese_2.py

1.52 kB

update about 1 year ago
mock.py

426 Bytes

update about 1 year ago
test_tokenizer.py

1.77 kB

update about 1 year ago