Models
Datasets
Spaces
Posts
Docs
Pricing
Log In
Sign Up

Spaces:

xu-song
/

tokenizer-arena

Running

App Files Files Community

tokenizer-arena / vocab /gpt_neox_chinese_v2

2 contributors

History: 1 commit

xu-song's picture

update

751936e about 1 year ago

20B_tokenizer_chinese.v2.json

3.65 MB

update about 1 year ago
README.md

0 Bytes

update about 1 year ago
build_tokenizer_chinese.py

1.53 kB

update about 1 year ago
merge_token.txt

27 Bytes

update about 1 year ago
test_error.py

555 Bytes

update about 1 year ago
test_tokenizer.py

1.91 kB

update about 1 year ago