zirui3
/

llm-zh-tokenizer

Model card Files Files and versions Community

llm-zh-tokenizer / README.md

zirui3's picture

Update README.md

fe523d6 over 1 year ago

|

88 Bytes

metadata

license: cc-by-nc-4.0

Chinese tokenizer trained on Baike by using BPE algorithm