Spaces:

eson
/

tokenizer-arena

Running

App Files Files Community

tokenizer-arena / vocab /gpt2_chinese /README.md

eson's picture

update

751936e 10 months ago

|

raw history blame

No virus

237 Bytes

跟 bert-chinese词典一样

https://huggingface.co/ckiplab/gpt2-base-chinese https://huggingface.co/uer/gpt2-chinese-cluecorpussmall

  "tokenizer_class": "BertTokenizerFast",
  "vocab_size": 21128

ss

ss