Spaces:

xu-song
/

tokenizer-arena

Running

App Files Files Community

tokenizer-arena / vocab /roberta_chinese_clue /README.md

xu-song's picture

add more tokenizers

f4973d4 about 1 year ago

|

80 Bytes

删除了无用字符，词典更小。增加了少部分

词典大小

-