tokenizer-arena / stats /compress_rate /bert_base_cased.en.json
xu-song's picture
update compress rate
988921c
raw
history blame
No virus
81 Bytes
{"vocab_size": 28996, "n_bytes": 1124813, "n_tokens": 288022, "n_chars": 1121360}