Spaces:

eson
/

tokenizer-arena

Running

tokenizer-arena / utils /compress_rate_util.py

add more tokenizer

c75633b 6 months ago

No virus

83 Bytes

	"""


	中文数据：clue superclue
	英文数据：glue cnn_dailymail gigaword

	"""