Spaces:

xu-song
/

tokenizer-arena

Running

tokenizer-arena / utils /digit_util.py

add compress rate

814ee6b 7 months ago

50 Bytes

	"""

	qwen segments numbers by single digits.


	"""